2020-11-16T21:10:08.319 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-16T21:10:08.321 DEBUG:teuthology.run:Teuthology command: teuthology --lock --archive /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916 --name pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi --block --verbose --owner scheduled_pdonnell@teuthology --description multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc} -- /tmp/teuthology-worker.znqg0qt5.tmp 2020-11-16T21:10:08.362 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T21:10:08.426 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916 branch: wip-pdonnell-testing-20201116.174825 description: multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc} email: pdonnell@redhat.com first_in_suite: false job_id: '5631916' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-pdonnell-testing-20201116.174825 ceph: cephfs: max_mds: 9 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 fuse default permissions: false rados mon op timeout: 15m rados osd op timeout: 15m global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Scrub error on inode log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6959059e6aae2fb5241382a4eeeca62785867604 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 6959059e6aae2fb5241382a4eeeca62785867604 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20201116.174825 sha1: 6959059e6aae2fb5241382a4eeeca62785867604 owner: scheduled_pdonnell@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - mds.a - mds.b - mds.c - mds.d - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mgr.x - mds.e - mds.f - mds.g - mds.h - mds.i - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - client.1 sha1: 6959059e6aae2fb5241382a4eeeca62785867604 sleep_before_teardown: 0 suite: multimds suite_branch: wip-pdonnell-testing-20201116.174825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 6959059e6aae2fb5241382a4eeeca62785867604 tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - check-counter: counters: mds: - mds.exported - mds.imported - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology_branch: master timestamp: 2020-11-16_20:59:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.24832 2020-11-16T21:10:08.427 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa; will attempt to use it 2020-11-16T21:10:08.427 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa/tasks 2020-11-16T21:10:08.428 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-16T21:10:08.429 INFO:teuthology.task.internal:Checking packages... 2020-11-16T21:10:08.448 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '6959059e6aae2fb5241382a4eeeca62785867604' 2020-11-16T21:10:08.448 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-16T21:10:08.449 INFO:teuthology.packaging:ref: None 2020-11-16T21:10:08.449 INFO:teuthology.packaging:tag: None 2020-11-16T21:10:08.449 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20201116.174825 2020-11-16T21:10:08.450 INFO:teuthology.packaging:sha1: 6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T21:10:08.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-pdonnell-testing-20201116.174825 2020-11-16T21:10:08.618 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7226-g6959059e-1bionic 2020-11-16T21:10:08.618 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-16T21:10:08.640 INFO:teuthology.task.internal:no buildpackages task found 2020-11-16T21:10:08.640 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-16T21:10:08.655 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-16T21:10:08.655 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T21:10:08.714 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:10:18.733 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:10:28.751 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:10:38.765 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:10:48.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:10:58.801 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:08.818 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:18.837 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:28.855 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:38.876 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:48.894 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:11:58.910 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:12:08.928 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:12:18.941 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:12:28.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:12:38.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:12:48.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:12:59.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:09.025 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:19.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:29.052 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:39.064 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:49.081 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:13:59.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:14:09.118 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:14:19.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:14:29.145 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:14:39.164 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:14:49.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:14:59.203 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:09.221 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:19.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:29.260 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:39.278 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:49.297 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:15:59.315 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:09.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:19.347 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:29.366 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:39.382 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:49.392 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:16:59.406 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:09.424 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:19.436 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:29.454 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:39.474 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:49.494 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:17:59.512 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:09.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:19.542 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:29.560 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:39.569 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:49.588 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:18:59.603 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:09.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:19.624 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:29.644 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:39.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:49.679 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:19:59.694 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:09.710 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:19.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:29.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:39.757 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:49.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:20:59.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:21:09.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:21:19.834 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:21:29.850 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:21:39.868 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:21:49.878 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:21:59.894 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:09.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:19.929 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:29.946 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:39.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:49.983 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:22:59.998 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:23:10.016 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:23:20.030 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:23:30.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:23:40.066 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:23:50.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:00.093 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:10.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:20.126 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:30.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:40.154 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:24:50.170 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:00.186 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:10.203 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:20.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:30.238 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:40.257 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:25:50.275 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:26:00.290 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:26:10.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:26:20.329 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:26:30.347 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:26:40.365 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:26:50.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:00.403 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:10.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:20.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:30.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:40.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:27:50.487 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:00.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:10.516 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:20.538 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:30.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:40.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:28:50.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:00.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:10.624 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:20.639 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:30.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:40.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:29:50.710 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:00.727 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:10.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:20.785 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:30.808 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:40.823 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:30:50.867 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:31:00.889 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:31:10.904 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:31:20.923 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:31:30.939 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:31:40.956 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:31:50.974 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:00.985 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:11.007 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:21.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:31.041 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:41.053 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:32:51.070 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:01.089 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:11.107 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:21.118 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:31.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:41.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:33:51.162 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:01.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:11.195 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:21.212 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:31.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:41.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:34:51.262 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:01.278 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:11.297 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:21.310 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:31.329 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:41.348 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:35:51.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:01.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:11.402 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:21.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:31.432 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:41.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:36:51.474 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:01.491 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:11.506 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:21.518 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:31.536 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:41.557 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:37:51.568 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:38:01.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:38:11.596 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:38:21.616 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:38:31.632 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:38:41.651 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:38:51.669 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:01.683 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:11.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:21.720 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:31.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:41.754 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:39:51.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:01.791 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:11.810 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:21.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:31.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:41.856 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:40:51.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:01.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:11.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:21.919 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:31.938 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:41.947 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:41:51.965 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:01.983 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:11.998 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:22.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:32.038 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:42.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:42:52.071 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:43:02.088 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:43:12.103 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:43:22.121 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:43:32.140 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:43:42.150 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:43:52.168 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:02.187 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:12.205 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:22.218 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:32.236 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:42.255 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:44:52.275 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:02.291 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:12.310 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:22.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:32.340 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:42.358 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:45:52.377 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:02.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:12.407 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:22.423 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:32.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:42.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:46:52.472 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:02.482 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:12.492 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:22.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:32.532 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:42.549 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:47:52.567 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:02.587 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:12.601 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:22.621 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:32.639 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:42.658 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:48:52.675 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:02.695 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:12.708 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:22.727 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:32.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:42.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:49:52.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:02.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:12.798 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:22.818 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:32.837 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:42.853 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:50:52.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:02.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:12.906 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:22.923 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:32.934 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:42.952 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:51:52.963 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:02.978 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:12.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:23.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:33.024 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:43.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:52:53.061 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:03.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:13.092 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:23.102 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:33.121 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:43.138 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:53:53.156 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:03.175 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:13.195 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:23.213 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:33.232 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:43.250 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:54:53.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:03.281 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:13.300 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:23.322 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:33.340 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:43.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:53.377 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:03.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:13.414 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:23.433 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:33.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:43.470 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:53.489 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:57:03.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:57:13.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:23.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:33.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:43.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:53.589 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:03.603 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:13.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:23.627 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:33.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:43.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:53.675 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:59:03.691 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:59:13.710 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:59:23.728 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:59:33.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:59:43.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:59:53.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T22:00:03.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T22:00:13.809 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T22:00:23.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:00:33.843 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:00:43.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:00:53.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:03.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:13.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:23.926 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:33.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:43.961 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:01:53.979 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:03.994 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:14.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:24.028 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:34.045 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:44.061 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:02:54.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:04.093 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:14.107 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:24.125 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:34.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:44.149 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:03:54.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:04.182 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:14.198 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:24.207 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:34.225 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:44.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:04:54.258 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:04.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:14.285 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:24.301 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:34.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:44.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:05:54.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:04.368 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:14.386 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:24.403 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:34.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:44.437 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:06:54.455 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:04.473 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:14.489 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:24.506 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:34.522 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:44.539 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:07:54.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:04.573 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:14.591 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:24.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:34.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:44.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:08:54.658 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:04.676 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:14.693 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:24.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:34.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:44.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:09:54.767 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T22:10:04.783 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 3, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916'} 2020-11-16T22:10:04.808 DEBUG:teuthology.lock.ops:locked ubuntu@smithi181.front.sepia.ceph.com, ubuntu@smithi175.front.sepia.ceph.com, ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:10:04.892 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea746f1d08>, signals=[15]) 2020-11-16T22:10:05.103 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of ubuntu 18.04 2020-11-16T22:10:05.105 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of ubuntu 18.04 2020-11-16T22:10:05.107 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of ubuntu 18.04 2020-11-16T22:10:05.377 INFO:teuthology.orchestra.console:Power off smithi106 2020-11-16T22:10:05.378 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:05.394 INFO:teuthology.orchestra.console:Power off smithi181 2020-11-16T22:10:05.395 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:05.409 INFO:teuthology.orchestra.console:Power off smithi175 2020-11-16T22:10:05.409 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:05.416 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:05.426 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:05.436 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:09.448 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:09.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:09.467 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:13.562 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:13.573 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:13.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:13.774 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2020-11-16T22:10:13.784 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2020-11-16T22:10:13.875 INFO:teuthology.orchestra.console:Power on smithi181 2020-11-16T22:10:13.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:13.885 INFO:teuthology.orchestra.console:Power on smithi106 2020-11-16T22:10:13.885 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:13.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:13.905 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:17.696 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:17.907 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2020-11-16T22:10:17.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:17.926 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:18.008 INFO:teuthology.orchestra.console:Power on smithi175 2020-11-16T22:10:18.009 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:18.029 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:22.026 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:22.039 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:22.049 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:22.277 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2020-11-16T22:10:22.334 INFO:teuthology.orchestra.console:Power on for smithi181 completed 2020-11-16T22:10:22.379 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2020-11-16T22:10:22.435 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2020-11-16T22:10:26.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:26.374 INFO:teuthology.orchestra.console:Power on for smithi175 completed 2020-11-16T22:10:26.475 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish 2020-11-16T22:13:08.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:23.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:27.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:32.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:41.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:47.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:50.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:51.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:56.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:59.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:00.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:05.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:08.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:10.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:14.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:19.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:28.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:14.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:15.024 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:15:15.026 INFO:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:15.423 INFO:teuthology.provision.fog.smithi181:Node is ready 2020-11-16T22:15:15.423 INFO:teuthology.orchestra.run.smithi181:> hostname 2020-11-16T22:15:15.472 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2020-11-16T22:15:15.472 INFO:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:15.521 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2020-11-16T22:15:15.521 INFO:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2020-11-16T22:15:15.620 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2020-11-16T22:15:15.635 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2020-11-16T22:15:15.694 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2020-11-16T22:15:15.747 INFO:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2020-11-16T22:15:15.803 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2020-11-16T22:15:15.803 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:20.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:21.071 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:15:21.071 INFO:teuthology.orchestra.run.smithi106:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:21.450 INFO:teuthology.provision.fog.smithi106:Node is ready 2020-11-16T22:15:21.451 INFO:teuthology.orchestra.run.smithi106:> hostname 2020-11-16T22:15:21.496 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2020-11-16T22:15:21.497 INFO:teuthology.orchestra.run.smithi106:> grep smithi106.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:21.545 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2020-11-16T22:15:21.545 INFO:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2020-11-16T22:15:21.664 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hosts 2020-11-16T22:15:21.679 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/172.21.15.106/172.21.15.106/g' /etc/hosts 2020-11-16T22:15:21.738 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hostname 2020-11-16T22:15:21.792 INFO:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2020-11-16T22:15:21.847 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2020-11-16T22:15:21.848 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:34.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:34.241 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:15:34.241 INFO:teuthology.orchestra.run.smithi175:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:34.627 INFO:teuthology.provision.fog.smithi175:Node is ready 2020-11-16T22:15:34.627 INFO:teuthology.orchestra.run.smithi175:> hostname 2020-11-16T22:15:34.675 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175.front.sepia.ceph.com 2020-11-16T22:15:34.675 INFO:teuthology.orchestra.run.smithi175:> grep smithi175.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:34.727 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175 smithi175.front.sepia.ceph.com smithi175 2020-11-16T22:15:34.727 INFO:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2020-11-16T22:15:34.885 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi175.front.sepia.ceph.com/smithi175/g' /etc/hosts 2020-11-16T22:15:34.898 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/172.21.15.175/172.21.15.175/g' /etc/hosts 2020-11-16T22:15:34.956 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi175.front.sepia.ceph.com/smithi175/g' /etc/hostname 2020-11-16T22:15:35.010 INFO:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2020-11-16T22:15:35.065 INFO:teuthology.provision.fog.smithi175:Deploy complete! 2020-11-16T22:15:35.066 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:47.641 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2020-11-16T22:15:47.642 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:47.661 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com 2020-11-16T22:15:47.661 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:47.674 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2020-11-16T22:15:47.675 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:47.688 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:47.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:47.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:47.761 INFO:teuthology.orchestra.run.smithi181:> true 2020-11-16T22:15:47.998 INFO:teuthology.orchestra.run.smithi181:> uname -m 2020-11-16T22:15:48.051 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2020-11-16T22:15:48.051 INFO:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2020-11-16T22:15:48.098 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu" 2020-11-16T22:15:48.098 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:15:48.098 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="18.04" 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:15:48.099 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:15:48.100 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:15:48.100 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=bionic 2020-11-16T22:15:48.100 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:15:48.101 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2020-11-16T22:15:48.117 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:48.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:48.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:48.190 INFO:teuthology.orchestra.run.smithi175:> true 2020-11-16T22:15:48.438 INFO:teuthology.orchestra.run.smithi175:> uname -m 2020-11-16T22:15:48.486 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2020-11-16T22:15:48.486 INFO:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2020-11-16T22:15:48.534 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Ubuntu" 2020-11-16T22:15:48.535 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:15:48.535 INFO:teuthology.orchestra.run.smithi175.stdout:ID=ubuntu 2020-11-16T22:15:48.535 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE=debian 2020-11-16T22:15:48.535 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:15:48.535 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="18.04" 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_CODENAME=bionic 2020-11-16T22:15:48.536 INFO:teuthology.orchestra.run.smithi175.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:15:48.537 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2020-11-16T22:15:48.559 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:48.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:48.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:48.635 INFO:teuthology.orchestra.run.smithi106:> true 2020-11-16T22:15:48.870 INFO:teuthology.orchestra.run.smithi106:> uname -m 2020-11-16T22:15:48.916 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2020-11-16T22:15:48.916 INFO:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2020-11-16T22:15:48.964 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Ubuntu" 2020-11-16T22:15:48.964 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:15:48.964 INFO:teuthology.orchestra.run.smithi106.stdout:ID=ubuntu 2020-11-16T22:15:48.964 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE=debian 2020-11-16T22:15:48.964 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:15:48.965 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="18.04" 2020-11-16T22:15:48.965 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:15:48.965 INFO:teuthology.orchestra.run.smithi106.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:15:48.965 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:15:48.966 INFO:teuthology.orchestra.run.smithi106.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:15:48.966 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_CODENAME=bionic 2020-11-16T22:15:48.966 INFO:teuthology.orchestra.run.smithi106.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:15:48.966 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2020-11-16T22:15:48.990 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-11-16T22:15:49.040 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC86IesroZTKWHbZ7ijHF7SSE6avm1jwyfyXzCcYdLVs7UIoPJnnAHZF1D0mCPr0yUxOhrmrgKZOmQDtA6Sy0zEcPukPNELacHXUlUCuL4OMhEuInKehZ+KwN3WWcQhFAvPfPdTLTep4vsk68NJwNn877kG/mq05EyCghjczKNUxCLhl41B38MTtwMH3Irg4LXGrQt3qFCgsBjQ8vr1fbybAv05W8NBQrlTDVNQKpqnHAjpw5uo+z+ionhCJJ7yYEXftceswHltbOGZQgwNIs64Rqg/L2m4Hk1aSyzsG5mqPNiTOegGoHmnJwdkhA28mihJlNxZbrppKUPA8Xl3efmj smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVOktKoMfu6uFtTEGcR66WDbENC3HFqNGgaHwkOsjpUUNHeDw4Dc58LnzQsfkIqFzXbJFzhs/GVsN4j2vPfR5K+0jfr1BoBjISJ0eyH7cKaGlx/z2Nm0LG+agurz8Db9rmcZkDC21wdDHxquGEq0k8JcpE+RQ0Rm+FSgtLgYTm8sWbbqtF1py7mJPxhg7Y4FWJfd76QosPpfe9yksBqqt4M8VoA9rhzTbWAQSRG27l6hhq5um3Z0EcbF7qL+Ojhw8A4SJpkBFegY2/H3hZpY7WZh0PJqRpEOjfJDDvqtnmUg1tQAI96gV5wpmB9XRT+25tTcgSgMj79DXLJfbAdnBX smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqruCyaTHoXvcJm3J4HRvwr51yZGpfwT7iQpOdzyr7CmAhcGqX1mfOEbBiSVxVFjwU6M9bfkREBEro9J2pDH4XaCsAzsq4wN4ZbVNy3is7FDfyTR6EDxzXd2ioTnNzkYok9bvbCg6U4t+5iQfJSuqM+jhZJ4iCB1FsbM6GWHfAe266oPF+tQOpwKcsZjxgtyhzTSQx+Rn1Uy5s/gji4TKAXPkVigaKwOAk1xH9VUo7jVMG5Hbdjqrs0SDYQnKfxhZG8wrOqp3a6Smy1EVJwTMbYiOfovK3s4fGPAj3MERC8i9Vv2KYucdmehyUqlFsXAuFfIZjjdUERE6kJXhm5xep 2020-11-16T22:15:49.040 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T22:15:49.099 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-16T22:15:49.105 INFO:teuthology.task.internal:Saving configuration 2020-11-16T22:15:49.122 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-16T22:15:49.131 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-16T22:15:49.195 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:04.804220', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJGN1Gcy3sEnGRzSUtm35qU2t0J62hWrqIfJ+4c5j13g', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi181.front.sepia.ceph.com'} 2020-11-16T22:15:49.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:04.804239', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKI4cEr6cy7lrLX9ohVSaUY/CFM8fLNht3lQTuwHpoeF', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi175.front.sepia.ceph.com'} 2020-11-16T22:15:49.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:04.804253', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICsL/okjL+YRYuQKQVOkziuK7mC0VYIcEpPZLjT5VfK8', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi106.front.sepia.ceph.com'} 2020-11-16T22:15:49.226 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-16T22:15:49.236 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'mds.d', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2020-11-16T22:15:49.236 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'mds.e', 'mds.f', 'mds.g', 'mds.h', 'mds.i', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-11-16T22:15:49.236 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['client.0', 'client.1'] 2020-11-16T22:15:49.236 INFO:teuthology.run_tasks:Running task console_log... 2020-11-16T22:16:14.141 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea7466a598>, signals=[15]) 2020-11-16T22:16:14.142 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-16T22:16:14.151 INFO:teuthology.task.internal:Opening connections... 2020-11-16T22:16:14.151 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:16:14.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:16:14.231 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:16:14.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:16:14.304 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:16:14.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:16:14.392 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-16T22:16:14.407 INFO:teuthology.orchestra.run.smithi181:> uname -m 2020-11-16T22:16:14.437 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2020-11-16T22:16:14.438 INFO:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2020-11-16T22:16:14.486 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu" 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="18.04" 2020-11-16T22:16:14.487 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:16:14.488 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:16:14.488 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:16:14.488 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:16:14.488 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=bionic 2020-11-16T22:16:14.488 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:16:14.489 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2020-11-16T22:16:14.504 INFO:teuthology.orchestra.run.smithi175:> uname -m 2020-11-16T22:16:14.510 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2020-11-16T22:16:14.511 INFO:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2020-11-16T22:16:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Ubuntu" 2020-11-16T22:16:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:16:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:ID=ubuntu 2020-11-16T22:16:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE=debian 2020-11-16T22:16:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:16:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="18.04" 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_CODENAME=bionic 2020-11-16T22:16:14.559 INFO:teuthology.orchestra.run.smithi175.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:16:14.560 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2020-11-16T22:16:14.576 INFO:teuthology.orchestra.run.smithi106:> uname -m 2020-11-16T22:16:14.592 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2020-11-16T22:16:14.592 INFO:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2020-11-16T22:16:14.639 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Ubuntu" 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:ID=ubuntu 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE=debian 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="18.04" 2020-11-16T22:16:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:16:14.641 INFO:teuthology.orchestra.run.smithi106.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:16:14.641 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:16:14.641 INFO:teuthology.orchestra.run.smithi106.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:16:14.641 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_CODENAME=bionic 2020-11-16T22:16:14.641 INFO:teuthology.orchestra.run.smithi106.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:16:14.641 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2020-11-16T22:16:14.656 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-16T22:16:14.702 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-16T22:16:14.710 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-16T22:16:14.710 INFO:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:16:14.713 INFO:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:16:14.715 INFO:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:16:14.724 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-16T22:16:14.733 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-16T22:16:14.733 INFO:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:16:14.760 INFO:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:16:14.762 INFO:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:16:14.789 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-16T22:16:14.845 INFO:teuthology.run_tasks:Running task kernel... 2020-11-16T22:16:14.862 DEBUG:teuthology.task.kernel:normalized config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}} 2020-11-16T22:16:14.863 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2020-11-16T22:16:14.863 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2020-11-16T22:16:14.917 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2020-11-16T22:16:14.922 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:16:14.922 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=testing 2020-11-16T22:16:15.278 DEBUG:teuthology.task.kernel:sha1 for client.0 is ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:16:15.279 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9"... 2020-11-16T22:16:15.279 INFO:teuthology.orchestra.run.smithi106:> uname -r 2020-11-16T22:16:15.285 INFO:teuthology.orchestra.run.smithi106.stdout:4.15.0-123-generic 2020-11-16T22:16:15.286 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-123-generic vs ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:16:15.286 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2020-11-16T22:16:15.287 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2020-11-16T22:16:15.287 INFO:teuthology.orchestra.run.smithi106:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2020-11-16T22:16:15.331 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2020-11-16T22:16:15.436 INFO:teuthology.orchestra.run.smithi106.stdout:Initialized empty Git repository in /lib/firmware/updates/.git/ 2020-11-16T22:16:15.438 INFO:teuthology.orchestra.run.smithi106:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2020-11-16T22:16:15.521 INFO:teuthology.orchestra.run.smithi106:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2020-11-16T22:16:15.578 INFO:teuthology.orchestra.run.smithi106:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master 2020-11-16T22:16:25.770 INFO:teuthology.orchestra.run.smithi106.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2020-11-16T22:16:25.771 INFO:teuthology.orchestra.run.smithi106.stderr: * [new branch] main -> origin/main 2020-11-16T22:16:25.771 INFO:teuthology.orchestra.run.smithi106.stderr: * [new branch] master -> origin/master 2020-11-16T22:16:25.795 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190312 -> 20190312 2020-11-16T22:16:25.795 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190416 -> 20190416 2020-11-16T22:16:25.795 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190514 -> 20190514 2020-11-16T22:16:25.795 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190618 -> 20190618 2020-11-16T22:16:25.795 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190717 -> 20190717 2020-11-16T22:16:25.796 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190815 -> 20190815 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20190923 -> 20190923 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20191022 -> 20191022 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20191215 -> 20191215 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200122 -> 20200122 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200316 -> 20200316 2020-11-16T22:16:25.797 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200421 -> 20200421 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200519 -> 20200519 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200619 -> 20200619 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200721 -> 20200721 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200817 -> 20200817 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20200918 -> 20200918 2020-11-16T22:16:25.798 INFO:teuthology.orchestra.run.smithi106.stderr: * [new tag] 20201022 -> 20201022 2020-11-16T22:16:27.924 INFO:teuthology.orchestra.run.smithi106.stderr:Checking out files: 53% (1247/2348) Checking out files: 54% (1268/2348) Checking out files: 55% (1292/2348) Checking out files: 56% (1315/2348) Checking out files: 57% (1339/2348) Checking out files: 58% (1362/2348) Checking out files: 59% (1386/2348) Checking out files: 60% (1409/2348) Checking out files: 61% (1433/2348) Checking out files: 62% (1456/2348) Checking out files: 63% (1480/2348) Checking out files: 64% (1503/2348) Checking out files: 65% (1527/2348) Checking out files: 66% (1550/2348) Checking out files: 67% (1574/2348) Checking out files: 68% (1597/2348) Checking out files: 69% (1621/2348) Checking out files: 70% (1644/2348) Checking out files: 71% (1668/2348) Checking out files: 72% (1691/2348) Checking out files: 73% (1715/2348) Checking out files: 74% (1738/2348) Checking out files: 75% (1761/2348) Checking out files: 76% (1785/2348) Checking out files: 77% (1808/2348) Checking out files: 78% (1832/2348) Checking out files: 79% (1855/2348) Checking out files: 80% (1879/2348) Checking out files: 81% (1902/2348) Checking out files: 82% (1926/2348) Checking out files: 83% (1949/2348) Checking out files: 84% (1973/2348) Checking out files: 85% (1996/2348) Checking out files: 85% (2012/2348) Checking out files: 86% (2020/2348) Checking out files: 87% (2043/2348) Checking out files: 88% (2067/2348) Checking out files: 89% (2090/2348) Checking out files: 90% (2114/2348) Checking out files: 91% (2137/2348) Checking out files: 92% (2161/2348) Checking out files: 93% (2184/2348) Checking out files: 94% (2208/2348) Checking out files: 95% (2231/2348) Checking out files: 96% (2255/2348) Checking out files: 97% (2278/2348) Checking out files: 98% (2302/2348) Checking out files: 99% (2325/2348) Checking out files: 100% (2348/2348) Checking out files: 100% (2348/2348), done. 2020-11-16T22:16:27.927 INFO:teuthology.orchestra.run.smithi106.stdout:HEAD is now at 2ea8667 rtlwifi: v88.2 firmware files for RTL8192CU 2020-11-16T22:16:27.936 INFO:teuthology.task.kernel:Downloading kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-16T22:16:27.936 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2020-11-16T22:16:28.025 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2020-11-16T22:16:28.030 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:16:28.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:16:28.203 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:16:28.203 INFO:teuthology.orchestra.run.smithi106:> rm -f /tmp/linux-image.deb && echo linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ --input-file=- 2020-11-16T22:16:28.205 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi106.front.sepia.ceph.com to complete... 2020-11-16T22:16:33.114 INFO:teuthology.orchestra.run.smithi106.stderr:2020-11-16 22:16:33 URL:https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb [33925808/33925808] -> "/tmp/linux-image.deb" [1] 2020-11-16T22:16:33.115 INFO:teuthology.orchestra.run.smithi106.stderr:FINISHED --2020-11-16 22:16:33-- 2020-11-16T22:16:33.115 INFO:teuthology.orchestra.run.smithi106.stderr:Total wall clock time: 4.9s 2020-11-16T22:16:33.115 INFO:teuthology.orchestra.run.smithi106.stderr:Downloaded: 1 files, 32M in 4.5s (7.17 MB/s) 2020-11-16T22:16:33.116 INFO:teuthology.task.kernel:Installing kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-16T22:16:33.117 INFO:teuthology.orchestra.run.smithi106:> sudo dpkg -i /tmp/linux-image.deb 2020-11-16T22:16:33.423 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-image-5.10.0-rc2-ceph-gac4f7b07f403. 2020-11-16T22:16:33.462 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:16:33.464 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack /tmp/linux-image.deb ... 2020-11-16T22:16:33.552 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-16T22:16:36.535 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi106.stderr:update-initramfs: Generating /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:53.963 INFO:teuthology.orchestra.run.smithi106.stderr:Sourcing file `/etc/default/grub' 2020-11-16T22:16:53.989 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2020-11-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:54.318 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:54.645 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T22:16:54.679 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T22:16:56.392 INFO:teuthology.orchestra.run.smithi106.stderr:done 2020-11-16T22:16:56.548 INFO:teuthology.orchestra.run.smithi106:> dpkg-deb -c /tmp/linux-image.deb 2020-11-16T22:16:56.557 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./ 2020-11-16T22:16:56.557 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./boot/ 2020-11-16T22:16:56.557 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 5164573 2020-11-13 14:15 ./boot/System.map-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:56.620 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 153248 2020-11-13 14:15 ./boot/config-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:56.622 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9847776 2020-11-13 14:15 ./boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:16:56.765 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/ 2020-11-16T22:16:56.765 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/ 2020-11-16T22:16:56.765 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postinst.d/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postrm.d/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/preinst.d/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/prerm.d/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/ 2020-11-16T22:16:56.766 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:16:56.767 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/ 2020-11-16T22:16:56.767 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/ 2020-11-16T22:16:56.767 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/ 2020-11-16T22:16:56.767 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ 2020-11-16T22:16:56.767 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 488985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/aesni-intel.ko 2020-11-16T22:16:56.768 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/blowfish-x86_64.ko 2020-11-16T22:16:56.768 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2020-11-16T22:16:56.768 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2020-11-16T22:16:56.769 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 86201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-x86_64.ko 2020-11-16T22:16:56.769 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 98425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2020-11-16T22:16:56.770 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 143041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2020-11-16T22:16:56.770 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crc32-pclmul.ko 2020-11-16T22:16:56.771 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2020-11-16T22:16:56.771 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2020-11-16T22:16:56.771 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2020-11-16T22:16:56.772 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/glue_helper.ko 2020-11-16T22:16:56.772 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2020-11-16T22:16:56.772 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx2.ko 2020-11-16T22:16:56.773 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2020-11-16T22:16:56.774 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha1-ssse3.ko 2020-11-16T22:16:56.774 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha256-ssse3.ko 2020-11-16T22:16:56.775 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha512-ssse3.ko 2020-11-16T22:16:56.775 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2020-11-16T22:16:56.776 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2020-11-16T22:16:56.776 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64.ko 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/ 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/ 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/ 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpuid.ko 2020-11-16T22:16:56.777 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/msr.ko 2020-11-16T22:16:56.778 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/ 2020-11-16T22:16:56.778 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 247537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-amd.ko 2020-11-16T22:16:56.780 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 826857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-intel.ko 2020-11-16T22:16:56.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1834553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm.ko 2020-11-16T22:16:56.811 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/ 2020-11-16T22:16:56.811 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 135385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/oprofile.ko 2020-11-16T22:16:56.813 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ 2020-11-16T22:16:56.813 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/af_alg.ko 2020-11-16T22:16:56.813 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_hash.ko 2020-11-16T22:16:56.814 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_skcipher.ko 2020-11-16T22:16:56.814 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ansi_cprng.ko 2020-11-16T22:16:56.814 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/anubis.ko 2020-11-16T22:16:56.815 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/arc4.ko 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/ 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/ 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_memcpy.ko 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_pq.ko 2020-11-16T22:16:56.816 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_raid6_recov.ko 2020-11-16T22:16:56.817 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_tx.ko 2020-11-16T22:16:56.817 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_xor.ko 2020-11-16T22:16:56.818 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/raid6test.ko 2020-11-16T22:16:56.818 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authenc.ko 2020-11-16T22:16:56.818 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authencesn.ko 2020-11-16T22:16:56.818 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blake2b_generic.ko 2020-11-16T22:16:56.818 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_common.ko 2020-11-16T22:16:56.819 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_generic.ko 2020-11-16T22:16:56.819 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/camellia_generic.ko 2020-11-16T22:16:56.819 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast5_generic.ko 2020-11-16T22:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast6_generic.ko 2020-11-16T22:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast_common.ko 2020-11-16T22:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ccm.ko 2020-11-16T22:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cmac.ko 2020-11-16T22:16:56.820 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crc32_generic.ko 2020-11-16T22:16:56.821 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cryptd.ko 2020-11-16T22:16:56.821 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_engine.ko 2020-11-16T22:16:56.822 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_simd.ko 2020-11-16T22:16:56.822 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_user.ko 2020-11-16T22:16:56.822 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cts.ko 2020-11-16T22:16:56.822 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/des_generic.ko 2020-11-16T22:16:56.822 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/dh_generic.ko 2020-11-16T22:16:56.823 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/echainiv.ko 2020-11-16T22:16:56.823 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/essiv.ko 2020-11-16T22:16:56.824 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/fcrypt.ko 2020-11-16T22:16:56.824 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/khazad.ko 2020-11-16T22:16:56.824 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lrw.ko 2020-11-16T22:16:56.824 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4hc.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/md4.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/michael_mic.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcbc.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcrypt.ko 2020-11-16T22:16:56.825 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd128.ko 2020-11-16T22:16:56.826 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd160.ko 2020-11-16T22:16:56.826 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd256.ko 2020-11-16T22:16:56.826 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd320.ko 2020-11-16T22:16:56.827 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/salsa20_generic.ko 2020-11-16T22:16:56.827 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/seed.ko 2020-11-16T22:16:56.827 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/serpent_generic.ko 2020-11-16T22:16:56.827 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 170649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tcrypt.ko 2020-11-16T22:16:56.828 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tea.ko 2020-11-16T22:16:56.828 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tgr192.ko 2020-11-16T22:16:56.828 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_common.ko 2020-11-16T22:16:56.829 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_generic.ko 2020-11-16T22:16:56.829 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/vmac.ko 2020-11-16T22:16:56.829 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/wp512.ko 2020-11-16T22:16:56.830 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xcbc.ko 2020-11-16T22:16:56.830 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xor.ko 2020-11-16T22:16:56.830 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xts.ko 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xxhash_generic.ko 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_extlog.ko 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_ipmi.ko 2020-11-16T22:16:56.831 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_pad.ko 2020-11-16T22:16:56.832 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/ 2020-11-16T22:16:56.832 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/einj.ko 2020-11-16T22:16:56.832 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ec_sys.ko 2020-11-16T22:16:56.833 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbs.ko 2020-11-16T22:16:56.833 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbshc.ko 2020-11-16T22:16:56.833 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/video.ko 2020-11-16T22:16:56.833 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ 2020-11-16T22:16:56.834 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/acard-ahci.ko 2020-11-16T22:16:56.834 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 84793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci.ko 2020-11-16T22:16:56.834 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci_platform.ko 2020-11-16T22:16:56.834 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 81825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci.ko 2020-11-16T22:16:56.835 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci_platform.ko 2020-11-16T22:16:56.835 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_acpi.ko 2020-11-16T22:16:56.836 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ali.ko 2020-11-16T22:16:56.836 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_amd.ko 2020-11-16T22:16:56.836 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_artop.ko 2020-11-16T22:16:56.837 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atiixp.ko 2020-11-16T22:16:56.837 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atp867x.ko 2020-11-16T22:16:56.837 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd640.ko 2020-11-16T22:16:56.837 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd64x.ko 2020-11-16T22:16:56.838 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cypress.ko 2020-11-16T22:16:56.838 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_efar.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt366.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt37x.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x2n.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x3.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it8213.ko 2020-11-16T22:16:56.839 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it821x.ko 2020-11-16T22:16:56.840 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_jmicron.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_legacy.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_marvell.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_mpiix.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_netcell.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ninja32.ko 2020-11-16T22:16:56.841 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87410.ko 2020-11-16T22:16:56.842 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87415.ko 2020-11-16T22:16:56.842 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_oldpiix.ko 2020-11-16T22:16:56.842 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_opti.ko 2020-11-16T22:16:56.842 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_optidma.ko 2020-11-16T22:16:56.842 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pcmcia.ko 2020-11-16T22:16:56.843 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc2027x.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc202xx_old.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_piccolo.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_platform.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_radisys.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rdc.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rz1000.ko 2020-11-16T22:16:56.844 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sch.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_serverworks.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sil680.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sl82c105.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_triflex.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_via.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pdc_adma.ko 2020-11-16T22:16:56.845 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_inic162x.ko 2020-11-16T22:16:56.846 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 76097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_mv.ko 2020-11-16T22:16:56.846 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_nv.ko 2020-11-16T22:16:56.846 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_promise.ko 2020-11-16T22:16:56.847 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_qstor.ko 2020-11-16T22:16:56.847 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil.ko 2020-11-16T22:16:56.847 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil24.ko 2020-11-16T22:16:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sis.ko 2020-11-16T22:16:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_svw.ko 2020-11-16T22:16:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sx4.ko 2020-11-16T22:16:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_uli.ko 2020-11-16T22:16:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_via.ko 2020-11-16T22:16:56.849 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_vsc.ko 2020-11-16T22:16:56.849 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ 2020-11-16T22:16:56.849 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ks0108.ko 2020-11-16T22:16:56.849 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/ 2020-11-16T22:16:56.849 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 123697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/bcma.ko 2020-11-16T22:16:56.850 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/ 2020-11-16T22:16:56.850 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/ 2020-11-16T22:16:56.850 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 115129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/aoe.ko 2020-11-16T22:16:56.851 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/cryptoloop.ko 2020-11-16T22:16:56.852 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/ 2020-11-16T22:16:56.852 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1050545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/drbd.ko 2020-11-16T22:16:56.865 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 176513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/floppy.ko 2020-11-16T22:16:56.867 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/ 2020-11-16T22:16:56.868 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 107353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/mtip32xx.ko 2020-11-16T22:16:56.869 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/nbd.ko 2020-11-16T22:16:56.870 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 92265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/null_blk.ko 2020-11-16T22:16:56.871 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 83089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/pktcdvd.ko 2020-11-16T22:16:56.872 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 260233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rbd.ko 2020-11-16T22:16:56.875 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/ 2020-11-16T22:16:56.875 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 113017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/rsxx.ko 2020-11-16T22:16:56.877 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 140897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/skd.ko 2020-11-16T22:16:56.878 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/sx8.ko 2020-11-16T22:16:56.879 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/umem.ko 2020-11-16T22:16:56.880 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/ 2020-11-16T22:16:56.880 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 101497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/xen-blkback.ko 2020-11-16T22:16:56.881 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/ 2020-11-16T22:16:56.881 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/zram.ko 2020-11-16T22:16:56.882 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ 2020-11-16T22:16:56.882 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/applicom.ko 2020-11-16T22:16:56.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hangcheck-timer.ko 2020-11-16T22:16:56.883 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/ 2020-11-16T22:16:56.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/amd-rng.ko 2020-11-16T22:16:56.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/intel-rng.ko 2020-11-16T22:16:56.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/timeriomem-rng.ko 2020-11-16T22:16:56.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/via-rng.ko 2020-11-16T22:16:56.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/virtio-rng.ko 2020-11-16T22:16:56.884 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ 2020-11-16T22:16:56.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_devintf.ko 2020-11-16T22:16:56.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 138969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2020-11-16T22:16:56.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2020-11-16T22:16:56.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 142913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_si.ko 2020-11-16T22:16:56.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_ssif.ko 2020-11-16T22:16:56.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2020-11-16T22:16:56.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/lp.ko 2020-11-16T22:16:56.890 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/ 2020-11-16T22:16:56.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/mwave.ko 2020-11-16T22:16:56.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/nvram.ko 2020-11-16T22:16:56.891 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/ 2020-11-16T22:16:56.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 102257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4000_cs.ko 2020-11-16T22:16:56.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4040_cs.ko 2020-11-16T22:16:56.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 86617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/synclink_cs.ko 2020-11-16T22:16:56.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ppdev.ko 2020-11-16T22:16:56.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/raw.ko 2020-11-16T22:16:56.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tlclk.ko 2020-11-16T22:16:56.895 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/ 2020-11-16T22:16:56.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_atmel.ko 2020-11-16T22:16:56.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2020-11-16T22:16:56.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2020-11-16T22:16:56.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2020-11-16T22:16:56.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_infineon.ko 2020-11-16T22:16:56.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_nsc.ko 2020-11-16T22:16:56.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/xen-tpmfront.ko 2020-11-16T22:16:56.897 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/ 2020-11-16T22:16:56.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_core.ko 2020-11-16T22:16:56.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_pcie.ko 2020-11-16T22:16:56.899 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/ 2020-11-16T22:16:56.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-palmas.ko 2020-11-16T22:16:56.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-s2mps11.ko 2020-11-16T22:16:56.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-si5351.ko 2020-11-16T22:16:56.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-twl6040.ko 2020-11-16T22:16:56.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-wm831x.ko 2020-11-16T22:16:56.900 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/ 2020-11-16T22:16:56.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2020-11-16T22:16:56.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/p4-clockmod.ko 2020-11-16T22:16:56.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/speedstep-lib.ko 2020-11-16T22:16:56.901 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ 2020-11-16T22:16:56.901 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ 2020-11-16T22:16:56.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 70121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp-crypto.ko 2020-11-16T22:16:56.902 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 203753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp.ko 2020-11-16T22:16:56.903 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-aes.ko 2020-11-16T22:16:56.904 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-sha.ko 2020-11-16T22:16:56.904 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/ 2020-11-16T22:16:56.904 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/ 2020-11-16T22:16:56.904 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 279065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2020-11-16T22:16:56.908 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/ 2020-11-16T22:16:56.908 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2020-11-16T22:16:56.908 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/ 2020-11-16T22:16:56.908 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/virtio_crypto.ko 2020-11-16T22:16:56.909 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/ 2020-11-16T22:16:56.909 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/dca.ko 2020-11-16T22:16:56.909 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ 2020-11-16T22:16:56.909 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/ 2020-11-16T22:16:56.910 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/dw_dmac.ko 2020-11-16T22:16:56.910 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ 2020-11-16T22:16:56.910 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 124161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ioatdma.ko 2020-11-16T22:16:56.911 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ 2020-11-16T22:16:56.911 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/amd64_edac_mod.ko 2020-11-16T22:16:56.911 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/e752x_edac.ko 2020-11-16T22:16:56.912 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/edac_mce_amd.ko 2020-11-16T22:16:56.913 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3000_edac.ko 2020-11-16T22:16:56.913 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3200_edac.ko 2020-11-16T22:16:56.914 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5000_edac.ko 2020-11-16T22:16:56.914 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5100_edac.ko 2020-11-16T22:16:56.914 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5400_edac.ko 2020-11-16T22:16:56.914 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7300_edac.ko 2020-11-16T22:16:56.915 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7core_edac.ko 2020-11-16T22:16:56.915 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i82975x_edac.ko 2020-11-16T22:16:56.915 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ie31200_edac.ko 2020-11-16T22:16:56.916 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/sb_edac.ko 2020-11-16T22:16:56.916 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/x38_edac.ko 2020-11-16T22:16:56.916 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/ 2020-11-16T22:16:56.917 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-gpio.ko 2020-11-16T22:16:56.917 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max14577.ko 2020-11-16T22:16:56.917 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max77693.ko 2020-11-16T22:16:56.918 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max8997.ko 2020-11-16T22:16:56.918 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-palmas.ko 2020-11-16T22:16:56.918 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-rt8973a.ko 2020-11-16T22:16:56.918 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-sm5502.ko 2020-11-16T22:16:56.919 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/ 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/dmi-sysfs.ko 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/ 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/efi-pstore.ko 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/iscsi_ibft.ko 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/ 2020-11-16T22:16:56.920 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5520.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5588.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-amd8111.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-arizona.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9052.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9055.ko 2020-11-16T22:16:56.921 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-dln2.ko 2020-11-16T22:16:56.923 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-f7188x.ko 2020-11-16T22:16:56.923 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-generic.ko 2020-11-16T22:16:56.923 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ich.ko 2020-11-16T22:16:56.923 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-janz-ttl.ko 2020-11-16T22:16:56.923 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-kempld.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-lp3943.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max7300.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max730x.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max732x.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ml-ioh.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pca953x.ko 2020-11-16T22:16:56.924 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pcf857x.ko 2020-11-16T22:16:56.925 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-rdc321x.ko 2020-11-16T22:16:56.925 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch.ko 2020-11-16T22:16:56.925 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch311x.ko 2020-11-16T22:16:56.925 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-tps65912.ko 2020-11-16T22:16:56.925 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl4030.ko 2020-11-16T22:16:56.927 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl6040.ko 2020-11-16T22:16:56.927 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-viperboard.ko 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-vx855.ko 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm831x.ko 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8350.ko 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8994.ko 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/ 2020-11-16T22:16:56.928 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-a4tech.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-apple.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-appleir.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-aureal.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-axff.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-belkin.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cherry.ko 2020-11-16T22:16:56.929 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-chicony.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cp2112.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cypress.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-dr.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elecom.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elo.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-emsff.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ezkey.ko 2020-11-16T22:16:56.930 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gaff.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-generic.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gt683r.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gyration.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-kbd.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-mouse.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtekff.ko 2020-11-16T22:16:56.931 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-hyperv.ko 2020-11-16T22:16:56.933 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-icade.ko 2020-11-16T22:16:56.933 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kensington.ko 2020-11-16T22:16:56.933 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-keytouch.ko 2020-11-16T22:16:56.933 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kye.ko 2020-11-16T22:16:56.933 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lcpower.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-led.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lenovo.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lg-g15.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-dj.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-hidpp.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 93089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech.ko 2020-11-16T22:16:56.934 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-magicmouse.ko 2020-11-16T22:16:56.935 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-microsoft.ko 2020-11-16T22:16:56.935 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-monterey.ko 2020-11-16T22:16:56.935 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-multitouch.ko 2020-11-16T22:16:56.936 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ntrig.ko 2020-11-16T22:16:56.936 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ortek.ko 2020-11-16T22:16:56.936 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-penmount.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-petalynx.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-picolcd.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-pl.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-plantronics.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-primax.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-rmi.ko 2020-11-16T22:16:56.937 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-arvo.ko 2020-11-16T22:16:56.938 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-common.ko 2020-11-16T22:16:56.938 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-isku.ko 2020-11-16T22:16:56.938 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kone.ko 2020-11-16T22:16:56.938 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-koneplus.ko 2020-11-16T22:16:56.939 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-konepure.ko 2020-11-16T22:16:56.939 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kovaplus.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-lua.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-pyra.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-ryos.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-savu.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-saitek.ko 2020-11-16T22:16:56.940 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-samsung.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sensor-hub.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sjoy.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sony.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-speedlink.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-steelseries.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sunplus.ko 2020-11-16T22:16:56.941 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tivo.ko 2020-11-16T22:16:56.942 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tmff.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-topseed.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-twinhan.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-uclogic.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-waltop.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 133729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-wiimote.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-xinmo.ko 2020-11-16T22:16:56.943 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zpff.ko 2020-11-16T22:16:56.944 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zydacron.ko 2020-11-16T22:16:56.944 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 284097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid.ko 2020-11-16T22:16:56.946 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/ 2020-11-16T22:16:56.946 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/i2c-hid.ko 2020-11-16T22:16:56.947 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/uhid.ko 2020-11-16T22:16:56.947 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/ 2020-11-16T22:16:56.947 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 125177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbhid.ko 2020-11-16T22:16:56.948 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbkbd.ko 2020-11-16T22:16:56.949 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbmouse.ko 2020-11-16T22:16:56.949 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 222905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/wacom.ko 2020-11-16T22:16:56.951 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/ 2020-11-16T22:16:56.952 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_balloon.ko 2020-11-16T22:16:56.952 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 79585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_utils.ko 2020-11-16T22:16:56.953 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 277809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_vmbus.ko 2020-11-16T22:16:56.956 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ 2020-11-16T22:16:56.956 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 65649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru.ko 2020-11-16T22:16:56.956 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru3.ko 2020-11-16T22:16:56.957 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/acpi_power_meter.ko 2020-11-16T22:16:56.957 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7414.ko 2020-11-16T22:16:56.957 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7418.ko 2020-11-16T22:16:56.958 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adc128d818.ko 2020-11-16T22:16:56.958 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1021.ko 2020-11-16T22:16:56.958 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1025.ko 2020-11-16T22:16:56.959 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 98513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1026.ko 2020-11-16T22:16:56.959 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1029.ko 2020-11-16T22:16:56.959 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1031.ko 2020-11-16T22:16:56.960 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm9240.ko 2020-11-16T22:16:56.960 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ads7828.ko 2020-11-16T22:16:56.960 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7410.ko 2020-11-16T22:16:56.961 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7411.ko 2020-11-16T22:16:56.961 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7462.ko 2020-11-16T22:16:56.961 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7470.ko 2020-11-16T22:16:56.962 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7475.ko 2020-11-16T22:16:56.962 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7x10.ko 2020-11-16T22:16:56.963 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/amc6821.ko 2020-11-16T22:16:56.963 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/applesmc.ko 2020-11-16T22:16:56.963 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asb100.ko 2020-11-16T22:16:56.964 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asc7621.ko 2020-11-16T22:16:56.964 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asus_atk0110.ko 2020-11-16T22:16:56.965 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/atxp1.ko 2020-11-16T22:16:56.965 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/coretemp.ko 2020-11-16T22:16:56.966 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9052-hwmon.ko 2020-11-16T22:16:56.966 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9055-hwmon.ko 2020-11-16T22:16:56.966 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dell-smm-hwmon.ko 2020-11-16T22:16:56.967 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 87601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dme1737.ko 2020-11-16T22:16:56.967 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds1621.ko 2020-11-16T22:16:56.967 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds620.ko 2020-11-16T22:16:56.968 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc1403.ko 2020-11-16T22:16:56.968 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc2103.ko 2020-11-16T22:16:56.968 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc6w201.ko 2020-11-16T22:16:56.969 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71805f.ko 2020-11-16T22:16:56.969 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 94745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71882fg.ko 2020-11-16T22:16:56.970 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f75375s.ko 2020-11-16T22:16:56.970 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fam15h_power.ko 2020-11-16T22:16:56.970 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 54809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fschmd.ko 2020-11-16T22:16:56.971 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g760a.ko 2020-11-16T22:16:56.971 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g762.ko 2020-11-16T22:16:56.971 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl518sm.ko 2020-11-16T22:16:56.972 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl520sm.ko 2020-11-16T22:16:56.972 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hih6130.ko 2020-11-16T22:16:56.972 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hwmon-vid.ko 2020-11-16T22:16:56.973 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5500_temp.ko 2020-11-16T22:16:56.973 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5k_amb.ko 2020-11-16T22:16:56.973 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmaem.ko 2020-11-16T22:16:56.973 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmpex.ko 2020-11-16T22:16:56.974 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina209.ko 2020-11-16T22:16:56.974 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina2xx.ko 2020-11-16T22:16:56.974 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 109417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/it87.ko 2020-11-16T22:16:56.975 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/jc42.ko 2020-11-16T22:16:56.975 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k10temp.ko 2020-11-16T22:16:56.975 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k8temp.ko 2020-11-16T22:16:56.975 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lineage-pem.ko 2020-11-16T22:16:56.976 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm63.ko 2020-11-16T22:16:56.976 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm73.ko 2020-11-16T22:16:56.976 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm75.ko 2020-11-16T22:16:56.977 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm77.ko 2020-11-16T22:16:56.977 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm78.ko 2020-11-16T22:16:56.977 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm80.ko 2020-11-16T22:16:56.978 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm83.ko 2020-11-16T22:16:56.978 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm85.ko 2020-11-16T22:16:56.978 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm87.ko 2020-11-16T22:16:56.978 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm90.ko 2020-11-16T22:16:56.979 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm92.ko 2020-11-16T22:16:56.979 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 104761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm93.ko 2020-11-16T22:16:56.980 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95234.ko 2020-11-16T22:16:56.980 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95241.ko 2020-11-16T22:16:56.980 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95245.ko 2020-11-16T22:16:56.981 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc2945.ko 2020-11-16T22:16:56.981 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4151.ko 2020-11-16T22:16:56.981 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4215.ko 2020-11-16T22:16:56.982 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4222.ko 2020-11-16T22:16:56.982 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4245.ko 2020-11-16T22:16:56.982 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4260.ko 2020-11-16T22:16:56.982 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4261.ko 2020-11-16T22:16:56.982 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max16065.ko 2020-11-16T22:16:56.983 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1619.ko 2020-11-16T22:16:56.983 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1668.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max197.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6639.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6642.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6650.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6697.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mc13783-adc.ko 2020-11-16T22:16:56.984 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mcp3021.ko 2020-11-16T22:16:56.985 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2020-11-16T22:16:56.985 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6683.ko 2020-11-16T22:16:56.985 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 124641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6775.ko 2020-11-16T22:16:56.986 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct7802.ko 2020-11-16T22:16:56.986 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ntc_thermistor.ko 2020-11-16T22:16:56.986 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 79793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87360.ko 2020-11-16T22:16:56.987 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87427.ko 2020-11-16T22:16:56.987 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pcf8591.ko 2020-11-16T22:16:56.988 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ 2020-11-16T22:16:56.988 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/adm1275.ko 2020-11-16T22:16:56.988 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/lm25066.ko 2020-11-16T22:16:56.988 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ltc2978.ko 2020-11-16T22:16:56.989 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max16064.ko 2020-11-16T22:16:56.989 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max34440.ko 2020-11-16T22:16:56.989 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max8688.ko 2020-11-16T22:16:56.989 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus.ko 2020-11-16T22:16:56.990 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2020-11-16T22:16:56.990 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/tps40422.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9000.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9200.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/zl6100.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/powr1220.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5627.ko 2020-11-16T22:16:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5636.ko 2020-11-16T22:16:56.992 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch56xx-common.ko 2020-11-16T22:16:56.992 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht15.ko 2020-11-16T22:16:56.992 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht21.ko 2020-11-16T22:16:56.993 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/shtc1.ko 2020-11-16T22:16:56.993 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sis5595.ko 2020-11-16T22:16:56.993 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smm665.ko 2020-11-16T22:16:56.993 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47b397.ko 2020-11-16T22:16:56.993 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m1.ko 2020-11-16T22:16:56.994 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m192.ko 2020-11-16T22:16:56.994 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/thmc50.ko 2020-11-16T22:16:56.994 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp102.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp103.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp401.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp421.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via-cputemp.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via686a.ko 2020-11-16T22:16:56.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt1211.ko 2020-11-16T22:16:56.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt8231.ko 2020-11-16T22:16:56.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627ehf.ko 2020-11-16T22:16:56.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627hf.ko 2020-11-16T22:16:56.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83781d.ko 2020-11-16T22:16:56.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 62953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83791d.ko 2020-11-16T22:16:56.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83792d.ko 2020-11-16T22:16:56.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 101689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83793.ko 2020-11-16T22:16:56.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 119921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83795.ko 2020-11-16T22:16:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l785ts.ko 2020-11-16T22:16:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l786ng.ko 2020-11-16T22:16:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm831x-hwmon.ko 2020-11-16T22:16:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm8350-hwmon.ko 2020-11-16T22:16:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/ 2020-11-16T22:16:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/ 2020-11-16T22:16:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2020-11-16T22:16:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2020-11-16T22:16:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/ 2020-11-16T22:16:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1535.ko 2020-11-16T22:16:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1563.ko 2020-11-16T22:16:57.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2020-11-16T22:16:57.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2020-11-16T22:16:57.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756.ko 2020-11-16T22:16:57.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd8111.ko 2020-11-16T22:16:57.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2020-11-16T22:16:57.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2020-11-16T22:16:57.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-core.ko 2020-11-16T22:16:57.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2020-11-16T22:16:57.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2020-11-16T22:16:57.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2020-11-16T22:16:57.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-dln2.ko 2020-11-16T22:16:57.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-gpio.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 64897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-i801.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-isch.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ismt.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-kempld.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2020-11-16T22:16:57.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2.ko 2020-11-16T22:16:57.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ocores.ko 2020-11-16T22:16:57.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-parport.ko 2020-11-16T22:16:57.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2020-11-16T22:16:57.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-piix4.ko 2020-11-16T22:16:57.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-scmi.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-simtec.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis5595.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis630.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis96x.ko 2020-11-16T22:16:57.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2020-11-16T22:16:57.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2020-11-16T22:16:57.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-via.ko 2020-11-16T22:16:57.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viapro.ko 2020-11-16T22:16:57.012 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viperboard.ko 2020-11-16T22:16:57.012 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-xiic.ko 2020-11-16T22:16:57.012 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-mux.ko 2020-11-16T22:16:57.012 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-smbus.ko 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-stub.ko 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/ 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ 2020-11-16T22:16:57.013 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ 2020-11-16T22:16:57.014 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 259017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_cm.ko 2020-11-16T22:16:57.014 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 848849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_core.ko 2020-11-16T22:16:57.024 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_umad.ko 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 309825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_uverbs.ko 2020-11-16T22:16:57.029 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/iw_cm.ko 2020-11-16T22:16:57.030 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 263657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_cm.ko 2020-11-16T22:16:57.033 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 62129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_ucm.ko 2020-11-16T22:16:57.033 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ 2020-11-16T22:16:57.034 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/ 2020-11-16T22:16:57.034 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 512473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2020-11-16T22:16:57.040 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/ 2020-11-16T22:16:57.040 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 420329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2020-11-16T22:16:57.046 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/ 2020-11-16T22:16:57.046 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 721305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2020-11-16T22:16:57.055 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ 2020-11-16T22:16:57.055 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 287121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2020-11-16T22:16:57.059 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ 2020-11-16T22:16:57.059 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 162257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2020-11-16T22:16:57.061 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/ 2020-11-16T22:16:57.062 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 177449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2020-11-16T22:16:57.063 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ 2020-11-16T22:16:57.063 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ 2020-11-16T22:16:57.063 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 272041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2020-11-16T22:16:57.066 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ 2020-11-16T22:16:57.067 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 105369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2020-11-16T22:16:57.068 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ 2020-11-16T22:16:57.068 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 114745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2020-11-16T22:16:57.069 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ 2020-11-16T22:16:57.069 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 133665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2020-11-16T22:16:57.071 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ 2020-11-16T22:16:57.071 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 131153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2020-11-16T22:16:57.072 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ 2020-11-16T22:16:57.072 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/evbug.ko 2020-11-16T22:16:57.073 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ff-memless.ko 2020-11-16T22:16:57.073 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ 2020-11-16T22:16:57.074 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/emu10k1-gp.ko 2020-11-16T22:16:57.074 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/fm801-gp.ko 2020-11-16T22:16:57.074 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/gameport.ko 2020-11-16T22:16:57.074 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/lightning.ko 2020-11-16T22:16:57.074 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ns558.ko 2020-11-16T22:16:57.075 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/input-polldev.ko 2020-11-16T22:16:57.075 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/joydev.ko 2020-11-16T22:16:57.075 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/ 2020-11-16T22:16:57.075 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5520-keys.ko 2020-11-16T22:16:57.076 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5588-keys.ko 2020-11-16T22:16:57.076 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5589-keys.ko 2020-11-16T22:16:57.076 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lkkbd.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8323.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8333.ko 2020-11-16T22:16:57.077 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/matrix_keypad.ko 2020-11-16T22:16:57.078 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/max7359_keypad.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mcs_touchkey.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/newtonkbd.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/opencores-kbd.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt1070.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt2160.ko 2020-11-16T22:16:57.079 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/samsung-keypad.ko 2020-11-16T22:16:57.080 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/stowaway.ko 2020-11-16T22:16:57.080 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/sunkbd.ko 2020-11-16T22:16:57.080 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca6416-keypad.ko 2020-11-16T22:16:57.080 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca8418_keypad.ko 2020-11-16T22:16:57.081 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/twl4030_keypad.ko 2020-11-16T22:16:57.081 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/xtkbd.ko 2020-11-16T22:16:57.081 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/matrix-keymap.ko 2020-11-16T22:16:57.081 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/ 2020-11-16T22:16:57.081 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/appletouch.ko 2020-11-16T22:16:57.082 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/bcm5974.ko 2020-11-16T22:16:57.082 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 124297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/cyapatp.ko 2020-11-16T22:16:57.082 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 94273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/elan_i2c.ko 2020-11-16T22:16:57.082 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/gpio_mouse.ko 2020-11-16T22:16:57.083 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 337393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/psmouse.ko 2020-11-16T22:16:57.087 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/sermouse.ko 2020-11-16T22:16:57.087 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_i2c.ko 2020-11-16T22:16:57.087 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_usb.ko 2020-11-16T22:16:57.087 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/vsxxxaa.ko 2020-11-16T22:16:57.088 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/ 2020-11-16T22:16:57.088 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 113449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/rmi_core.ko 2020-11-16T22:16:57.089 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ 2020-11-16T22:16:57.089 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/altera_ps2.ko 2020-11-16T22:16:57.089 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/arc_ps2.ko 2020-11-16T22:16:57.089 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ct82c710.ko 2020-11-16T22:16:57.090 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/hyperv-keyboard.ko 2020-11-16T22:16:57.090 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/parkbd.ko 2020-11-16T22:16:57.090 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/pcips2.ko 2020-11-16T22:16:57.091 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ps2mult.ko 2020-11-16T22:16:57.091 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serio_raw.ko 2020-11-16T22:16:57.091 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serport.ko 2020-11-16T22:16:57.091 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/sparse-keymap.ko 2020-11-16T22:16:57.092 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/ 2020-11-16T22:16:57.092 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/ 2020-11-16T22:16:57.092 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/iommu_v2.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/ 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-88pm860x.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-adp5520.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-bd2802.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-blinkm.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-clevo-mail.ko 2020-11-16T22:16:57.093 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da903x.ko 2020-11-16T22:16:57.094 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da9052.ko 2020-11-16T22:16:57.094 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-gpio.ko 2020-11-16T22:16:57.094 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3530.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3533.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm355x.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3642.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp3944.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp8788.ko 2020-11-16T22:16:57.096 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-max8997.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-mc13783.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-menf21bmc.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca9532.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca955x.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca963x.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-ss4200.ko 2020-11-16T22:16:57.097 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-tca6507.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm831x-status.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm8350.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-audio.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-camera.ko 2020-11-16T22:16:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2020-11-16T22:16:57.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2020-11-16T22:16:57.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2020-11-16T22:16:57.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2020-11-16T22:16:57.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-timer.ko 2020-11-16T22:16:57.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-transient.ko 2020-11-16T22:16:57.100 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/ 2020-11-16T22:16:57.100 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/ 2020-11-16T22:16:57.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 633113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/bcache.ko 2020-11-16T22:16:57.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bio-prison.ko 2020-11-16T22:16:57.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bufio.ko 2020-11-16T22:16:57.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache-smq.ko 2020-11-16T22:16:57.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 146689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache.ko 2020-11-16T22:16:57.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 98065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-crypt.ko 2020-11-16T22:16:57.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-delay.ko 2020-11-16T22:16:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-era.ko 2020-11-16T22:16:57.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-flakey.ko 2020-11-16T22:16:57.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log-userspace.ko 2020-11-16T22:16:57.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log.ko 2020-11-16T22:16:57.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-mirror.ko 2020-11-16T22:16:57.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-multipath.ko 2020-11-16T22:16:57.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-queue-length.ko 2020-11-16T22:16:57.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-raid.ko 2020-11-16T22:16:57.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-region-hash.ko 2020-11-16T22:16:57.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-round-robin.ko 2020-11-16T22:16:57.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-service-time.ko 2020-11-16T22:16:57.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-snapshot.ko 2020-11-16T22:16:57.115 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-switch.ko 2020-11-16T22:16:57.115 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 182441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-thin-pool.ko 2020-11-16T22:16:57.117 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-verity.ko 2020-11-16T22:16:57.118 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-zero.ko 2020-11-16T22:16:57.118 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/faulty.ko 2020-11-16T22:16:57.118 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/linear.ko 2020-11-16T22:16:57.119 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/multipath.ko 2020-11-16T22:16:57.119 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/ 2020-11-16T22:16:57.119 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 179553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2020-11-16T22:16:57.120 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid0.ko 2020-11-16T22:16:57.121 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 111753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid1.ko 2020-11-16T22:16:57.122 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 156737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid10.ko 2020-11-16T22:16:57.124 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 385993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid456.ko 2020-11-16T22:16:57.129 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/ 2020-11-16T22:16:57.130 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/ 2020-11-16T22:16:57.130 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 194137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptbase.ko 2020-11-16T22:16:57.132 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 70409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptctl.ko 2020-11-16T22:16:57.133 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptfc.ko 2020-11-16T22:16:57.133 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptlan.ko 2020-11-16T22:16:57.134 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 129617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptsas.ko 2020-11-16T22:16:57.136 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 90705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptscsih.ko 2020-11-16T22:16:57.136 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptspi.ko 2020-11-16T22:16:57.137 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ 2020-11-16T22:16:57.138 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm800.ko 2020-11-16T22:16:57.138 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm805.ko 2020-11-16T22:16:57.138 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm80x.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ab3100-otp.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/arizona-i2c.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/bcm590xx.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/cros_ec_dev.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/dln2.ko 2020-11-16T22:16:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/htc-pasic3.ko 2020-11-16T22:16:57.140 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/janz-cmodio.ko 2020-11-16T22:16:57.140 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/kempld-core.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-core.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-ctrlbank.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lp3943.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_ich.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_sch.ko 2020-11-16T22:16:57.141 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/max8907.ko 2020-11-16T22:16:57.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-core.ko 2020-11-16T22:16:57.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-i2c.ko 2020-11-16T22:16:57.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/menf21bmc.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-adc.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-gpio.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/rdc321x-southbridge.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/retu-mfd.ko 2020-11-16T22:16:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 76449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/si476x-core.ko 2020-11-16T22:16:57.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/sm501.ko 2020-11-16T22:16:57.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ti_am335x_tscadc.ko 2020-11-16T22:16:57.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps65010.ko 2020-11-16T22:16:57.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps6507x.ko 2020-11-16T22:16:57.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/viperboard.ko 2020-11-16T22:16:57.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/vx855.ko 2020-11-16T22:16:57.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wl1273-core.ko 2020-11-16T22:16:57.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wm8994.ko 2020-11-16T22:16:57.146 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ 2020-11-16T22:16:57.146 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot-i2c.ko 2020-11-16T22:16:57.146 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot.ko 2020-11-16T22:16:57.146 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/ 2020-11-16T22:16:57.147 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2020-11-16T22:16:57.147 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds9802als.ko 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds990x.ko 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/bh1770glc.ko 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/ 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/core.ko 2020-11-16T22:16:57.148 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/ 2020-11-16T22:16:57.149 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/cb710.ko 2020-11-16T22:16:57.149 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ds1682.ko 2020-11-16T22:16:57.150 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/dummy-irq.ko 2020-11-16T22:16:57.150 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/ 2020-11-16T22:16:57.150 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/echo.ko 2020-11-16T22:16:57.150 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/ 2020-11-16T22:16:57.150 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/at24.ko 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom.ko 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/max6875.ko 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/enclosure.ko 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/ 2020-11-16T22:16:57.151 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 151177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/genwqe_card.ko 2020-11-16T22:16:57.152 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hmc6352.ko 2020-11-16T22:16:57.152 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hpilo.ko 2020-11-16T22:16:57.152 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ 2020-11-16T22:16:57.152 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ibmasm.ko 2020-11-16T22:16:57.153 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ics932s401.ko 2020-11-16T22:16:57.153 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29003.ko 2020-11-16T22:16:57.153 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29020.ko 2020-11-16T22:16:57.154 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/ 2020-11-16T22:16:57.154 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2020-11-16T22:16:57.154 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2020-11-16T22:16:57.154 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/ 2020-11-16T22:16:57.155 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 121553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-me.ko 2020-11-16T22:16:57.155 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-txe.ko 2020-11-16T22:16:57.156 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 279201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei.ko 2020-11-16T22:16:57.159 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/phantom.ko 2020-11-16T22:16:57.160 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/pvpanic.ko 2020-11-16T22:16:57.160 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/ 2020-11-16T22:16:57.160 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/st_drv.ko 2020-11-16T22:16:57.160 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_7xx1.ko 2020-11-16T22:16:57.161 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_core.ko 2020-11-16T22:16:57.161 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tsl2550.ko 2020-11-16T22:16:57.161 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_balloon.ko 2020-11-16T22:16:57.162 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/ 2020-11-16T22:16:57.162 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 179385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2020-11-16T22:16:57.164 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ 2020-11-16T22:16:57.164 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/ 2020-11-16T22:16:57.165 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 433953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/bonding.ko 2020-11-16T22:16:57.170 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/ 2020-11-16T22:16:57.170 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/ 2020-11-16T22:16:57.170 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 96929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/b53_common.ko 2020-11-16T22:16:57.171 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/bcm-sf2.ko 2020-11-16T22:16:57.171 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6060.ko 2020-11-16T22:16:57.172 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/ 2020-11-16T22:16:57.172 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 346377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2020-11-16T22:16:57.175 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dummy.ko 2020-11-16T22:16:57.176 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/eql.ko 2020-11-16T22:16:57.176 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ 2020-11-16T22:16:57.176 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/ 2020-11-16T22:16:57.176 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2020-11-16T22:16:57.177 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2020-11-16T22:16:57.177 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 123353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c59x.ko 2020-11-16T22:16:57.178 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/typhoon.ko 2020-11-16T22:16:57.179 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ 2020-11-16T22:16:57.179 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/8390.ko 2020-11-16T22:16:57.180 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2020-11-16T22:16:57.180 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2020-11-16T22:16:57.180 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 96881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2020-11-16T22:16:57.182 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/ 2020-11-16T22:16:57.182 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/starfire.ko 2020-11-16T22:16:57.182 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/ 2020-11-16T22:16:57.182 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/et131x.ko 2020-11-16T22:16:57.183 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/ 2020-11-16T22:16:57.183 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/acenic.ko 2020-11-16T22:16:57.184 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/ 2020-11-16T22:16:57.184 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/altera_tse.ko 2020-11-16T22:16:57.185 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/ 2020-11-16T22:16:57.185 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/amd8111e.ko 2020-11-16T22:16:57.185 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2020-11-16T22:16:57.186 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 90281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/pcnet32.ko 2020-11-16T22:16:57.186 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/ 2020-11-16T22:16:57.187 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/ 2020-11-16T22:16:57.187 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 91593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2020-11-16T22:16:57.188 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/ 2020-11-16T22:16:57.188 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 98001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2020-11-16T22:16:57.189 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/ 2020-11-16T22:16:57.189 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 89785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2020-11-16T22:16:57.190 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/ 2020-11-16T22:16:57.190 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 86737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2020-11-16T22:16:57.191 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 70129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2020-11-16T22:16:57.192 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/ 2020-11-16T22:16:57.192 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 78065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/b44.ko 2020-11-16T22:16:57.193 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 167025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2020-11-16T22:16:57.195 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2020-11-16T22:16:57.196 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1528577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2020-11-16T22:16:57.216 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 130113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/cnic.ko 2020-11-16T22:16:57.218 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/ 2020-11-16T22:16:57.218 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 129345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2020-11-16T22:16:57.219 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 314625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/tg3.ko 2020-11-16T22:16:57.224 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/ 2020-11-16T22:16:57.224 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/ 2020-11-16T22:16:57.225 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 330209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/ 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/ 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 187345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2020-11-16T22:16:57.231 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2020-11-16T22:16:57.231 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 349953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2020-11-16T22:16:57.235 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2020-11-16T22:16:57.236 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1039081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2020-11-16T22:16:57.249 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2020-11-16T22:16:57.250 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 165321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2020-11-16T22:16:57.251 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2020-11-16T22:16:57.251 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2020-11-16T22:16:57.252 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/ 2020-11-16T22:16:57.252 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/ 2020-11-16T22:16:57.252 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 205121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2020-11-16T22:16:57.254 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/ 2020-11-16T22:16:57.254 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/ 2020-11-16T22:16:57.254 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2020-11-16T22:16:57.255 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 97801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de4x5.ko 2020-11-16T22:16:57.256 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 57089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2020-11-16T22:16:57.257 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 147153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2020-11-16T22:16:57.259 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2020-11-16T22:16:57.259 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2020-11-16T22:16:57.260 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2020-11-16T22:16:57.261 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/ 2020-11-16T22:16:57.261 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/dl2k.ko 2020-11-16T22:16:57.261 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/sundance.ko 2020-11-16T22:16:57.262 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dnet.ko 2020-11-16T22:16:57.262 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ec_bhf.ko 2020-11-16T22:16:57.263 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/ 2020-11-16T22:16:57.263 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/ 2020-11-16T22:16:57.263 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 287161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2020-11-16T22:16:57.267 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ethoc.ko 2020-11-16T22:16:57.267 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fealnx.ko 2020-11-16T22:16:57.268 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/ 2020-11-16T22:16:57.268 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2020-11-16T22:16:57.268 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ 2020-11-16T22:16:57.268 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 91697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e100.ko 2020-11-16T22:16:57.269 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/ 2020-11-16T22:16:57.269 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 293081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2020-11-16T22:16:57.273 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/ 2020-11-16T22:16:57.273 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 549313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2020-11-16T22:16:57.280 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/ 2020-11-16T22:16:57.280 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 247433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2020-11-16T22:16:57.283 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/ 2020-11-16T22:16:57.283 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 892665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2020-11-16T22:16:57.295 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/ 2020-11-16T22:16:57.296 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 250857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2020-11-16T22:16:57.298 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/ 2020-11-16T22:16:57.298 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 509433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/igb.ko 2020-11-16T22:16:57.304 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/ 2020-11-16T22:16:57.305 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 90673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2020-11-16T22:16:57.305 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ 2020-11-16T22:16:57.306 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 123905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2020-11-16T22:16:57.307 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ 2020-11-16T22:16:57.307 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 743497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2020-11-16T22:16:57.316 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ 2020-11-16T22:16:57.316 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 147065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2020-11-16T22:16:57.318 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/jme.ko 2020-11-16T22:16:57.319 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/ 2020-11-16T22:16:57.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2020-11-16T22:16:57.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 103057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/skge.ko 2020-11-16T22:16:57.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 130193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/sky2.ko 2020-11-16T22:16:57.322 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/ 2020-11-16T22:16:57.322 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/ 2020-11-16T22:16:57.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 739097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2020-11-16T22:16:57.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 273433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2020-11-16T22:16:57.336 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/ 2020-11-16T22:16:57.336 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2020-11-16T22:16:57.336 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 920849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2020-11-16T22:16:57.347 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ 2020-11-16T22:16:57.347 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8842.ko 2020-11-16T22:16:57.347 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8851_mll.ko 2020-11-16T22:16:57.348 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 94705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2020-11-16T22:16:57.349 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/ 2020-11-16T22:16:57.349 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/ 2020-11-16T22:16:57.349 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2020-11-16T22:16:57.350 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ 2020-11-16T22:16:57.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2020-11-16T22:16:57.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2020-11-16T22:16:57.352 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/ 2020-11-16T22:16:57.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 156657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/s2io.ko 2020-11-16T22:16:57.354 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/ 2020-11-16T22:16:57.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 221913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/vxge.ko 2020-11-16T22:16:57.357 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/ 2020-11-16T22:16:57.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 142057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2020-11-16T22:16:57.359 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/ 2020-11-16T22:16:57.359 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2020-11-16T22:16:57.360 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2020-11-16T22:16:57.360 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/ 2020-11-16T22:16:57.360 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/ 2020-11-16T22:16:57.360 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 242793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2020-11-16T22:16:57.363 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2020-11-16T22:16:57.364 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2020-11-16T22:16:57.364 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 706337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2020-11-16T22:16:57.373 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/ 2020-11-16T22:16:57.373 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/r6040.ko 2020-11-16T22:16:57.374 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/ 2020-11-16T22:16:57.374 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139cp.ko 2020-11-16T22:16:57.374 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 83753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139too.ko 2020-11-16T22:16:57.375 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/atp.ko 2020-11-16T22:16:57.376 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 173241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/r8169.ko 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/ 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/ 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 104689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/ 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 824569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/sfc.ko 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/ 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/sc92031.ko 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/ 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis190.ko 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 64625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis900.ko 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/ 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 62105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/epic100.ko 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2020-11-16T22:16:57.395 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/ 2020-11-16T22:16:57.395 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/ 2020-11-16T22:16:57.395 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2020-11-16T22:16:57.396 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2020-11-16T22:16:57.396 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2020-11-16T22:16:57.396 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 381753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/ 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 106913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/cassini.ko 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 176753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/niu.ko 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sungem.ko 2020-11-16T22:16:57.405 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sunhme.ko 2020-11-16T22:16:57.406 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/ 2020-11-16T22:16:57.406 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2020-11-16T22:16:57.406 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/ 2020-11-16T22:16:57.407 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 78681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/tlan.ko 2020-11-16T22:16:57.407 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/ 2020-11-16T22:16:57.407 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-rhine.ko 2020-11-16T22:16:57.408 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 86073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-velocity.ko 2020-11-16T22:16:57.409 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/ 2020-11-16T22:16:57.409 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5100.ko 2020-11-16T22:16:57.410 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5300.ko 2020-11-16T22:16:57.410 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/ 2020-11-16T22:16:57.410 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2020-11-16T22:16:57.411 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/ 2020-11-16T22:16:57.411 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/defxx.ko 2020-11-16T22:16:57.411 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/ 2020-11-16T22:16:57.412 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 236713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/skfp.ko 2020-11-16T22:16:57.415 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 78057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/geneve.ko 2020-11-16T22:16:57.415 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/ 2020-11-16T22:16:57.416 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 227313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/hv_netvsc.ko 2020-11-16T22:16:57.419 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/ 2020-11-16T22:16:57.419 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/fakelb.ko 2020-11-16T22:16:57.419 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ifb.ko 2020-11-16T22:16:57.419 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ 2020-11-16T22:16:57.419 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 84633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ipvlan.ko 2020-11-16T22:16:57.420 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 57241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvlan.ko 2020-11-16T22:16:57.420 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvtap.ko 2020-11-16T22:16:57.421 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/ 2020-11-16T22:16:57.421 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2020-11-16T22:16:57.421 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bitbang.ko 2020-11-16T22:16:57.421 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-gpio.ko 2020-11-16T22:16:57.422 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio.ko 2020-11-16T22:16:57.422 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mii.ko 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/netconsole.ko 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/nlmon.ko 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/ 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/pcs-xpcs.ko 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ 2020-11-16T22:16:57.423 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/amd.ko 2020-11-16T22:16:57.424 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm-phy-lib.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm7xxx.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm87xx.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/broadcom.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/cicada.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/davicom.ko 2020-11-16T22:16:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/et1011c.ko 2020-11-16T22:16:57.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/icplus.ko 2020-11-16T22:16:57.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/lxt.ko 2020-11-16T22:16:57.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 64985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/marvell.ko 2020-11-16T22:16:57.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/micrel.ko 2020-11-16T22:16:57.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/national.ko 2020-11-16T22:16:57.427 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/phylink.ko 2020-11-16T22:16:57.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/qsemi.ko 2020-11-16T22:16:57.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/realtek.ko 2020-11-16T22:16:57.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/smsc.ko 2020-11-16T22:16:57.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ste10Xp.ko 2020-11-16T22:16:57.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/vitesse.ko 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/ 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/plip.ko 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/bsd_comp.ko 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_async.ko 2020-11-16T22:16:57.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_deflate.ko 2020-11-16T22:16:57.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_mppe.ko 2020-11-16T22:16:57.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_synctty.ko 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppoe.ko 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppox.ko 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pptp.ko 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sb1000.ko 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/ 2020-11-16T22:16:57.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/slip.ko 2020-11-16T22:16:57.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sungem_phy.ko 2020-11-16T22:16:57.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/tap.ko 2020-11-16T22:16:57.432 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/ 2020-11-16T22:16:57.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 102721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team.ko 2020-11-16T22:16:57.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_activebackup.ko 2020-11-16T22:16:57.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_broadcast.ko 2020-11-16T22:16:57.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_loadbalance.ko 2020-11-16T22:16:57.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_random.ko 2020-11-16T22:16:57.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_roundrobin.ko 2020-11-16T22:16:57.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/veth.ko 2020-11-16T22:16:57.434 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/ 2020-11-16T22:16:57.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 116905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/vmxnet3.ko 2020-11-16T22:16:57.435 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 171137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vxlan.ko 2020-11-16T22:16:57.438 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/ 2020-11-16T22:16:57.438 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 143721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/xen-netback.ko 2020-11-16T22:16:57.440 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ 2020-11-16T22:16:57.440 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ntb.ko 2020-11-16T22:16:57.440 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/ 2020-11-16T22:16:57.440 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/ 2020-11-16T22:16:57.440 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 209665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme-core.ko 2020-11-16T22:16:57.442 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 92745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme.ko 2020-11-16T22:16:57.444 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/ 2020-11-16T22:16:57.444 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 130825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport.ko 2020-11-16T22:16:57.445 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_ax88796.ko 2020-11-16T22:16:57.445 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_cs.ko 2020-11-16T22:16:57.446 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 84865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_pc.ko 2020-11-16T22:16:57.446 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_serial.ko 2020-11-16T22:16:57.447 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/ 2020-11-16T22:16:57.447 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/ 2020-11-16T22:16:57.447 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2020-11-16T22:16:57.448 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2020-11-16T22:16:57.448 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2020-11-16T22:16:57.448 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/pci-stub.ko 2020-11-16T22:16:57.448 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/xen-pcifront.ko 2020-11-16T22:16:57.449 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/ 2020-11-16T22:16:57.449 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/i82092.ko 2020-11-16T22:16:57.449 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 150113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia.ko 2020-11-16T22:16:57.450 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_core.ko 2020-11-16T22:16:57.450 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2020-11-16T22:16:57.451 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pd6729.ko 2020-11-16T22:16:57.451 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 111129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/yenta_socket.ko 2020-11-16T22:16:57.452 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/ 2020-11-16T22:16:57.453 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/ 2020-11-16T22:16:57.453 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2020-11-16T22:16:57.453 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/ 2020-11-16T22:16:57.453 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2020-11-16T22:16:57.454 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/ 2020-11-16T22:16:57.454 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/ 2020-11-16T22:16:57.454 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2020-11-16T22:16:57.454 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/ 2020-11-16T22:16:57.454 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/ 2020-11-16T22:16:57.455 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2020-11-16T22:16:57.455 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec.ko 2020-11-16T22:16:57.455 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ 2020-11-16T22:16:57.456 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acer-wmi.ko 2020-11-16T22:16:57.457 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acerhdf.ko 2020-11-16T22:16:57.457 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/alienware-wmi.ko 2020-11-16T22:16:57.457 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/apple-gmux.ko 2020-11-16T22:16:57.457 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-laptop.ko 2020-11-16T22:16:57.458 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-nb-wmi.ko 2020-11-16T22:16:57.458 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-wmi.ko 2020-11-16T22:16:57.458 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/classmate-laptop.ko 2020-11-16T22:16:57.459 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dcdbas.ko 2020-11-16T22:16:57.459 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-smo8800.ko 2020-11-16T22:16:57.662 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-wmi-aio.ko 2020-11-16T22:16:57.662 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell_rbu.ko 2020-11-16T22:16:57.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-laptop.ko 2020-11-16T22:16:57.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-wmi.ko 2020-11-16T22:16:57.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-laptop.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-tablet.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hdaps.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wireless.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wmi.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp_accel.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ibm_rtl.ko 2020-11-16T22:16:57.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-rst.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-smartconnect.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_ips.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_menlow.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/msi-wmi.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/mxm-wmi.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/panasonic-laptop.ko 2020-11-16T22:16:57.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-laptop.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-q10.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 260993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/thinkpad_acpi.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/topstar-laptop.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_haps.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi-bmof.ko 2020-11-16T22:16:57.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi.ko 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/ 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_battery.ko 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_charger.ko 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq2415x_charger.ko 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24190_charger.ko 2020-11-16T22:16:57.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24735-charger.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9030_battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9052-battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2760_battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2780_battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2781_battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2782_battery.ko 2020-11-16T22:16:57.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/gpio-charger.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/isp1704_charger.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/lp8727_charger.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max14577_charger.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17040_battery.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17042_battery.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8903_charger.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8925_power.ko 2020-11-16T22:16:57.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pcf50633-charger.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pda_power.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/sbs-battery.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/smb347-charger.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/test_power.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/tps65090-charger.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_backup.ko 2020-11-16T22:16:57.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_power.ko 2020-11-16T22:16:57.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm8350_power.ko 2020-11-16T22:16:57.681 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/ 2020-11-16T22:16:57.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_common.ko 2020-11-16T22:16:57.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_msr.ko 2020-11-16T22:16:57.681 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/ 2020-11-16T22:16:57.882 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/ 2020-11-16T22:16:57.882 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-gpio.ko 2020-11-16T22:16:57.882 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-ldisc.ko 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps_parport.ko 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/pps_core.ko 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp.ko 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp_kvm.ko 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/ 2020-11-16T22:16:57.883 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm80x.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm860x.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ab3100.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq32k.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq4802.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9052.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9055.ko 2020-11-16T22:16:57.884 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9063.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1286.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 65017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1307.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1374.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1511.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1553.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1672.ko 2020-11-16T22:16:57.885 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1742.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds2404.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds3232.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-em3027.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-fm3130.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl12022.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl1208.ko 2020-11-16T22:16:57.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-lp8788.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m41t80.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t35.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t59.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t86.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max6900.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8907.ko 2020-11-16T22:16:57.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8925.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8997.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8998.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-mc13xxx.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-msm6242.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-palmas.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf2127.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf50633.ko 2020-11-16T22:16:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf85063.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8523.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8563.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8583.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rc5t583.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rp5c01.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rs5c372.ko 2020-11-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rv3029c2.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8025.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8581.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s35390a.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s5m.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-stk17ta8.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps6586x.ko 2020-11-16T22:16:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps65910.ko 2020-11-16T22:16:57.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps80031.ko 2020-11-16T22:16:57.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-v3020.ko 2020-11-16T22:16:57.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm831x.ko 2020-11-16T22:16:58.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm8350.ko 2020-11-16T22:16:58.098 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-x1205.ko 2020-11-16T22:16:58.098 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 80673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-9xxx.ko 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 59617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-sas.ko 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-xxxx.ko 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 132001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/BusLogic.ko 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/a100u2w.ko 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/ 2020-11-16T22:16:58.099 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 286449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/aacraid.ko 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 101329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/advansys.ko 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/ 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 346097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 297009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/ 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 194297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/aic94xx.ko 2020-11-16T22:16:58.100 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/am53c974.ko 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/ 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 116369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/arcmsr.ko 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/atp870u.ko 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/ 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 265265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/ 2020-11-16T22:16:58.101 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1164361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/bfa.ko 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/ 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 252833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/ 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 119577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/bnx2i.ko 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ch.ko 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/ 2020-11-16T22:16:58.102 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 312289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/csiostor.ko 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/ 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/ 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 80249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/ 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 110649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 153137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2020-11-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 76961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dc395x.ko 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dmx3191d.ko 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 79977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dpt_i2o.ko 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/ 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 213561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/esas2r.ko 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esp_scsi.ko 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/ 2020-11-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 78793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/fcoe.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 162897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/libfcoe.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fdomain.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/ 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 280281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/fnic.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 107993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/gdth.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 215137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hpsa.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hptiop.ko 2020-11-16T22:16:58.105 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hv_storvsc.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/imm.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 48489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/initio.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 294193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ipr.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 110105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ips.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/ 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 335225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/isci.ko 2020-11-16T22:16:58.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_tcp.ko 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/ 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 321337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/libfc.ko 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 144753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi.ko 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi_tcp.ko 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/ 2020-11-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 199625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/libsas.ko 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/ 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 2047225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/lpfc.ko 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/ 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 80961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 334937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2020-11-16T22:16:58.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid.ko 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/ 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 652873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/ 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 122993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/mvsas.ko 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvumi.ko 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/ 2020-11-16T22:16:58.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 81921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/ 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 420593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/pm80xx.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 137097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pmcraid.ko 2020-11-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ppa.ko 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla1280.ko 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/ 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1702009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/ 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 610673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2020-11-16T22:16:58.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qlogicfas408.ko 2020-11-16T22:16:58.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/raid_class.ko 2020-11-16T22:16:58.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 238049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_debug.ko 2020-11-16T22:16:58.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 142097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_fc.ko 2020-11-16T22:16:58.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 265049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_iscsi.ko 2020-11-16T22:16:58.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 95017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_sas.ko 2020-11-16T22:16:58.142 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 72697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_spi.ko 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_srp.ko 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ses.ko 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 118033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/st.ko 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/stex.ko 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/ 2020-11-16T22:16:58.143 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 166257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 404305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-core.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pci.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pltfrm.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/virtio_scsi.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/vmw_pvscsi.ko 2020-11-16T22:16:58.144 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/wd719x.ko 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/xen-scsifront.ko 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 127201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ssb.ko 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/ 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/ 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 747873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/ 2020-11-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/tcm_loop.ko 2020-11-16T22:16:58.146 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_file.ko 2020-11-16T22:16:58.146 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_iblock.ko 2020-11-16T22:16:58.146 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 931017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_mod.ko 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_pscsi.ko 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 108177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_user.ko 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/ 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 73449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/tcm_fc.ko 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/ 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/ 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/ 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_powerclamp.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/cyclades.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 79297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ipwireless.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/isicom.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/moxa.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/mxser.ko 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_gsm.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_hdlc.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracerouter.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracesink.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/nozomi.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 85873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/rocket.ko 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/ 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/ 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/8250_dw.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/serial_cs.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_jtaguart.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_uart.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/arc_uart.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/fsl_lpuart.ko 2020-11-16T22:16:58.355 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/ 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 106905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/jsm.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/rp2.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/sc16is7xx.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 119353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 109697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink_gt.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 110737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclinkmp.ko 2020-11-16T22:16:58.356 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/ 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 42305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_aec.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_cif.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_dmem_genirq.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_mf624.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_netx.ko 2020-11-16T22:16:58.357 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pci_generic.ko 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pdrv_genirq.ko 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_sercos3.ko 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/ 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/ 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/c67x00.ko 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ 2020-11-16T22:16:58.358 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 120065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc.ko 2020-11-16T22:16:58.359 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2020-11-16T22:16:58.562 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2020-11-16T22:16:58.562 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2020-11-16T22:16:58.562 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/ 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 85273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-acm.ko 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-wdm.ko 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usblp.ko 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 93385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usbtmc.ko 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ulpi.ko 2020-11-16T22:16:58.563 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/ 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/dwc2_pci.ko 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/ 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-haps.ko 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-pci.ko 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 432665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3.ko 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/ 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/ 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_ether.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_serial.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 92065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 120497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 57481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 59753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 83169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/ 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ether.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_hid.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_printer.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_serial.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_zero.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 143505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/libcomposite.ko 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/ 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/ 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 160057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc_pci.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/fotg210-udc.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/goku_udc.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 54921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/gr_udc.ko 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 70817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 57361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_udc.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 73193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2272.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2280.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 73825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pch_udc.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 66497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 70881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2020-11-16T22:16:58.571 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 144217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/udc-core.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/bcma-hcd.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 128681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/fotg210-hcd.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 76001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/isp116x-hcd.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 88825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/oxu210hp-hcd.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 79145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/r8a66597-hcd.ko 2020-11-16T22:16:58.572 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811-hcd.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811_cs.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ssb-hcd.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 97993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/u132-hcd.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/xhci-plat-hcd.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/ 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/mdc800.ko 2020-11-16T22:16:58.573 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/microtek.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/adutux.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/appledisplay.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cypress_cy7c63.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cytherm.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ehset.ko 2020-11-16T22:16:58.574 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi26.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi62.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ezusb.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ftdi-elan.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/idmouse.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/iowarrior.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/isight_firmware.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ldusb.ko 2020-11-16T22:16:58.575 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/legousbtower.ko 2020-11-16T22:16:58.576 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/lvstest.ko 2020-11-16T22:16:58.576 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/ 2020-11-16T22:16:58.576 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2020-11-16T22:16:58.576 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/trancevibrator.ko 2020-11-16T22:16:58.576 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usb3503.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usblcd.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbsevseg.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbtest.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/uss720.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/yurex.ko 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/ 2020-11-16T22:16:58.577 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 69937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/usbmon.ko 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/ 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 293313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/musb_hdrc.ko 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/ 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-generic.ko 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-isp1301.ko 2020-11-16T22:16:58.578 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-tahvo.ko 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/aircable.ko 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ark3116.ko 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/belkin_sa.ko 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ch341.ko 2020-11-16T22:16:58.579 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cp210x.ko 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cyberjack.ko 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 59169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cypress_m8.ko 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/digi_acceleport.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/empeg.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/f81232.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 174849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ftdi_sio.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/garmin_gps.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 140417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_edgeport.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 124153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_ti.ko 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipaq.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipw.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ir-usb.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 57473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/iuu_phoenix.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 106265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan_pda.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kl5kusb105.ko 2020-11-16T22:16:58.784 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kobil_sct.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mct_u232.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/metro-usb.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7720.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 78913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7840.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mxuport.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/navman.ko 2020-11-16T22:16:58.785 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/omninet.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/opticon.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 176777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/option.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 39665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/oti6858.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/pl2303.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcaux.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcserial.ko 2020-11-16T22:16:58.786 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/quatech2.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/safe_serial.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/sierra.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/spcp8x5.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ssu100.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/symbolserial.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2020-11-16T22:16:58.787 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb-serial-simple.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_debug.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_wwan.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 118233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usbserial.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/visor.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/whiteheat.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/wishbone-serial.ko 2020-11-16T22:16:58.788 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/xsens_mt.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/uas.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-alauda.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-cypress.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-datafab.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-eneub6250.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-freecom.ko 2020-11-16T22:16:58.789 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-isd200.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-jumpshot.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-karma.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-onetouch.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-realtek.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr09.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr55.ko 2020-11-16T22:16:58.790 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-usbat.ko 2020-11-16T22:16:58.791 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 173801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/usb-storage.ko 2020-11-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/ 2020-11-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-core.ko 2020-11-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 71161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-host.ko 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 107913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/vhci-hcd.ko 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/ 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/ 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 149025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/vfio-pci.ko 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 72673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio.ko 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 68809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_iommu_type1.ko 2020-11-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_virqfd.ko 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/ 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 99057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost.ko 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_iotlb.ko 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_net.ko 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 84649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_scsi.ko 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/ 2020-11-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/ 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/88pm860x_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/aat2870_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp5520_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8860_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8870_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/apple_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/as3711_bl.ko 2020-11-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/bd6107.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da903x_bl.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da9052_bl.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/gpio_backlight.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/kb3886_bl.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lcd.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3533_bl.ko 2020-11-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3639_bl.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lv5207lp.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/max8925_bl.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pandora_bl.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pcf50633-backlight.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/platform_lcd.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/wm831x_bl.ko 2020-11-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/ 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/virtio_mem.ko 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/ 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds1wm.ko 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2482.ko 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2490.ko 2020-11-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/matrox_w1.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/w1-gpio.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/ 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2406.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2408.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2413.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2423.ko 2020-11-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2431.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2433.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2780.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2781.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds28e04.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_smem.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_therm.ko 2020-11-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 99985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/wire.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/acquirewdt.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/advantechwdt.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim1535_wdt.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim7101_wdt.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/cpu5wdt.ko 2020-11-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9052_wdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9055_wdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9063_wdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/dw_wdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/eurotechwdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/f71808e_wdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/hpwdt.ko 2020-11-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/i6300esb.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_vendor_support.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_wdt.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ib700wdt.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ibmasr.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ie6xx_wdt.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it8712f_wdt.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it87_wdt.ko 2020-11-16T22:16:59.005 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/kempld_wdt.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/machzwd.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/mena21_wdt.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/menf21bmc_wdt.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/nv_tco.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/of_xilinx_wdt.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pc87413_wdt.ko 2020-11-16T22:16:59.006 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_pci.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_usb.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/retu_wdt.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc60xxwdt.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_epx_c3.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sc1200wdt.ko 2020-11-16T22:16:59.007 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sch311x_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/smsc37b787_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/softdog.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sp5100_tco.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/twl4030_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/via_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83627hf_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83877f_wdt.ko 2020-11-16T22:16:59.008 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83977f_wdt.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wafer5823wdt.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wdt_pci.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm831x_wdt.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm8350_wdt.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/xen_wdt.ko 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/ 2020-11-16T22:16:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-evtchn.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntalloc.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 43137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntdev.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/ 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 178577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-privcmd.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-scsiback.ko 2020-11-16T22:16:59.010 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/ 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/xenfs.ko 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/ 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 147729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/9p.ko 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/ 2020-11-16T22:16:59.011 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/adfs.ko 2020-11-16T22:16:59.214 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/ 2020-11-16T22:16:59.214 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 162417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/affs.ko 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/ 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 972889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/kafs.ko 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/ 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 106865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/autofs4.ko 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/ 2020-11-16T22:16:59.215 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/befs.ko 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/ 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/bfs.ko 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/binfmt_misc.ko 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/ 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 3192737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/btrfs.ko 2020-11-16T22:16:59.216 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/ 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 194401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/cachefiles.ko 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1125913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ceph.ko 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/ 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 2585001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/cifs.ko 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/ 2020-11-16T22:16:59.217 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 96281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/coda.ko 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/ 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/configfs.ko 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/ 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/cramfs.ko 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/ 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 438425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/dlm.ko 2020-11-16T22:16:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/ 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/efs.ko 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/ 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1367417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/f2fs.ko 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/ 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/msdos.ko 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/ 2020-11-16T22:16:59.219 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/freevxfs.ko 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/ 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 317913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/fscache.ko 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/ 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/cuse.ko 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/ 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1193977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/gfs2.ko 2020-11-16T22:16:59.220 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/ 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 141505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/hfs.ko 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/ 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 229553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/hfsplus.ko 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/ 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 193625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/hpfs.ko 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/ 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/isofs.ko 2020-11-16T22:16:59.221 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/ 2020-11-16T22:16:59.222 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 413889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/jfs.ko 2020-11-16T22:16:59.222 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/ 2020-11-16T22:16:59.222 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 268617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/lockd.ko 2020-11-16T22:16:59.222 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/ 2020-11-16T22:16:59.222 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 83041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/minix.ko 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/ 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/ 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 87177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/ 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 64561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2020-11-16T22:16:59.223 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/ 2020-11-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 123673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2020-11-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 899209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfs.ko 2020-11-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 65457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv2.ko 2020-11-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv3.ko 2020-11-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1849921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv4.ko 2020-11-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/ 2020-11-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/grace.ko 2020-11-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_acl.ko 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_ssc.ko 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/ 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1361097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/nfsd.ko 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/ 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 557369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/nilfs2.ko 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/ 2020-11-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-celtic.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-centeuro.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-croatian.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-cyrillic.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-gaelic.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-greek.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-iceland.ko 2020-11-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-inuit.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-roman.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-romanian.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-turkish.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_ascii.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1250.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1251.ko 2020-11-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1255.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp737.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp775.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp850.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp852.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp855.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp857.ko 2020-11-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp860.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp861.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp862.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp863.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp864.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp865.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp866.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp869.ko 2020-11-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp874.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 93241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp932.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 141825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp936.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 164961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp949.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 117337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp950.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_euc-jp.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-1.ko 2020-11-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-13.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-14.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-15.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-2.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-3.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-4.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-5.ko 2020-11-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-6.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-7.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-9.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-r.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-ru.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-u.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_utf8.ko 2020-11-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ 2020-11-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 212417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ntfs.ko 2020-11-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ 2020-11-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ 2020-11-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 235897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2020-11-16T22:16:59.638 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ 2020-11-16T22:16:59.638 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 435145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2020-11-16T22:16:59.638 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 2561401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2.ko 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_user.ko 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stackglue.ko 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/ 2020-11-16T22:16:59.639 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/omfs.ko 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/ 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 288641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/overlay.ko 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 55033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ramoops.ko 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/ 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/qnx4.ko 2020-11-16T22:16:59.640 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/ 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/qnx6.ko 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/ 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_tree.ko 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v1.ko 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v2.ko 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/ 2020-11-16T22:16:59.641 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 521233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/reiserfs.ko 2020-11-16T22:16:59.642 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/ 2020-11-16T22:16:59.642 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/romfs.ko 2020-11-16T22:16:59.642 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/ 2020-11-16T22:16:59.642 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 125585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/squashfs.ko 2020-11-16T22:16:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/ 2020-11-16T22:16:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 86857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/sysv.ko 2020-11-16T22:16:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/ 2020-11-16T22:16:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 249097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/udf.ko 2020-11-16T22:16:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 157929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ufs.ko 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/ 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 3641945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/xfs.ko 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/ 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/ 2020-11-16T22:16:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 56513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/locktorture.ko 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/ 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/test_udelay.ko 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/torture.ko 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc-itu-t.ko 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc64.ko 2020-11-16T22:16:59.671 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc7.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc8.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/ 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libarc4.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libdes.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/interval_tree_test.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/libcrc32c.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lru_cache.ko 2020-11-16T22:16:59.672 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/ 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4_compress.ko 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4hc_compress.ko 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/ 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/cordic.ko 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/memory-notifier-error-inject.ko 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/notifier-error-inject.ko 2020-11-16T22:16:59.673 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/percpu_test.ko 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/pm-notifier-error-inject.ko 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/ 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 141225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/raid6_pq.ko 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/rbtree_test.ko 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/ 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/reed_solomon.ko 2020-11-16T22:16:59.674 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-kstrtox.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-string_helpers.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1714593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_bpf.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_firmware.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 5545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_module.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_user_copy.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_bm.ko 2020-11-16T22:16:59.675 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_fsm.ko 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_kmp.ko 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/ 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/xz_dec_test.ko 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/ 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 259025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/zstd_compress.ko 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/ 2020-11-16T22:16:59.676 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/hwpoison-inject.ko 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/ 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/6lowpan.ko 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_dest.ko 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_fragment.ko 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_hop.ko 2020-11-16T22:16:59.677 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_ipv6.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_mobility.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_routing.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_udp.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/ 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/garp.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/mrp.ko 2020-11-16T22:16:59.678 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/p8022.ko 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/psnap.ko 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/stp.ko 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/ 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 82849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/8021q.ko 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/ 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 208001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet.ko 2020-11-16T22:16:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_rdma.ko 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_virtio.ko 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/ 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 647129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/batman-adv.ko 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/ 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 81361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/br_netfilter.ko 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 610185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/bridge.ko 2020-11-16T22:16:59.680 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_802_3.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_among.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arp.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arpreply.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_dnat.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip6.ko 2020-11-16T22:16:59.681 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_limit.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_log.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark_m.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_nflog.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_pkttype.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_redirect.ko 2020-11-16T22:16:59.682 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_snat.ko 2020-11-16T22:16:59.683 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_stp.ko 2020-11-16T22:16:59.683 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_vlan.ko 2020-11-16T22:16:59.683 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_broute.ko 2020-11-16T22:16:59.683 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_filter.ko 2020-11-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_nat.ko 2020-11-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtables.ko 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nf_log_bridge.ko 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/ 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 836073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/libceph.ko 2020-11-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/ 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 127473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/pktgen.ko 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/ 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 201017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp.ko 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_diag.ko 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv4.ko 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 54585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv6.ko 2020-11-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/ 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 263865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/decnet.ko 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/ 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/dn_rtmsg.ko 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/ 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 144145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/dsa_core.ko 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_brcm.ko 2020-11-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_dsa.ko 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_edsa.ko 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_trailer.ko 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/ 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 91321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/hsr.ko 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ 2020-11-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 220801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154_socket.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ah4.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/esp4.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 51945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/fou.ko 2020-11-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/gre.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/inet_diag.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_gre.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 64593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_tunnel.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_vti.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipcomp.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipip.ko 2020-11-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arp_tables.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arpt_mangle.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arptable_filter.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ip_tables.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 45657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ECN.ko 2020-11-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ah.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_filter.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_mangle.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_nat.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_raw.ko 2020-11-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_security.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_flow_table_ipv4.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_arp.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_ipv4.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2020-11-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2020-11-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_bic.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_dctcp.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_diag.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_highspeed.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_htcp.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_hybla.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_illinois.ko 2020-11-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_lp.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_scalable.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_vegas.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_veno.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_westwood.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_yeah.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tunnel4.ko 2020-11-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_diag.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 40649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_tunnel.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/xfrm4_tunnel.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ah6.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/esp6.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/fou6.ko 2020-11-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 93417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_gre.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 101977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_tunnel.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_udp_tunnel.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_vti.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ipcomp6.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/mip6.ko 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ 2020-11-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6_tables.ko 2020-11-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2020-11-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2020-11-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2020-11-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ah.ko 2020-11-16T22:17:00.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2020-11-16T22:17:00.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_frag.ko 2020-11-16T22:17:00.106 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_mh.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rt.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_filter.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2020-11-16T22:17:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_nat.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_raw.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_security.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_flow_table_ipv6.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_log_ipv6.ko 2020-11-16T22:17:00.108 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 90545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/sit.ko 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/tunnel6.ko 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/xfrm6_tunnel.ko 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/ 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 94849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/af_key.ko 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/ 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 127681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_core.ko 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_debugfs.ko 2020-11-16T22:17:00.110 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_eth.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip6.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_netlink.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ppp.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/ 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/lapb.ko 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/ 2020-11-16T22:17:00.111 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc.ko 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 263433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc2.ko 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/ 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 218937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/mac802154.ko 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/ 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/mpls_gso.ko 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ 2020-11-16T22:17:00.112 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 135633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 101529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 102593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2020-11-16T22:17:00.113 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 103425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 104705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 108489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 110513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2020-11-16T22:17:00.114 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 110633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2020-11-16T22:17:00.115 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2020-11-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_list_set.ko 2020-11-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ 2020-11-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 384017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs.ko 2020-11-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2020-11-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 47153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conncount.ko 2020-11-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 414313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_amanda.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_broadcast.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_ftp.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 118865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_h323.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_irc.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2020-11-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 105089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netlink.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_pptp.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sane.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 61369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sip.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_snmp.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_tftp.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_dup_netdev.ko 2020-11-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 75841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table_inet.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_log_common.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 85969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_amanda.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_ftp.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_irc.ko 2020-11-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 27417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_sip.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_tftp.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 53193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_synproxy_core.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 492617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_tables.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_acct.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_cttimeout.ko 2020-11-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_log.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_osf.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 60217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_queue.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_chain_nat.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_compat.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_counter.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_ct.ko 2020-11-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_dup_netdev.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fib.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fwd_netdev.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_hash.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_limit.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_log.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_masq.ko 2020-11-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_nat.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_numgen.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_osf.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_queue.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_redir.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject_inet.ko 2020-11-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_socket.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_synproxy.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_tproxy.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 91641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/x_tables.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_AUDIT.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CHECKSUM.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CLASSIFY.ko 2020-11-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CONNSECMARK.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CT.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_DSCP.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HL.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HMARK.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_IDLETIMER.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LED.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LOG.ko 2020-11-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_MASQUERADE.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NETMAP.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFLOG.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFQUEUE.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 21489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_RATEEST.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_REDIRECT.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_SECMARK.ko 2020-11-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPMSS.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TEE.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TPROXY.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TRACE.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_addrtype.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_bpf.ko 2020-11-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cgroup.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cluster.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_comment.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connbytes.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlabel.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlimit.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connmark.ko 2020-11-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_conntrack.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cpu.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dccp.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_devgroup.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dscp.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ecn.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_esp.ko 2020-11-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hashlimit.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_helper.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hl.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipcomp.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_iprange.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipvs.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 16273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_l2tp.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_length.ko 2020-11-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_limit.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mac.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mark.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_multiport.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nat.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nfacct.ko 2020-11-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_osf.ko 2020-11-16T22:17:00.538 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_owner.ko 2020-11-16T22:17:00.538 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_physdev.ko 2020-11-16T22:17:00.538 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_pkttype.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_policy.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_quota.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_rateest.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_realm.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_recent.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_sctp.ko 2020-11-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_set.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_socket.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_state.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_statistic.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_string.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpmss.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpudp.ko 2020-11-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_time.ko 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_u32.ko 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/ 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 13329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/netlink_diag.ko 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/ 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 11729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/nsh.ko 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/ 2020-11-16T22:17:00.541 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 367929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/openvswitch.ko 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-geneve.ko 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-gre.ko 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-vxlan.ko 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/ 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/af_packet_diag.ko 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/ 2020-11-16T22:17:00.542 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 123433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/phonet.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 54833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/pn_pep.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/ 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 239729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 158753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_rdma.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_tcp.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/ 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 698057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/rxrpc.ko 2020-11-16T22:17:00.543 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/ 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_csum.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_gact.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 37049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_ipt.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 36497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_mirred.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_nat.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_pedit.ko 2020-11-16T22:17:00.544 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_police.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_simple.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_skbedit.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_vlan.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 24641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_basic.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_bpf.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_cgroup.ko 2020-11-16T22:17:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_flow.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_fw.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_route.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 35065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp6.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 41113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_tcindex.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 54513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_u32.ko 2020-11-16T22:17:00.546 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_cmp.ko 2020-11-16T22:17:00.547 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_ipset.ko 2020-11-16T22:17:00.547 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 31985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_meta.ko 2020-11-16T22:17:00.547 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_nbyte.ko 2020-11-16T22:17:00.547 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_text.ko 2020-11-16T22:17:00.750 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_u32.ko 2020-11-16T22:17:00.750 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_cbq.ko 2020-11-16T22:17:00.750 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_choke.ko 2020-11-16T22:17:00.750 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_codel.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 30513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_drr.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 32857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_dsmark.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq_codel.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_gred.ko 2020-11-16T22:17:00.751 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 46041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hfsc.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 20081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hhf.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 50497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_htb.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_ingress.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_mqprio.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_multiq.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 33361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_netem.ko 2020-11-16T22:17:00.752 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_pie.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_plug.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 22969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_prio.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 44257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_qfq.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 28633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_red.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfb.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 29041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfq.ko 2020-11-16T22:17:00.753 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 26089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_tbf.ko 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 25633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_teql.ko 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/ 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 879937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp.ko 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 18081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp_diag.ko 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/ 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/ 2020-11-16T22:17:00.754 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 334985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 62473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1415977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/sunrpc.ko 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/ 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 644937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/ 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/diag.ko 2020-11-16T22:17:00.755 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 1118753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/tipc.ko 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/ 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/unix_diag.ko 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/ 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 73233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 67177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 77529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock.ko 2020-11-16T22:17:00.756 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 9953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_diag.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_loopback.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/ 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 126641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/x25.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/ 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_algo.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_ipcomp.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 81177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_user.ko 2020-11-16T22:17:00.757 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/ 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/ 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 14473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/irqbypass.ko 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 614204 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 599988 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias.bin 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 7746 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin 2020-11-16T22:17:00.758 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 10228 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.bin 2020-11-16T22:17:00.759 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 63287 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.modinfo 2020-11-16T22:17:00.759 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 146392 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep 2020-11-16T22:17:00.759 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 235760 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep.bin 2020-11-16T22:17:00.759 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 210 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.devname 2020-11-16T22:17:00.759 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 74729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.order 2020-11-16T22:17:00.962 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 574 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.softdep 2020-11-16T22:17:00.962 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 238541 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols 2020-11-16T22:17:00.962 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 289923 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols.bin 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/ 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/ 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/ 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 178 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/changelog.Debian.gz 2020-11-16T22:17:00.963 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r-- root/root 692 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/copyright 2020-11-16T22:17:00.964 DEBUG:teuthology.task.kernel:get_image_version: 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:17:00.965 INFO:teuthology.task.kernel:searching for kernel 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:17:00.965 INFO:teuthology.orchestra.run.smithi106:> egrep '(submenu|menuentry.*5.10.0-rc2-ceph-gac4f7b07f403).*{' /boot/grub/grub.cfg 2020-11-16T22:17:00.974 INFO:teuthology.orchestra.run.smithi106.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:17:00.974 INFO:teuthology.orchestra.run.smithi106.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:17:00.974 INFO:teuthology.orchestra.run.smithi106.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-recovery-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:17:00.975 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2020-11-16T22:17:00.975 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:17:00.976 INFO:teuthology.orchestra.run.smithi106:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2020-11-16T22:17:01.016 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi106.front.sepia.ceph.com to complete... 2020-11-16T22:17:01.947 INFO:teuthology.orchestra.run.smithi106.stderr:Sourcing file `/etc/default/grub' 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2020-11-16T22:17:02.203 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:17:02.216 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:17:02.521 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T22:17:02.534 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi106.stderr:done 2020-11-16T22:17:34.937 INFO:teuthology.misc:Re-opening connections... 2020-11-16T22:17:34.937 INFO:teuthology.misc:trying to connect to ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:17:34.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:17:34.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:17:35.017 INFO:teuthology.orchestra.run.smithi181:> true 2020-11-16T22:17:35.258 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:17:35.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:17:35.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:17:53.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2020-11-16T22:17:53.727 DEBUG:teuthology.misc:waited 18.789788007736206 2020-11-16T22:17:54.728 INFO:teuthology.misc:trying to connect to ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:17:54.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:17:54.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:17:54.807 INFO:teuthology.orchestra.run.smithi175:> true 2020-11-16T22:17:55.038 DEBUG:teuthology.misc:waited 20.10094666481018 2020-11-16T22:17:56.039 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:17:56.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:17:56.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:17:59.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2020-11-16T22:17:59.102 DEBUG:teuthology.misc:waited 24.16518235206604 2020-11-16T22:18:00.103 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:18:00.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:18:00.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:18:02.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2020-11-16T22:18:02.175 DEBUG:teuthology.misc:waited 27.237250328063965 2020-11-16T22:18:03.176 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:18:03.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:18:03.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:18:05.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2020-11-16T22:18:05.246 DEBUG:teuthology.misc:waited 30.30919122695923 2020-11-16T22:18:06.248 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:18:06.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:18:06.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:18:08.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2020-11-16T22:18:08.319 DEBUG:teuthology.misc:waited 33.381248235702515 2020-11-16T22:18:09.319 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:18:09.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:18:09.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:19:09.323 DEBUG:teuthology.orchestra.remote:timed out 2020-11-16T22:19:09.325 DEBUG:teuthology.misc:waited 94.3877227306366 2020-11-16T22:19:10.326 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:19:10.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:19:10.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:19:10.377 INFO:teuthology.orchestra.run.smithi106:> true 2020-11-16T22:19:10.856 DEBUG:teuthology.misc:waited 95.91843438148499 2020-11-16T22:19:11.857 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2020-11-16T22:19:11.858 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.10.0-rc2-ceph-gac4f7b07f403"... 2020-11-16T22:19:11.858 INFO:teuthology.orchestra.run.smithi106:> uname -r 2020-11-16T22:19:11.874 INFO:teuthology.orchestra.run.smithi106.stdout:5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:19:11.875 DEBUG:teuthology.task.kernel:current kernel version is 5.10.0-rc2-ceph-gac4f7b07f403 vs 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:19:11.875 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-16T22:19:11.875 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-16T22:19:12.876 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-16T22:19:12.886 INFO:teuthology.task.internal:Creating test directory... 2020-11-16T22:19:12.886 INFO:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:19:12.892 INFO:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:19:12.894 INFO:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:19:12.905 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-16T22:19:12.913 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-16T22:19:12.926 INFO:teuthology.task.internal:Creating archive directory... 2020-11-16T22:19:12.926 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:19:12.932 INFO:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:19:12.943 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:19:12.958 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-16T22:19:12.966 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-16T22:19:12.967 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:19:13.005 INFO:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:19:13.007 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:19:13.026 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.032 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.035 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.037 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.047 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.066 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:19:13.070 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-16T22:19:13.077 INFO:teuthology.task.internal:Configuring sudo... 2020-11-16T22:19:13.077 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:19:13.082 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:19:13.084 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:19:13.108 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-16T22:19:13.117 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-16T22:19:13.118 INFO:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:19:13.123 INFO:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:19:13.139 INFO:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:19:13.151 INFO:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:19:13.199 INFO:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:19:13.247 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:19:13.248 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.293 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:19:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:19:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:152 bytes copied, 3.1475e-05 s, 4.8 MB/s 2020-11-16T22:19:13.312 INFO:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:19:13.318 INFO:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:19:13.368 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:19:13.368 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.414 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.430 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:19:13.430 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:19:13.431 INFO:teuthology.orchestra.run.smithi175.stderr:152 bytes copied, 3.253e-05 s, 4.7 MB/s 2020-11-16T22:19:13.431 INFO:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:19:13.450 INFO:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:19:13.462 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:19:13.462 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.470 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:19:13.496 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:19:13.496 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:19:13.496 INFO:teuthology.orchestra.run.smithi106.stderr:152 bytes copied, 8.2948e-05 s, 1.8 MB/s 2020-11-16T22:19:13.499 INFO:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2020-11-16T22:19:13.505 INFO:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2020-11-16T22:19:13.506 INFO:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2020-11-16T22:19:13.608 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-16T22:19:13.616 INFO:teuthology.task.internal:Starting timer... 2020-11-16T22:19:13.616 INFO:teuthology.run_tasks:Running task pcp... 2020-11-16T22:19:13.633 INFO:teuthology.run_tasks:Running task selinux... 2020-11-16T22:19:13.667 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-16T22:19:13.667 DEBUG:teuthology.task.selinux:Excluding smithi181: OS 'ubuntu' does not support SELinux 2020-11-16T22:19:13.668 DEBUG:teuthology.task.selinux:Excluding smithi175: OS 'ubuntu' does not support SELinux 2020-11-16T22:19:13.668 DEBUG:teuthology.task.selinux:Excluding smithi106: OS 'ubuntu' does not support SELinux 2020-11-16T22:19:13.668 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-16T22:19:13.668 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-11-16T22:19:13.668 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-16T22:19:13.669 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-16T22:19:13.677 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-11-16T22:19:13.684 INFO:teuthology.repo_utils:Fetching master from origin 2020-11-16T22:19:13.734 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-16T22:19:13.747 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-16T22:19:13.748 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi181.front.sepia.ceph.com,smithi175.front.sepia.ceph.com,smithi106.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-16T22:19:14.047 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-16T22:19:14.813 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-16T22:19:21.064 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-16T22:19:21.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:19:21 +0000 (0:00:00.065) 0:00:00.065 ******* 2020-11-16T22:19:25.259 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:19:25 +0000 (0:00:04.176) 0:00:04.241 ******* 2020-11-16T22:19:29.541 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:19:29 +0000 (0:00:04.281) 0:00:08.523 ******* 2020-11-16T22:20:28.634 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi181.front.sepia.ceph.com] ok: [smithi175.front.sepia.ceph.com] Monday 16 November 2020 22:19:35 +0000 (0:00:05.920) 0:00:14.443 ******* Monday 16 November 2020 22:19:35 +0000 (0:00:00.029) 0:00:14.473 ******* TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} Monday 16 November 2020 22:19:35 +0000 (0:00:00.245) 0:00:14.718 ******* Monday 16 November 2020 22:19:35 +0000 (0:00:00.030) 0:00:14.749 ******* TASK [Gathering Facts] ********************************************************* ok: [smithi106.front.sepia.ceph.com] Monday 16 November 2020 22:19:36 +0000 (0:00:00.352) 0:00:15.101 ******* TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi181.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} Monday 16 November 2020 22:19:36 +0000 (0:00:00.103) 0:00:15.204 ******* changed: [smithi175.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} Monday 16 November 2020 22:19:36 +0000 (0:00:00.041) 0:00:15.246 ******* TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} Monday 16 November 2020 22:19:36 2020-11-16T22:20:28.636 INFO:teuthology.task.ansible.out:+0000 (0:00:00.075) 0:00:15.321 ******* TASK [ansible-managed : Delete the ansible users password.] ******************** changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.070553", "end": "2020-11-16 22:19:36.522122", "rc": 0, "start": "2020-11-16 22:19:36.451569", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} Monday 16 November 2020 22:19:36 +0000 (0:00:00.226) 0:00:15.548 ******* changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.089137", "end": "2020-11-16 22:19:36.543243", "rc": 0, "start": "2020-11-16 22:19:36.454106", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} Monday 16 November 2020 22:19:36 +0000 (0:00:00.041) 0:00:15.590 ******* TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} Monday 16 November 2020 22:19:36 +0000 (0:00:00.132) 0:00:15.722 ******* TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} Monday 16 November 2020 22:19:36 +0000 (0:00:00.102) 0:00:15.825 ******* Monday 16 November 2020 22:19:36 +0000 (0:00:00.030) 0:00:15.856 ******* TASK [ansible-managed : Delete the ansible users password.] ******************** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, 2020-11-16T22:20:28.638 INFO:teuthology.task.ansible.out:"cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.297104", "end": "2020-11-16 22:19:37.197183", "rc": 0, "start": "2020-11-16 22:19:36.900079", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} Monday 16 November 2020 22:19:37 +0000 (0:00:00.376) 0:00:16.233 ******* TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} Monday 16 November 2020 22:19:37 +0000 (0:00:00.209) 0:00:16.443 ******* TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565176.8561163-175005612594874/source", "state": "file", "uid": 0} changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565176.8884652-266731940040913/source", "state": "file", "uid": 0} Monday 16 November 2020 22:19:37 +0000 (0:00:00.148) 0:00:16.591 ******* Monday 16 November 2020 22:19:37 +0000 (0:00:00.033) 0:00:16.624 ******* TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5R 2020-11-16T22:20:28.639 INFO:teuthology.task.ansible.out:daVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZ 2020-11-16T22:20:28.641 INFO:teuthology.task.ansible.out:XE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZ 2020-11-16T22:20:28.642 INFO:teuthology.task.ansible.out:q0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] ********** 2020-11-16T22:20:28.644 INFO:teuthology.task.ansible.out:***** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565177.4742002-136082494825381/source", "state": "file", "uid": 0} Monday 16 November 2020 22:19:38 +0000 (0:00:00.440) 0:00:17.065 ******* TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MU 2020-11-16T22:20:28.645 INFO:teuthology.task.ansible.out:XJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} PLAY [teuthology] ************************************************************** skipping: no hosts matched PLAY [testnodes] *************************************************************** Monday 16 November 2020 22:19:38 +0000 (0:00:00.451) 0:00:17.516 ******* Monday 16 November 2020 22:19:38 +0000 (0:00:00.030) 0:00:17.547 2020-11-16T22:20:28.647 INFO:teuthology.task.ansible.out: ******* Monday 16 November 2020 22:19:38 +0000 (0:00:00.031) 0:00:17.578 ******* TASK [Gathering Facts] ********************************************************* ok: [smithi175.front.sepia.ceph.com] [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Monday 16 November 2020 22:19:39 +0000 (0:00:00.780) 0:00:18.359 ******* TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi175.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.184597", "end": "2020-11-16 22:19:39.712514", "rc": 0, "start": "2020-11-16 22:19:39.527917", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} Monday 16 November 2020 22:19:39 +0000 (0:00:00.389) 0:00:18.748 ******* TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013001", "end": "2020-11-16 22:19:39.897442", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:19:39.884441", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [Gathering Facts] ********************************************************* ok: [smithi106.front.sepia.ceph.com] Monday 16 November 2020 22:19:39 +0000 (0:00:00.213) 0:00:18.962 ******* Monday 16 November 2020 22:19:40 +0000 (0:00:00.047) 0:00:19.009 ******* TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} Monday 16 Novembe 2020-11-16T22:20:28.648 INFO:teuthology.task.ansible.out:r 2020 22:19:40 +0000 (0:00:00.041) 0:00:19.050 ******* 2020-11-16T22:20:28.659 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-16T22:20:28.661 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] 2020-11-16T22:20:28.666 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-16T22:20:28.668 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.149587", "end": "2020-11-16 22:19:40.326309", "rc": 0, "start": "2020-11-16 22:19:40.176722", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:20:28.706 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:48.638) 0:01:07.689 ******* 2020-11-16T22:20:28.737 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:20:28.739 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:28.770 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.063) 0:01:07.752 ******* 2020-11-16T22:20:28.804 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.034) 0:01:07.787 ******* 2020-11-16T22:20:28.810 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T22:20:28.811 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:28.845 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.040) 0:01:07.828 ******* 2020-11-16T22:20:28.872 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T22:20:28.874 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:28.906 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.060) 0:01:07.888 ******* 2020-11-16T22:20:28.912 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T22:20:28.913 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014643", "end": "2020-11-16 22:20:28.878899", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:20:28.864256", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:20:28.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.045) 0:01:07.933 ******* 2020-11-16T22:20:28.956 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T22:20:28.958 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:28.995 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:28 +0000 (0:00:00.044) 0:01:07.978 ******* 2020-11-16T22:20:29.002 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:20:29.004 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T22:20:29.042 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.047) 0:01:08.025 ******* 2020-11-16T22:20:29.048 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T22:20:29.050 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.039) 0:01:08.065 ******* 2020-11-16T22:20:29.088 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:20:29.089 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.129 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.046) 0:01:08.111 ******* 2020-11-16T22:20:29.136 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:20:29.138 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.138 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.139 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi181.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.152107", "end": "2020-11-16 22:20:29.111800", "rc": 0, "start": "2020-11-16 22:20:28.959693", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.140 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677 2020-11-16T22:20:29.141 INFO:teuthology.task.ansible.out:897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.173 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.044) 0:01:08.156 ******* 2020-11-16T22:20:29.180 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.181 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.183 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.183 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYl 2020-11-16T22:20:29.184 INFO:teuthology.task.ansible.out:hLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreD 2020-11-16T22:20:29.186 INFO:teuthology.task.ansible.out:KF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item" 2020-11-16T22:20:29.187 INFO:teuthology.task.ansible.out:, "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.225 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.052) 0:01:08.208 ******* 2020-11-16T22:20:29.233 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.233 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.235 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.236 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.237 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.237 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.239 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef 2020-11-16T22:20:29.240 INFO:teuthology.task.ansible.out:168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.249 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.250 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:29.454 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:29 +0000 (0:00:00.063) 0:01:08.272 ******* skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea 2020-11-16T22:20:29.456 INFO:teuthology.task.ansible.out:304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaR 2020-11-16T22:20:29.458 INFO:teuthology.task.ansible.out:zabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop X 2020-11-16T22:20:29.459 INFO:teuthology.task.ansible.out:AUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:29 +0000 (0:00:00.062) 0:01:08.334 ******* TASK [users : Check ho 2020-11-16T22:20:29.461 INFO:teuthology.task.ansible.out:st's /keys-repo-sha1 sentinel file] ********************** changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012443", "end": "2020-11-16 22:20:29.313527", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:20:29.301084", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851e 2020-11-16T22:20:29.663 INFO:teuthology.task.ansible.out:c855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"n 2020-11-16T22:20:29.665 INFO:teuthology.task.ansible.out:ame": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:29 +0000 (0:00:00.045) 0:01:08.380 ******* skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7 2020-11-16T22:20:29.867 INFO:teuthology.task.ansible.out:f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"n 2020-11-16T22:20:29.869 INFO:teuthology.task.ansible.out:ame": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:30.070 INFO:teuthology.task.ansible.out: skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result 2020-11-16T22:20:30.072 INFO:teuthology.task.ansible.out: was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": 2020-11-16T22:20:30.275 INFO:teuthology.task.ansible.out:false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix 2020-11-16T22:20:30.277 INFO:teuthology.task.ansible.out:@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139 2020-11-16T22:20:30.483 INFO:teuthology.task.ansible.out:fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "c 2020-11-16T22:20:30.485 INFO:teuthology.task.ansible.out:hanged": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smith 2020-11-16T22:20:30.691 INFO:teuthology.task.ansible.out:i175.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} sk 2020-11-16T22:20:30.693 INFO:teuthology.task.ansible.out:ipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'nam 2020-11-16T22:20:30.899 INFO:teuthology.task.ansible.out:e': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba 2020-11-16T22:20:30.901 INFO:teuthology.task.ansible.out:0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:30.901 INFO:teuthology.task.ansible.out: 2020-11-16T22:20:30.903 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:31.106 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'shyukr 2020-11-16T22:20:31.108 INFO:teuthology.task.ansible.out:i', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643ea 2020-11-16T22:20:31.109 INFO:teuthology.task.ansible.out:ca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": 2020-11-16T22:20:31.319 INFO:teuthology.task.ansible.out: "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item= 2020-11-16T22:20:31.321 INFO:teuthology.task.ansible.out:{'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaum 2020-11-16T22:20:31.523 INFO:teuthology.task.ansible.out:OyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f9 2020-11-16T22:20:31.525 INFO:teuthology.task.ansible.out:3d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "it 2020-11-16T22:20:31.727 INFO:teuthology.task.ansible.out:em", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditio 2020-11-16T22:20:31.729 INFO:teuthology.task.ansible.out:nal result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => { 2020-11-16T22:20:31.931 INFO:teuthology.task.ansible.out:"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": 2020-11-16T22:20:31.932 INFO:teuthology.task.ansible.out: "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:31.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:31 +0000 (0:00:02.587) 0:01:10.968 ******* 2020-11-16T22:20:32.019 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.033) 0:01:11.002 ******* 2020-11-16T22:20:32.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.036) 0:01:11.039 ******* 2020-11-16T22:20:32.067 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.068 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.070 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.278 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441 2020-11-16T22:20:32.280 INFO:teuthology.task.ansible.out:b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph 2020-11-16T22:20:32.281 INFO:teuthology.task.ansible.out:.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:32 +0000 (0:00:00.050) 0:01:11.089 ******* skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smi 2020-11-16T22:20:32.487 INFO:teuthology.task.ansible.out:thfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefb 2020-11-16T22:20:32.489 INFO:teuthology.task.ansible.out:d6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c 2020-11-16T22:20:32.691 INFO:teuthology.task.ansible.out:50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9b 2020-11-16T22:20:32.693 INFO:teuthology.task.ansible.out:GMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1c 2020-11-16T22:20:32.895 INFO:teuthology.task.ansible.out:f401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:32 +0000 (0:00:00.044) 0:01:11.134 ******* skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@ 2020-11-16T22:20:32.897 INFO:teuthology.task.ansible.out:redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80I 2020-11-16T22:20:32.897 INFO:teuthology.task.ansible.out:CNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.899 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.899 INFO:teuthology.task.ansible.out:skipping: [smithi106.fr 2020-11-16T22:20:32.901 INFO:teuthology.task.ansible.out:ont.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen 2020-11-16T22:20:32.902 INFO:teuthology.task.ansible.out:@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2020-11-16T22:20:32.904 INFO:teuthology.task.ansible.out:2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_l 2020-11-16T22:20:33.107 INFO:teuthology.task.ansible.out:oop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller 2020-11-16T22:20:33.109 INFO:teuthology.task.ansible.out:@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed2 2020-11-16T22:20:33.110 INFO:teuthology.task.ansible.out:1b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02b 2020-11-16T22:20:33.112 INFO:teuthology.task.ansible.out:fed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280 2020-11-16T22:20:33.319 INFO:teuthology.task.ansible.out:db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "ch 2020-11-16T22:20:33.321 INFO:teuthology.task.ansible.out:anged": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d57979 2020-11-16T22:20:33.322 INFO:teuthology.task.ansible.out:3601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) = 2020-11-16T22:20:33.324 INFO:teuthology.task.ansible.out:> {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.324 INFO:teuthology.task.ansible.out: 2020-11-16T22:20:33.326 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.326 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.326 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRX 2020-11-16T22:20:33.328 INFO:teuthology.task.ansible.out:lHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqM 2020-11-16T22:20:33.535 INFO:teuthology.task.ansible.out:YwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN 2020-11-16T22:20:33.537 INFO:teuthology.task.ansible.out:4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg 2020-11-16T22:20:33.538 INFO:teuthology.task.ansible.out: ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": 2020-11-16T22:20:33.540 INFO:teuthology.task.ansible.out:"Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622 2020-11-16T22:20:33.541 INFO:teuthology.task.ansible.out:f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@mileston 2020-11-16T22:20:33.543 INFO:teuthology.task.ansible.out:es utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc0 2020-11-16T22:20:33.747 INFO:teuthology.task.ansible.out:01c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106. 2020-11-16T22:20:33.749 INFO:teuthology.task.ansible.out:front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@m 2020-11-16T22:20:33.750 INFO:teuthology.task.ansible.out:acprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.751 INFO:teuthology.task.ansible.out: 2020-11-16T22:20:33.752 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.752 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.753 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ff 2020-11-16T22:20:33.754 INFO:teuthology.task.ansible.out:ddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.755 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda 2020-11-16T22:20:33.756 INFO:teuthology.task.ansible.out:086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.756 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localho 2020-11-16T22:20:33.758 INFO:teuthology.task.ansible.out:st.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 2020-11-16T22:20:33.759 INFO:teuthology.task.ansible.out:7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:01.660) 0:01:12.795 ******* 2020-11-16T22:20:34.015 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.037) 0:01:12.833 ******* Monday 16 November 2020 22:20:33 +0000 (0:00:00.032) 0:01:12.866 ******* TASK [users : Create all users without sudo access.] *************************** TASK [users : Clone the keys repo] ********************************************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:33 +0000 (0:00:00.049) 0:01:12.915 ******* Monday 16 November 2020 22:20:33 +0000 (0:00:00.032) 0:01:12.947 ******* TASK [users : Filter the managed_users list] *********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.017 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.052) 0:01:13.000 ******* 2020-11-16T22:20:34.050 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.032) 0:01:13.033 ******* 2020-11-16T22:20:34.057 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-16T22:20:34.059 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.099 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.048) 0:01:13.081 ******* 2020-11-16T22:20:34.132 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.033) 0:01:13.115 ******* 2020-11-16T22:20:34.138 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T22:20:34.140 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.183 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.051) 0:01:13.166 ******* 2020-11-16T22:20:34.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.033) 0:01:13.199 ******* 2020-11-16T22:20:34.223 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T22:20:34.225 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.264 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.047) 0:01:13.247 ******* 2020-11-16T22:20:34.298 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.033) 0:01:13.280 ******* 2020-11-16T22:20:34.305 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T22:20:34.307 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.048) 0:01:13.329 ******* 2020-11-16T22:20:34.381 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.034) 0:01:13.363 ******* 2020-11-16T22:20:34.388 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.389 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:20:34.390 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.390 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.390 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.391 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.391 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.428 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.047) 0:01:13.410 ******* 2020-11-16T22:20:34.436 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.437 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.439 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.439 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.439 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.441 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.452 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:20:34.454 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.464 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:20:34.466 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b 2020-11-16T22:20:34.467 INFO:teuthology.task.ansible.out:19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove 2020-11-16T22:20:34.469 INFO:teuthology.task.ansible.out:revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.478 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.480 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.480 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:20:34.481 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed 2020-11-16T22:20:34.483 INFO:teuthology.task.ansible.out:": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.492 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.493 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.494 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCz 2020-11-16T22:20:34.495 INFO:teuthology.task.ansible.out:WYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e 2020-11-16T22:20:34.497 INFO:teuthology.task.ansible.out:34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.506 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:20:34.507 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.518 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.519 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55 2020-11-16T22:20:34.723 INFO:teuthology.task.ansible.out:rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_rea 2020-11-16T22:20:34.725 INFO:teuthology.task.ansible.out:son": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "cha 2020-11-16T22:20:34.931 INFO:teuthology.task.ansible.out:nged": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": fals 2020-11-16T22:20:34.933 INFO:teuthology.task.ansible.out:e, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2v 2020-11-16T22:20:35.139 INFO:teuthology.task.ansible.out:xZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping 2020-11-16T22:20:35.141 INFO:teuthology.task.ansible.out:: [smithi106.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175. 2020-11-16T22:20:35.347 INFO:teuthology.task.ansible.out:front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Condition 2020-11-16T22:20:35.349 INFO:teuthology.task.ansible.out:al result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] * 2020-11-16T22:20:35.551 INFO:teuthology.task.ansible.out:******************************************* skipping: [smithi106.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668 2020-11-16T22:20:35.551 INFO:teuthology.task.ansible.out:079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:35.553 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:35.754 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.s 2020-11-16T22:20:35.756 INFO:teuthology.task.ansible.out:epia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} TASK [users 2020-11-16T22:20:35.758 INFO:teuthology.task.ansible.out: : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbP 2020-11-16T22:20:35.963 INFO:teuthology.task.ansible.out:KJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f96 2020-11-16T22:20:35.965 INFO:teuthology.task.ansible.out:85dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} TA 2020-11-16T22:20:36.167 INFO:teuthology.task.ansible.out:SK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de 2020-11-16T22:20:36.169 INFO:teuthology.task.ansible.out:53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi175.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: 2020-11-16T22:20:36.375 INFO:teuthology.task.ansible.out: [smithi181.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SK 2020-11-16T22:20:36.377 INFO:teuthology.task.ansible.out:arQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed" 2020-11-16T22:20:36.578 INFO:teuthology.task.ansible.out:: false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayto 2020-11-16T22:20:36.580 INFO:teuthology.task.ansible.out:n@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364 2020-11-16T22:20:36.581 INFO:teuthology.task.ansible.out:b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2 2020-11-16T22:20:36.783 INFO:teuthology.task.ansible.out:"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={' 2020-11-16T22:20:36.785 INFO:teuthology.task.ansible.out:name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c372 2020-11-16T22:20:36.988 INFO:teuthology.task.ansible.out:9eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"an 2020-11-16T22:20:36.989 INFO:teuthology.task.ansible.out:sible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item= 2020-11-16T22:20:37.191 INFO:teuthology.task.ansible.out:{'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "o 2020-11-16T22:20:37.193 INFO:teuthology.task.ansible.out:vpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:37.394 INFO:teuthology.task.ansible.out: skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchang 2020-11-16T22:20:37.396 INFO:teuthology.task.ansible.out:ir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4Fi 2020-11-16T22:20:37.398 INFO:teuthology.task.ansible.out:Q ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e 2020-11-16T22:20:37.603 INFO:teuthology.task.ansible.out:53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var" 2020-11-16T22:20:37.605 INFO:teuthology.task.ansible.out:: "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] 2020-11-16T22:20:37.807 INFO:teuthology.task.ansible.out:=> (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0b 2020-11-16T22:20:37.809 INFO:teuthology.task.ansible.out:e8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_ 2020-11-16T22:20:38.019 INFO:teuthology.task.ansible.out:var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia. 2020-11-16T22:20:38.021 INFO:teuthology.task.ansible.out:ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3 2020-11-16T22:20:38.223 INFO:teuthology.task.ansible.out:FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.224 INFO:teuthology.task.ansible.out: skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.225 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05 2020-11-16T22:20:38.226 INFO:teuthology.task.ansible.out:ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.430 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9 2020-11-16T22:20:38.432 INFO:teuthology.task.ansible.out:e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9 2020-11-16T22:20:38.433 INFO:teuthology.task.ansible.out:dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:38 +0000 (0:00:03.875) 0:01:17.286 ******* Monday 16 November 2020 22:20:38 +0000 (0:00:00.031) 0:01:17.318 ******* Monday 16 N 2020-11-16T22:20:38.639 INFO:teuthology.task.ansible.out:ovember 2020 22:20:38 +0000 (0:00:00.032) 0:01:17.350 ******* TASK [users : Create all users without sudo access.] *************************** Monday 16 November 2020 22:20:38 +0000 (0:00:00.062) 0:01:17.413 ******* TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:38 +0000 (0:00:00.066) 0:01:17.479 ******* TASK [users : Clone the keys repo] ********************************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:38 +0000 (0:00:00.060) 0:01:17.540 ******* TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:38 +0000 (0:00:00.060) 0:01:17.601 ******* 2020-11-16T22:20:38.645 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T22:20:38.647 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.061) 0:01:17.662 ******* 2020-11-16T22:20:38.706 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-16T22:20:38.708 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.739 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.059) 0:01:17.722 ******* 2020-11-16T22:20:38.767 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:20:38.768 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.970 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} Monday 16 November 2020 22:20:38 +0000 (0:00:00.200) 0:01:17.922 ******* TASK [users : Remove revoked users] ******************************************** skipping: [smithi181.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=alfredodeza) => {"an 2020-11-16T22:20:38.972 INFO:teuthology.task.ansible.out:sible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [users : Remove revoked users] ******************************************** skipping: [smithi181.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} TASK [common : Log th 2020-11-16T22:20:38.973 INFO:teuthology.task.ansible.out:e OS name, version and release] *************************** ok: [smithi175.front.sepia.ceph.com] => { "msg": "Host smithi175.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [users : Remove revoked users] ******************************************** skipping: [smithi181.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.174 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.160) 0:01:18.083 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.032) 0:01:18.115 ******* skipping: [smithi181.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "C 2020-11-16T22:20:39.176 INFO:teuthology.task.ansible.out:onditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditiona 2020-11-16T22:20:39.379 INFO:teuthology.task.ansible.out:l result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi106.front.sepia.ceph.com] => { "msg": "Host smithi106.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" 2020-11-16T22:20:39.381 INFO:teuthology.task.ansible.out: } Monday 16 November 2020 22:20:39 +0000 (0:00:00.059) 0:01:18.174 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.031) 0:01:18.206 ******* TASK [common : Including major version specific variables.] ******************** ok: [smithi175.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.046) 0:01:18.253 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.284 ******* TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.045) 0:01:18.329 ******* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.385 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.038) 0:01:18.368 ******* 2020-11-16T22:20:39.391 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:20:39.595 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.044) 0:01:18.412 ******* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.039) 0:01:18.452 ******* TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.051) 0:01:18.503 ******* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} Monday 16 November 2020 22:20:39 +0000 (0:00:00.041) 0:01:18.544 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.574 ******* 2020-11-16T22:20:39.597 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T22:20:39.599 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.638 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.046) 0:01:18.620 ******* 2020-11-16T22:20:39.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.651 ******* 2020-11-16T22:20:39.675 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T22:20:39.677 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => { "msg": "Host smithi181.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.046) 0:01:18.698 ******* 2020-11-16T22:20:39.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.728 ******* 2020-11-16T22:20:39.751 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.753 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:39.792 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.046) 0:01:18.774 ******* 2020-11-16T22:20:39.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.805 ******* 2020-11-16T22:20:39.829 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:20:39.831 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.034 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.046) 0:01:18.852 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.029) 0:01:18.882 ******* TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:39 +0000 (0:00:00.045) 0:01:18.927 ******* Monday 16 November 2020 22:20:39 +0000 (0:00:00.030) 0:01:18.958 ******* TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi106.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} M 2020-11-16T22:20:40.036 INFO:teuthology.task.ansible.out:onday 16 November 2020 22:20:40 +0000 (0:00:00.050) 0:01:19.008 ******* 2020-11-16T22:20:40.055 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.029) 0:01:19.037 ******* 2020-11-16T22:20:40.060 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:20:40.061 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.100 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.045) 0:01:19.083 ******* 2020-11-16T22:20:40.132 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.031) 0:01:19.115 ******* 2020-11-16T22:20:40.138 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:20:40.140 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.178 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.046) 0:01:19.161 ******* 2020-11-16T22:20:40.207 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.028) 0:01:19.190 ******* 2020-11-16T22:20:40.212 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T22:20:40.214 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:40.418 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.044) 0:01:19.234 ******* Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.265 ******* TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:40 +0000 (0:00:00.048) 0:01:19.313 ******* Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.344 ******* TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi175.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:40 +0000 (0:00:00.045) 0:01:19.389 ******* 2020-11-16T22:20:40.437 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.420 ******* 2020-11-16T22:20:40.443 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T22:20:40.445 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.487 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.049) 0:01:19.469 ******* 2020-11-16T22:20:40.518 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.031) 0:01:19.501 ******* 2020-11-16T22:20:40.524 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T22:20:40.526 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.565 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.046) 0:01:19.547 ******* 2020-11-16T22:20:40.595 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.578 ******* 2020-11-16T22:20:40.602 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T22:20:40.604 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.642 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.047) 0:01:19.625 ******* 2020-11-16T22:20:40.673 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.656 ******* 2020-11-16T22:20:40.875 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:40 +0000 (0:00:00.046) 0:01:19.702 ******* Monday 16 November 2020 22:20:40 +0000 (0:00:00.031) 0:01:19.734 ******* TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:40 +0000 (0:00:00.045) 0:01:19.779 ******* Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.810 ******* TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:40 +0000 (0:00:00.046) 0:01:19.857 ******* 2020-11-16T22:20:40.905 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:01:19.887 ******* 2020-11-16T22:20:40.911 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:20:40.912 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.955 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.049) 0:01:19.937 ******* 2020-11-16T22:20:40.986 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.031) 0:01:19.969 ******* 2020-11-16T22:20:40.991 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:20:40.993 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.031 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.045) 0:01:20.014 ******* 2020-11-16T22:20:41.061 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:01:20.044 ******* 2020-11-16T22:20:41.067 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T22:20:41.068 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.044) 0:01:20.089 ******* Monday 16 November 2020 22:20:41 +0000 (0:00:00.029) 0:01:20.118 ******* TASK [common : Register with subscription-manager.] **************************** skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:41 +0000 (0:00:00.046) 0:01:20.165 ******* Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:01:20.195 ******* TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi181.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:41 +0000 (0:00:00.045) 0:01:20.240 ******* 2020-11-16T22:20:41.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.029) 0:01:20.269 ******* 2020-11-16T22:20:41.292 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T22:20:41.293 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:41.332 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.045) 0:01:20.315 ******* 2020-11-16T22:20:41.367 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.034) 0:01:20.350 ******* 2020-11-16T22:20:41.374 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T22:20:41.376 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.048) 0:01:20.398 ******* 2020-11-16T22:20:41.448 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.032) 0:01:20.430 ******* 2020-11-16T22:20:41.455 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T22:20:41.456 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.495 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.046) 0:01:20.477 ******* 2020-11-16T22:20:41.528 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.033) 0:01:20.511 ******* 2020-11-16T22:20:41.731 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:41 +0000 (0:00:00.047) 0:01:20.559 ******* Monday 16 November 2020 22:20:41 +0000 (0:00:00.032) 0:01:20.591 ******* TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 2020-11-16T22:20:41.733 INFO:teuthology.task.ansible.out: 22:20:41 +0000 (0:00:00.047) 0:01:20.639 ******* Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:01:20.670 ******* TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi106.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.734 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.046) 0:01:20.717 ******* 2020-11-16T22:20:41.764 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:01:20.747 ******* 2020-11-16T22:20:41.770 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T22:20:41.772 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.810 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.045) 0:01:20.792 ******* 2020-11-16T22:20:41.846 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.035) 0:01:20.828 ******* 2020-11-16T22:20:42.051 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:41 +0000 (0:00:00.040) 0:01:20.869 ******* TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:41 +0000 (0:00:00.041) 0:01:20.911 ******* TASK [common : Register with subscription-manager.] **************************** skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:41 +0000 (0:00:00.040) 0:01:20.951 ******* TASK [common : Clean yum cache] ************************************************ skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:01:20.991 ******* TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.024504", "end": "2020-11-16 22:20:42.002051", "rc": 0, "start": "2020-11-16 22:20:41.977547", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:20:42.254 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.047) 0:01:21.039 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.069 ******* TASK [common : Get list of enab 2020-11-16T22:20:42.459 INFO:teuthology.task.ansible.out:led RHSM repos] ********************************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.055) 0:01:21.125 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.036) 0:01:21.161 ******* TASK [common : List CDN releases available to system] ************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.046) 0:01:21.207 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.238 ******* TASK [common : Store list of enabled repos] ************************************ skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.045) 0:01:21.284 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.314 ******* TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason" 2020-11-16T22:20:42.867 INFO:teuthology.task.ansible.out:: "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.050) 0:01:21.364 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.395 ******* TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.046) 0:01:21.442 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.473 ******* TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.047) 0:01:21.520 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.031) 0:01:21.551 ******* TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.046) 0:01:21.597 ******* Monday 16 November 2020 22:20:42 +0000 (0:00:00.030) 0:01:21.628 ******* TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi181.front.sep 2020-11-16T22:20:43.071 INFO:teuthology.task.ansible.out:ia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:42 +0000 (0:00:00.042) 0:01:21.671 ******* TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.041) 0:01:21.713 ******* TASK [common : Clean yum cache] ************************************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:01:21.753 ******* TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:01:21.792 ******* TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:01:21.832 ******* TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:01:21.872 ******* TASK [common : Store list of enabled repos] ************************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.044) 0:01:21.917 ******* TASK [common : Remove old apt-mirror 2020-11-16T22:20:43.275 INFO:teuthology.task.ansible.out:repository definition.] ******************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:01:21.956 ******* TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:01:21.997 ******* TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:01:22.037 ******* TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.078 ******* TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansibl 2020-11-16T22:20:43.277 INFO:teuthology.task.ansible.out:e_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:01:22.119 ******* TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.160 ******* TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi175.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.202 ******* TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.243 ******* TASK [common : Clean yum cache] ************************************************ skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.302 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.285 ******* 2020-11-16T22:20:43.507 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.042) 0:01:22.327 ******* TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.059) 0:01:22.387 ******* TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.059) 0:01:22.446 ******* TASK [common : Get the current timezone] *************************************** ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014591", "end": "2020-11-16 22:20:43.444209", "rc": 0, "start": "2020-11-16 22:20:43.429618", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} Monday 16 November 2020 22:20:43 +0000 (0:00:00.042) 0:01:22.488 ******* 2020-11-16T22:20:43.512 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:20:43.513 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.551 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.045) 0:01:22.534 ******* 2020-11-16T22:20:43.557 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:20:43.558 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.593 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.575 ******* 2020-11-16T22:20:43.599 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:20:43.601 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.724630", "end": "2020-11-16 22:20:43.500980", "rc": 0, "start": "2020-11-16 22:20:42.776350", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi181.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.802 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.048) 0:01:22.624 ******* Monday 16 November 2020 22:20:43 +0000 (0:00:00.030) 0:01:22.655 ******* TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.047) 0:01:22.702 ******* Monday 16 November 2020 22:20:43 +0000 (0:00:00.030) 0:01:22.733 ******* TASK [common : Clean yum cache] ************************************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:20:43 +0000 (0:00:00.042) 0:01:22.776 ******* TASK [common : Get the current timezone] *************************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, 2020-11-16T22:20:43.804 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.833 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:01:22.816 ******* 2020-11-16T22:20:43.860 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:20:43.862 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.059) 0:01:22.876 ******* 2020-11-16T22:20:43.919 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:20:43.921 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.953 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.059) 0:01:22.935 ******* 2020-11-16T22:20:43.960 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:20:43.962 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.018540", "end": "2020-11-16 22:20:43.932680", "rc": 0, "start": "2020-11-16 22:20:43.914140", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:20:43.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:01:22.977 ******* 2020-11-16T22:20:44.000 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:20:44.002 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean apt cache] ************************************************ [WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.091637", "end": "2020-11-16 22:20:43.966010", "rc": 0, "start": "2020-11-16 22:20:43.874373", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:44.041 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.047) 0:01:23.024 ******* 2020-11-16T22:20:44.071 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.030) 0:01:23.054 ******* 2020-11-16T22:20:44.078 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:20:44.079 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.044) 0:01:23.098 ******* 2020-11-16T22:20:44.122 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:20:44.123 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.157 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.040) 0:01:23.139 ******* 2020-11-16T22:20:44.167 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:20:44.169 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.206 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.048) 0:01:23.188 ******* 2020-11-16T22:20:44.233 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:20:44.235 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.268 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.062) 0:01:23.250 ******* 2020-11-16T22:20:44.295 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:20:44.296 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.328 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.060) 0:01:23.311 ******* 2020-11-16T22:20:44.356 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:20:44.357 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.389 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.061) 0:01:23.372 ******* 2020-11-16T22:20:44.416 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:20:44.417 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.449 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.059) 0:01:23.432 ******* 2020-11-16T22:20:44.475 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:20:44.477 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.059) 0:01:23.491 ******* 2020-11-16T22:20:44.514 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:20:44.515 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.218960", "end": "2020-11-16 22:20:44.500358", "rc": 0, "start": "2020-11-16 22:20:44.281398", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:44.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.041) 0:01:23.532 ******* 2020-11-16T22:20:44.577 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:20:44.578 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.059) 0:01:23.592 ******* 2020-11-16T22:20:44.771 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:20:44.773 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.041148", "end": "2020-11-16 22:20:44.771703", "rc": 0, "start": "2020-11-16 22:20:44.730555", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:44.805 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.194) 0:01:23.787 ******* 2020-11-16T22:20:45.502 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:20:45.504 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.873521", "end": "2020-11-16 22:20:45.500433", "rc": 0, "start": "2020-11-16 22:20:44.626912", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:45.543 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.738) 0:01:24.526 ******* 2020-11-16T22:20:45.570 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:20:45.571 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.060) 0:01:24.586 ******* 2020-11-16T22:20:45.795 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:20:45.797 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.067426", "end": "2020-11-16 22:20:45.790704", "rc": 0, "start": "2020-11-16 22:20:45.723278", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:45.836 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.232) 0:01:24.819 ******* 2020-11-16T22:20:50.772 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:20:50.774 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605565248, "cache_updated": true, "changed": true} 2020-11-16T22:20:50.806 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:04.970) 0:01:29.789 ******* 2020-11-16T22:20:51.669 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:20:51.670 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:20:51.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:51 +0000 (0:00:00.895) 0:01:30.684 ******* 2020-11-16T22:20:51.728 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:20:51.729 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:51.761 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:51 +0000 (0:00:00.059) 0:01:30.744 ******* 2020-11-16T22:20:51.778 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:20:51.779 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605565249, "cache_updated": true, "changed": true} 2020-11-16T22:20:51.813 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:51 +0000 (0:00:00.052) 0:01:30.796 ******* 2020-11-16T22:20:52.056 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:20:52.058 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:20:52.093 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.280) 0:01:31.076 ******* 2020-11-16T22:20:52.316 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:20:52.318 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605565249, "cache_updated": true, "changed": true} 2020-11-16T22:20:52.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.263) 0:01:31.339 ******* 2020-11-16T22:20:52.664 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:20:52.665 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:20:52.698 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.341) 0:01:31.681 ******* 2020-11-16T22:20:52.725 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:20:52.726 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:52.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.059) 0:01:31.740 ******* 2020-11-16T22:20:52.941 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:20:52.942 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:20:52.975 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.217) 0:01:31.958 ******* 2020-11-16T22:20:53.054 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:20:53.056 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:20:53.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.113) 0:01:32.072 ******* 2020-11-16T22:20:53.293 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:53.294 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:20:53.377 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:20:53.379 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:20:53.417 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.327) 0:01:32.399 ******* 2020-11-16T22:20:53.444 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:20:53.446 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.066) 0:01:32.466 ******* 2020-11-16T22:20:53.562 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:53.564 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:20:53.829 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:20:53.864 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.380) 0:01:32.846 ******* 2020-11-16T22:20:53.933 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:20:53.935 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} TASK [common : Add krb5 config file] ******************************************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:20:53.975 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.111) 0:01:32.957 ******* 2020-11-16T22:20:54.006 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.031) 0:01:32.989 ******* 2020-11-16T22:20:54.013 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:20:54.015 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:54.048 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.042) 0:01:33.031 ******* 2020-11-16T22:20:54.317 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:54.319 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:20:54.339 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:54.341 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:20:54.589 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:54.590 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:20:54.610 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:54.612 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:20:54.849 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:54.851 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:20:54.884 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.835) 0:01:33.867 ******* 2020-11-16T22:20:54.890 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:54.892 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:20:54.928 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.043) 0:01:33.911 ******* 2020-11-16T22:20:54.970 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:20:54.972 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:20:55.003 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.074) 0:01:33.985 ******* 2020-11-16T22:20:55.010 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:20:55.011 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:55.045 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.042) 0:01:34.027 ******* 2020-11-16T22:20:55.052 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:20:55.053 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:20:55.093 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.047) 0:01:34.075 ******* 2020-11-16T22:20:55.129 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.036) 0:01:34.112 ******* 2020-11-16T22:20:55.158 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:20:55.160 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:55.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.062) 0:01:34.174 ******* 2020-11-16T22:20:55.233 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:20:55.234 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:55.244 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:55.246 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:55.279 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.087) 0:01:34.261 ******* 2020-11-16T22:20:55.307 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:20:55.308 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:55.341 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.061) 0:01:34.323 ******* 2020-11-16T22:20:55.348 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:55.349 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:20:55.578 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:55.580 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:20:55.589 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:55.591 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:20:55.852 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:20:55.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.548) 0:01:34.871 ******* 2020-11-16T22:20:55.916 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:20:55.918 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:20:55.950 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:55 +0000 (0:00:00.060) 0:01:34.932 ******* 2020-11-16T22:20:55.978 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:20:55.980 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:56.011 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.060) 0:01:34.993 ******* 2020-11-16T22:20:56.028 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:56.029 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:20:56.039 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:56.072 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.061) 0:01:35.054 ******* 2020-11-16T22:20:56.110 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:20:56.111 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:56.121 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:56.123 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:56.155 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.082) 0:01:35.137 ******* 2020-11-16T22:20:56.181 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:56.213 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.058) 0:01:35.196 ******* 2020-11-16T22:20:56.250 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:20:56.252 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:20:56.284 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.070) 0:01:35.267 ******* 2020-11-16T22:20:56.436 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:20:56.438 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:20:56.482 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.197) 0:01:35.464 ******* 2020-11-16T22:20:56.581 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:20:56.582 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T22:20:56.616 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.134) 0:01:35.599 ******* 2020-11-16T22:20:56.656 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:20:56.657 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:56.695 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.078) 0:01:35.678 ******* 2020-11-16T22:20:56.910 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:20:56.911 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:20:56.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:56 +0000 (0:00:00.249) 0:01:35.927 ******* 2020-11-16T22:20:57.075 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:20:57.077 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:20:57.118 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:20:57.119 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:20:57.152 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.207) 0:01:36.134 ******* 2020-11-16T22:20:57.169 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:57.170 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:20:57.180 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:20:57.182 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:20:57.216 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.064) 0:01:36.199 ******* 2020-11-16T22:20:57.244 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:20:57.246 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:57.277 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.060) 0:01:36.260 ******* 2020-11-16T22:20:57.304 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:20:57.306 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:57.337 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.060) 0:01:36.320 ******* 2020-11-16T22:20:57.365 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:20:57.367 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:57.398 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.060) 0:01:36.381 ******* 2020-11-16T22:20:57.456 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:20:57.458 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T22:20:57.491 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.092) 0:01:36.473 ******* 2020-11-16T22:20:57.600 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:57.602 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:20:57.706 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:20:57.708 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:20:57.745 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.253) 0:01:36.727 ******* 2020-11-16T22:20:57.794 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:20:57.795 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:20:57.828 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:57 +0000 (0:00:00.083) 0:01:36.811 ******* 2020-11-16T22:20:57.968 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:20:57.970 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:20:58.011 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:20:58.013 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:20:58.053 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.224) 0:01:37.035 ******* 2020-11-16T22:20:58.071 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:20:58.073 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:20:58.107 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.054) 0:01:37.090 ******* 2020-11-16T22:20:58.114 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:20:58.115 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:20:58.154 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.047) 0:01:37.137 ******* 2020-11-16T22:20:58.161 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:20:58.163 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.195 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.041) 0:01:37.178 ******* 2020-11-16T22:20:58.201 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:20:58.203 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.241 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.045) 0:01:37.224 ******* 2020-11-16T22:20:58.247 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:20:58.249 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:58.286 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.044) 0:01:37.269 ******* 2020-11-16T22:20:58.293 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:20:58.295 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:20:58.333 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.047) 0:01:37.316 ******* 2020-11-16T22:20:58.340 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:20:58.342 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.375 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.041) 0:01:37.358 ******* 2020-11-16T22:20:58.382 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:20:58.384 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.394 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.395 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:20:58.396 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ActiveEnterTimestampMonotonic": "18157291", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-tmpfiles-setup.service system.slice nss-lookup.target -.mount sysinit.target var-run.mount local-fs.target basic.target remote-fs.target time-sync.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:58 UTC", "AssertTimestampMonotonic": "18156338", "Before": "xdm.service shutdown.target plymouth-quit.service getty@tty1.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ConditionTimestampMonotonic": "18156338", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-16T22:20:58.397 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1160", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ExecMainStartTimestampMonotonic": "18157258", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:58 UTC", "InactiveExitTimestampMonotonic": "18157291", "InvocationID": "8c0463e01b1345f8b148fb2bc9046a81", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-16T22:20:58.399 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1160", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-16T22:20:58.401 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:58 UTC", "StateChangeTimestampMonotonic": "18157291", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:58 UTC", "WatchdogTimestampMonotonic": "18157290", "WatchdogUSec": "0"}} TASK [common : Install nrpe without epel] ************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sep 2020-11-16T22:20:58.402 INFO:teuthology.task.ansible.out:ia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.443 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.067) 0:01:37.426 ******* 2020-11-16T22:20:58.474 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.031) 0:01:37.457 ******* 2020-11-16T22:20:58.482 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:20:58.483 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.047) 0:01:37.504 ******* 2020-11-16T22:20:58.529 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:20:58.530 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.563 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.041) 0:01:37.546 ******* 2020-11-16T22:20:58.591 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T22:20:58.592 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.623 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.060) 0:01:37.606 ******* 2020-11-16T22:20:58.651 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-16T22:20:58.652 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.684 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.060) 0:01:37.667 ******* 2020-11-16T22:20:58.691 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:20:58.692 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:20:58.726 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.041) 0:01:37.708 ******* 2020-11-16T22:20:58.734 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-16T22:20:58.735 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.041) 0:01:37.750 ******* 2020-11-16T22:20:58.795 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T22:20:58.797 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.828 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.060) 0:01:37.810 ******* 2020-11-16T22:20:58.856 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:20:58.858 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.061) 0:01:37.872 ******* 2020-11-16T22:20:58.916 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:20:58.918 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:58.949 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.059) 0:01:37.931 ******* 2020-11-16T22:20:58.955 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:20:58.957 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ActiveEnterTimestampMonotonic": "18028609", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target -.mount network.target nss-lookup.target var-run.mount remote-fs.target local-fs.target time-sync.target systemd-journald.socket system.slice basic.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:38 UTC", "AssertTimestampMonotonic": "18027855", "Before": "multi-user.target getty@tty1.service shutdown.target plymouth-quit.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ConditionTimestampMonotonic": "18027855", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-16T22:20:58.958 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1086", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ExecMainStartTimestampMonotonic": "18028577", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:38 UTC", "InactiveExitTimestampMonotonic": "18028609", "InvocationID": "4804150922a14e36b5a2e2886d4e9045", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-16T22:20:58.960 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1086", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-16T22:20:58.961 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:38 UTC", "StateChangeTimestampMonotonic": "18028609", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:38 UTC", "WatchdogTimestampMonotonic": "18028608", "WatchdogUSec": "0"}} 2020-11-16T22:20:58.991 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:58 +0000 (0:00:00.042) 0:01:37.973 ******* 2020-11-16T22:20:58.997 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:20:58.998 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.030 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.039) 0:01:38.013 ******* 2020-11-16T22:20:59.038 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:20:59.039 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.070 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.040) 0:01:38.053 ******* 2020-11-16T22:20:59.076 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-16T22:20:59.078 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.115 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.044) 0:01:38.097 ******* 2020-11-16T22:20:59.122 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T22:20:59.123 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.156 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.040) 0:01:38.138 ******* 2020-11-16T22:20:59.163 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:20:59.165 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.180 ******* 2020-11-16T22:20:59.204 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-16T22:20:59.206 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.239 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.221 ******* 2020-11-16T22:20:59.245 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:20:59.247 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:20:59.280 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.040) 0:01:38.262 ******* 2020-11-16T22:20:59.286 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-16T22:20:59.288 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.321 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.304 ******* 2020-11-16T22:20:59.327 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:20:59.328 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T22:20:59.362 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.345 ******* 2020-11-16T22:20:59.369 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T22:20:59.370 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.404 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.386 ******* 2020-11-16T22:20:59.410 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:20:59.412 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-16T22:20:59.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.428 ******* 2020-11-16T22:20:59.452 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:20:59.453 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.487 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.469 ******* 2020-11-16T22:20:59.492 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:20:59.494 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:20:59.527 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.040) 0:01:38.510 ******* 2020-11-16T22:20:59.534 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:20:59.535 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.569 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.551 ******* 2020-11-16T22:20:59.576 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:20:59.577 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:20:59.615 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.046) 0:01:38.598 ******* 2020-11-16T22:20:59.622 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:20:59.623 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.041) 0:01:38.639 ******* 2020-11-16T22:20:59.671 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:20:59.672 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:20:59.704 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.047) 0:01:38.686 ******* 2020-11-16T22:20:59.710 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-16T22:20:59.711 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.745 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.040) 0:01:38.727 ******* 2020-11-16T22:20:59.772 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.805 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.060) 0:01:38.787 ******* 2020-11-16T22:20:59.831 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:20:59.832 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:20:59.864 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.059) 0:01:38.847 ******* 2020-11-16T22:20:59.895 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.030) 0:01:38.878 ******* 2020-11-16T22:20:59.902 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-16T22:20:59.903 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:59.938 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.042) 0:01:38.921 ******* 2020-11-16T22:20:59.945 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:20:59.946 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T22:20:59.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:59 +0000 (0:00:00.047) 0:01:38.968 ******* 2020-11-16T22:20:59.991 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:20:59.993 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:00.026 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.040) 0:01:39.008 ******* 2020-11-16T22:21:00.032 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:21:00.033 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-16T22:21:00.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.041) 0:01:39.050 ******* 2020-11-16T22:21:00.116 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} Monday 16 November 2020 22:21:00 +0000 (0:00:00.047) 0:01:39.098 ******* 2020-11-16T22:21:00.122 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:21:00.124 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:21:00.158 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.042) 0:01:39.141 ******* 2020-11-16T22:21:00.175 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:21:00.177 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T22:21:00.210 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.052) 0:01:39.193 ******* 2020-11-16T22:21:00.278 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:21:00.280 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:21:00.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.109) 0:01:39.302 ******* 2020-11-16T22:21:00.450 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:21:00.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.163) 0:01:39.466 ******* 2020-11-16T22:21:00.511 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:00.543 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.059) 0:01:39.526 ******* 2020-11-16T22:21:00.570 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:21:00.572 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:21:00.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.066) 0:01:39.593 ******* 2020-11-16T22:21:00.641 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.031) 0:01:39.624 ******* 2020-11-16T22:21:00.673 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.031) 0:01:39.656 ******* 2020-11-16T22:21:00.680 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:21:00.681 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi175"}, "changed": false} 2020-11-16T22:21:00.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.041) 0:01:39.697 ******* 2020-11-16T22:21:00.794 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:21:00.796 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:21:00.827 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:21:00.828 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T22:21:00.860 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.145) 0:01:39.842 ******* 2020-11-16T22:21:00.949 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:21:00.951 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:21:00.990 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:00 +0000 (0:00:00.129) 0:01:39.972 ******* 2020-11-16T22:21:01.017 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:21:01.019 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:01.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.066) 0:01:40.039 ******* 2020-11-16T22:21:01.083 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:21:01.085 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:21:01.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.065) 0:01:40.104 ******* 2020-11-16T22:21:01.148 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:21:01.150 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:01.186 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.064) 0:01:40.169 ******* 2020-11-16T22:21:01.346 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:21:01.347 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:21:01.379 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.193) 0:01:40.362 ******* 2020-11-16T22:21:01.406 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:21:01.408 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi181"}, "changed": false} 2020-11-16T22:21:01.439 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.060) 0:01:40.422 ******* 2020-11-16T22:21:01.561 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:21:01.563 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi175", "ansible_hostname": "smithi175", "ansible_nodename": "smithi175"}, "changed": false, "name": "smithi175"} 2020-11-16T22:21:01.595 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.155) 0:01:40.578 ******* 2020-11-16T22:21:01.654 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:21:01.655 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:21:01.694 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.098) 0:01:40.677 ******* 2020-11-16T22:21:01.887 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:21:01.889 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi181", "ansible_hostname": "smithi181", "ansible_nodename": "smithi181"}, "changed": false, "name": "smithi181"} 2020-11-16T22:21:02.090 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:01 +0000 (0:00:00.227) 0:01:40.904 ******* TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} Monday 16 November 2020 22:21:01 +0000 (0:00:00.042) 0:01:40.946 ******* TASK [common : Make sure nagios nrpe service is running.] ********************** [WARNING]: The service (nagios-nrpe-server) is actually an init script but the system is managed by systemd fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring Monday 16 November 2020 22:21:02 +0000 (0:00:00.079) 0:01:41.025 ******* TASK [common : Get SELinux status] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.108 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.065) 0:01:41.091 ******* 2020-11-16T22:21:02.135 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T22:21:02.136 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.173 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.065) 0:01:41.156 ******* 2020-11-16T22:21:02.200 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-16T22:21:02.201 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.070) 0:01:41.226 ******* 2020-11-16T22:21:02.251 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:21:02.252 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T22:21:02.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.043) 0:01:41.269 ******* 2020-11-16T22:21:02.418 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:21:02.420 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.458 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.170) 0:01:41.440 ******* 2020-11-16T22:21:02.494 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.036) 0:01:41.477 ******* 2020-11-16T22:21:02.500 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T22:21:02.502 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.546 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.052) 0:01:41.529 ******* 2020-11-16T22:21:02.575 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.067) 0:01:41.597 ******* 2020-11-16T22:21:02.641 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:21:02.642 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.065) 0:01:41.662 ******* 2020-11-16T22:21:02.707 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:21:02.708 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.066) 0:01:41.729 ******* 2020-11-16T22:21:02.753 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:21:02.754 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:21:02.787 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.041) 0:01:41.770 ******* 2020-11-16T22:21:02.793 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-16T22:21:02.795 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.834 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.046) 0:01:41.816 ******* 2020-11-16T22:21:02.860 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:21:02.862 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:02.899 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.065) 0:01:41.882 ******* 2020-11-16T22:21:02.926 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:21:02.927 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:21:02.965 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:02 +0000 (0:00:00.065) 0:01:41.948 ******* 2020-11-16T22:21:03.003 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:21:03.005 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T22:21:03.047 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.082) 0:01:42.030 ******* 2020-11-16T22:21:03.085 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:21:03.087 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi175.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:21:03.125 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.077) 0:01:42.108 ******* 2020-11-16T22:21:03.156 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.030) 0:01:42.138 ******* 2020-11-16T22:21:03.164 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:21:03.166 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:21:03.203 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.047) 0:01:42.186 ******* 2020-11-16T22:21:03.210 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:21:03.212 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.246 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.042) 0:01:42.228 ******* 2020-11-16T22:21:03.263 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:21:03.264 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.050) 0:01:42.279 ******* 2020-11-16T22:21:03.328 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.031) 0:01:42.311 ******* 2020-11-16T22:21:03.335 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:21:03.336 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.369 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.040) 0:01:42.351 ******* 2020-11-16T22:21:03.374 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:21:03.376 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.407 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.038) 0:01:42.390 ******* 2020-11-16T22:21:03.414 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:21:03.415 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:21:03.454 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.046) 0:01:42.437 ******* 2020-11-16T22:21:03.487 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.032) 0:01:42.470 ******* 2020-11-16T22:21:03.494 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:21:03.496 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.528 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.040) 0:01:42.511 ******* 2020-11-16T22:21:03.533 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:03.535 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.567 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.038) 0:01:42.550 ******* 2020-11-16T22:21:03.581 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:21:03.582 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.613 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.045) 0:01:42.596 ******* 2020-11-16T22:21:03.618 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:03.620 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.652 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.039) 0:01:42.635 ******* 2020-11-16T22:21:03.658 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:21:03.659 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:21:03.699 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.046) 0:01:42.682 ******* 2020-11-16T22:21:03.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.030) 0:01:42.712 ******* 2020-11-16T22:21:03.735 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:03.736 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.775 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.044) 0:01:42.757 ******* 2020-11-16T22:21:03.806 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.030) 0:01:42.788 ******* 2020-11-16T22:21:03.812 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:03.814 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.852 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.046) 0:01:42.834 ******* 2020-11-16T22:21:03.882 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.029) 0:01:42.864 ******* 2020-11-16T22:21:03.888 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T22:21:03.890 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.925 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.043) 0:01:42.907 ******* 2020-11-16T22:21:03.931 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:03.932 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:03.965 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.039) 0:01:42.947 ******* 2020-11-16T22:21:03.971 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:21:04.005 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:03 +0000 (0:00:00.040) 0:01:42.988 ******* 2020-11-16T22:21:04.011 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:04.013 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.053 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.047) 0:01:43.035 ******* 2020-11-16T22:21:04.060 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:21:04.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.057) 0:01:43.093 ******* 2020-11-16T22:21:04.119 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:21:04.120 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T22:21:04.122 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.160 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.050) 0:01:43.143 ******* 2020-11-16T22:21:04.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.031) 0:01:43.174 ******* 2020-11-16T22:21:04.198 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:21:04.233 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.216 ******* 2020-11-16T22:21:04.240 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:21:04.274 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.257 ******* 2020-11-16T22:21:04.280 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:21:04.282 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.315 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.040) 0:01:43.298 ******* 2020-11-16T22:21:04.326 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:21:04.361 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.046) 0:01:43.344 ******* 2020-11-16T22:21:04.367 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:21:04.368 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.401 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.040) 0:01:43.384 ******* 2020-11-16T22:21:04.408 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:21:04.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.040) 0:01:43.424 ******* 2020-11-16T22:21:04.449 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:21:04.451 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.040) 0:01:43.465 ******* 2020-11-16T22:21:04.490 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:21:04.491 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.524 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.507 ******* 2020-11-16T22:21:04.531 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:04.532 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.566 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.042) 0:01:43.549 ******* 2020-11-16T22:21:04.573 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:21:04.575 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.590 ******* 2020-11-16T22:21:04.613 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:21:04.614 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.043) 0:01:43.633 ******* 2020-11-16T22:21:04.658 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:21:04.659 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.693 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.675 ******* 2020-11-16T22:21:04.699 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:21:04.701 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.048) 0:01:43.723 ******* 2020-11-16T22:21:04.773 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.031) 0:01:43.755 ******* 2020-11-16T22:21:04.779 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:04.781 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set hostname var] ********************************************* ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi106"}, "changed": false} 2020-11-16T22:21:04.819 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.046) 0:01:43.802 ******* 2020-11-16T22:21:04.851 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.031) 0:01:43.833 ******* 2020-11-16T22:21:04.858 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:21:04.859 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.894 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.043) 0:01:43.876 ******* 2020-11-16T22:21:04.900 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:21:04.902 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.935 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.041) 0:01:43.918 ******* 2020-11-16T22:21:04.942 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T22:21:04.944 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:04.978 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:04 +0000 (0:00:00.043) 0:01:43.961 ******* 2020-11-16T22:21:04.986 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T22:21:04.987 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.020 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.041) 0:01:44.003 ******* 2020-11-16T22:21:05.028 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:21:05.030 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.062 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.042) 0:01:44.045 ******* 2020-11-16T22:21:05.069 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:21:05.071 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.104 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.041) 0:01:44.087 ******* 2020-11-16T22:21:05.110 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:21:05.112 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.145 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.041) 0:01:44.128 ******* 2020-11-16T22:21:05.153 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T22:21:05.154 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.046) 0:01:44.174 ******* 2020-11-16T22:21:05.199 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:05.200 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.234 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.042) 0:01:44.217 ******* 2020-11-16T22:21:05.243 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:21:05.245 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.277 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.042) 0:01:44.260 ******* 2020-11-16T22:21:05.284 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:21:05.286 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.042) 0:01:44.303 ******* 2020-11-16T22:21:05.328 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:21:05.329 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi106.front.sepia.ceph.com", "ansible_hostname": "smithi106", "ansible_nodename": "smithi106"}, "changed": true, "name": "smithi106"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.368 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.047) 0:01:44.350 ******* 2020-11-16T22:21:05.400 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.031) 0:01:44.382 ******* 2020-11-16T22:21:05.406 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:05.408 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.042) 0:01:44.425 ******* 2020-11-16T22:21:05.448 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:05.449 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.466 ******* 2020-11-16T22:21:05.489 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:05.491 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.525 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.041) 0:01:44.507 ******* 2020-11-16T22:21:05.532 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:21:05.533 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.566 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.041) 0:01:44.549 ******* 2020-11-16T22:21:05.572 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:21:05.573 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.590 ******* 2020-11-16T22:21:05.613 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:05.615 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.648 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.630 ******* 2020-11-16T22:21:05.653 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:21:05.655 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.688 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.671 ******* 2020-11-16T22:21:05.696 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:05.697 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.729 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.712 ******* 2020-11-16T22:21:05.735 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:21:05.736 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.775 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.045) 0:01:44.757 ******* 2020-11-16T22:21:05.781 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:21:05.783 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.815 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.798 ******* 2020-11-16T22:21:05.821 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:21:05.823 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T22:21:05.862 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.046) 0:01:44.845 ******* 2020-11-16T22:21:05.892 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.029) 0:01:44.875 ******* 2020-11-16T22:21:05.899 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:21:05.901 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.933 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.915 ******* 2020-11-16T22:21:05.939 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:21:05.940 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:05.973 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:05 +0000 (0:00:00.040) 0:01:44.955 ******* 2020-11-16T22:21:05.978 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:21:05.980 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.013 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.039) 0:01:44.995 ******* 2020-11-16T22:21:06.019 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:21:06.021 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.053 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.040) 0:01:45.036 ******* 2020-11-16T22:21:06.060 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:21:06.062 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.094 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.040) 0:01:45.076 ******* 2020-11-16T22:21:06.099 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:21:06.101 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.133 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.039) 0:01:45.116 ******* 2020-11-16T22:21:06.141 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:21:06.142 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.174 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.041) 0:01:45.157 ******* 2020-11-16T22:21:06.180 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:21:06.181 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.215 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.040) 0:01:45.197 ******* 2020-11-16T22:21:06.221 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:21:06.223 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.046) 0:01:45.244 ******* 2020-11-16T22:21:06.288 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:21:06.290 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.325 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.063) 0:01:45.307 ******* 2020-11-16T22:21:06.332 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:21:06.333 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:21:06.378 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.052) 0:01:45.360 ******* 2020-11-16T22:21:06.386 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:21:06.388 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005241", "end": "2020-11-16 22:21:06.350864", "rc": 0, "start": "2020-11-16 22:21:06.345623", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:06.425 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.047) 0:01:45.408 ******* 2020-11-16T22:21:06.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.031) 0:01:45.440 ******* 2020-11-16T22:21:06.462 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:21:06.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.045) 0:01:45.485 ******* 2020-11-16T22:21:06.592 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:21:06.594 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005206", "end": "2020-11-16 22:21:06.592499", "rc": 0, "start": "2020-11-16 22:21:06.587293", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:06.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.124) 0:01:45.609 ******* 2020-11-16T22:21:06.654 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:21:06.687 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.059) 0:01:45.669 ******* 2020-11-16T22:21:06.818 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:21:06.819 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:21:06.852 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.165) 0:01:45.835 ******* 2020-11-16T22:21:06.880 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:21:06.881 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:06.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.060) 0:01:45.896 ******* 2020-11-16T22:21:06.962 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:21:06.963 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:21:07.002 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:06 +0000 (0:00:00.088) 0:01:45.985 ******* 2020-11-16T22:21:07.009 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:21:07.011 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:21:07.043 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.041) 0:01:46.026 ******* 2020-11-16T22:21:07.050 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:21:07.052 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.090 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.046) 0:01:46.073 ******* 2020-11-16T22:21:07.097 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:21:07.098 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.130 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.040) 0:01:46.113 ******* 2020-11-16T22:21:07.137 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:21:07.138 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.181 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.050) 0:01:46.164 ******* 2020-11-16T22:21:07.209 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:21:07.210 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.247 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.066) 0:01:46.230 ******* 2020-11-16T22:21:07.274 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:21:07.276 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.313 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.065) 0:01:46.295 ******* 2020-11-16T22:21:07.339 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:07.341 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.378 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.065) 0:01:46.361 ******* 2020-11-16T22:21:07.404 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:07.405 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.064) 0:01:46.425 ******* 2020-11-16T22:21:07.469 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:07.471 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.065) 0:01:46.491 ******* 2020-11-16T22:21:07.535 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T22:21:07.536 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.573 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.064) 0:01:46.556 ******* 2020-11-16T22:21:07.599 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:21:07.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.064) 0:01:46.620 ******* 2020-11-16T22:21:07.664 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:21:07.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.065) 0:01:46.685 ******* 2020-11-16T22:21:07.729 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:21:07.771 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.068) 0:01:46.753 ******* 2020-11-16T22:21:07.797 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:21:07.799 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.836 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.064) 0:01:46.818 ******* 2020-11-16T22:21:07.872 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:21:07.874 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.911 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.075) 0:01:46.894 ******* 2020-11-16T22:21:07.938 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:21:07.940 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:07.977 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:07 +0000 (0:00:00.065) 0:01:46.960 ******* 2020-11-16T22:21:08.004 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:08.006 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.042 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.025 ******* 2020-11-16T22:21:08.069 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:21:08.071 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.107 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.064) 0:01:47.090 ******* 2020-11-16T22:21:08.134 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T22:21:08.135 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.172 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.155 ******* 2020-11-16T22:21:08.200 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:21:08.201 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.238 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.221 ******* 2020-11-16T22:21:08.265 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.305 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.066) 0:01:47.287 ******* 2020-11-16T22:21:08.311 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:21:08.313 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} ok: [smithi181.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T22:21:08.324 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:21:08.325 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.370 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.353 ******* 2020-11-16T22:21:08.397 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:21:08.399 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.435 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.418 ******* 2020-11-16T22:21:08.462 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:08.464 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.501 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.484 ******* 2020-11-16T22:21:08.528 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.567 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.549 ******* 2020-11-16T22:21:08.593 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:08.594 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.631 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.064) 0:01:47.614 ******* 2020-11-16T22:21:08.658 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:08.660 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.697 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.679 ******* 2020-11-16T22:21:08.724 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:21:08.726 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.762 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.745 ******* 2020-11-16T22:21:08.789 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:21:08.790 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.827 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.064) 0:01:47.810 ******* 2020-11-16T22:21:08.854 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:21:08.856 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.065) 0:01:47.875 ******* 2020-11-16T22:21:08.920 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:21:08.922 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:08.963 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:08 +0000 (0:00:00.070) 0:01:47.946 ******* 2020-11-16T22:21:08.990 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:21:08.991 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:09.028 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.064) 0:01:48.011 ******* 2020-11-16T22:21:09.054 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:21:09.056 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:09.093 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.064) 0:01:48.075 ******* 2020-11-16T22:21:09.119 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:21:09.120 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:09.157 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.064) 0:01:48.140 ******* 2020-11-16T22:21:09.185 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:21:09.186 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:09.240 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.082) 0:01:48.222 ******* 2020-11-16T22:21:09.245 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:21:09.247 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T22:21:09.247 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T22:21:09.282 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.042) 0:01:48.265 ******* 2020-11-16T22:21:09.313 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.030) 0:01:48.295 ******* 2020-11-16T22:21:09.434 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:21:09.435 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.014308", "end": "2020-11-16 22:21:09.422363", "rc": 0, "start": "2020-11-16 22:21:09.408055", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:09.474 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.161) 0:01:48.457 ******* 2020-11-16T22:21:09.502 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:21:09.540 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:09 +0000 (0:00:00.066) 0:01:48.523 ******* 2020-11-16T22:21:09.988 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:21:09.990 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:21:10.027 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.486) 0:01:49.009 ******* 2020-11-16T22:21:10.054 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:21:10.055 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:10.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.065) 0:01:49.075 ******* 2020-11-16T22:21:10.180 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:21:10.182 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:21:10.216 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.124) 0:01:49.199 ******* 2020-11-16T22:21:10.224 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:10.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.041) 0:01:49.241 ******* 2020-11-16T22:21:10.265 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:21:10.300 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.041) 0:01:49.282 ******* 2020-11-16T22:21:10.342 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.042) 0:01:49.324 ******* 2020-11-16T22:21:10.465 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:21:10.467 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.034403", "end": "2020-11-16 22:21:10.459158", "rc": 0, "start": "2020-11-16 22:21:10.424755", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:10.501 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.159) 0:01:49.484 ******* 2020-11-16T22:21:10.519 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033169", "end": "2020-11-16 22:21:10.507426", "rc": 0, "start": "2020-11-16 22:21:10.474257", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:10.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:10 +0000 (0:00:00.055) 0:01:49.540 ******* 2020-11-16T22:21:11.940 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:21:11.942 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T22:21:13.376 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:21:13.378 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:21:13.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:13 +0000 (0:00:02.852) 0:01:52.392 ******* 2020-11-16T22:21:13.478 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:13.513 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:13 +0000 (0:00:00.103) 0:01:52.495 ******* 2020-11-16T22:21:13.716 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:21:13.717 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T22:21:13.756 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:13 +0000 (0:00:00.243) 0:01:52.739 ******* 2020-11-16T22:21:14.795 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:21:14.797 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:14.834 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:14 +0000 (0:00:01.077) 0:01:53.816 ******* 2020-11-16T22:21:14.861 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:21:14.902 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:14 +0000 (0:00:00.067) 0:01:53.884 ******* 2020-11-16T22:21:15.166 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:21:15.168 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.088472", "end": "2020-11-16 22:21:15.158349", "rc": 0, "start": "2020-11-16 22:21:15.069877", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:15.205 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:15 +0000 (0:00:00.303) 0:01:54.188 ******* 2020-11-16T22:21:19.350 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:21:19.352 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:19.389 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:19 +0000 (0:00:04.183) 0:01:58.372 ******* 2020-11-16T22:21:22.946 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T22:21:22.948 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-16T22:21:22.949 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T22:21:22.951 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T22:21:22.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:22 +0000 (0:00:03.591) 0:02:01.963 ******* 2020-11-16T22:21:23.051 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-16T22:21:23.255 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T22:21:23.257 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:21:23 +0000 (0:00:00.105) 0:02:02.069 ******* 2020-11-16T22:21:23.852 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:23.854 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:23.886 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:23 +0000 (0:00:00.800) 0:02:02.869 ******* 2020-11-16T22:21:23.913 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:21:23.915 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:23.946 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:23 +0000 (0:00:00.059) 0:02:02.929 ******* 2020-11-16T22:21:23.974 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:23.975 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:21:24.008 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:24 +0000 (0:00:00.062) 0:02:02.991 ******* 2020-11-16T22:21:24.036 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:21:24.037 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:24.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:24 +0000 (0:00:00.060) 0:02:03.051 ******* 2020-11-16T22:21:31.070 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T22:21:31.072 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104569 files and directories 2020-11-16T22:21:31.074 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T22:21:31.075 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104569 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T22:21:31.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:31 +0000 (0:00:07.041) 0:02:10.093 ******* 2020-11-16T22:21:32.162 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:32.164 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:32.201 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:32 +0000 (0:00:01.090) 0:02:11.183 ******* 2020-11-16T22:21:32.227 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:32.265 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:32 +0000 (0:00:00.064) 0:02:11.248 ******* 2020-11-16T22:21:32.698 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:32.699 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9572 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T22:21:32.701 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T22:21:32.702 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9572 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T22:21:32.704 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T22:21:32.735 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:32 +0000 (0:00:00.469) 0:02:11.717 ******* 2020-11-16T22:21:32.743 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9692 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T22:21:32.745 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T22:21:32.746 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9692 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T22:21:32.748 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T22:21:32.779 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:32 +0000 (0:00:00.044) 0:02:11.762 ******* 2020-11-16T22:21:33.634 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T22:21:33.636 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:33.669 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.889) 0:02:12.651 ******* 2020-11-16T22:21:33.675 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:21:33.709 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.040) 0:02:12.692 ******* 2020-11-16T22:21:33.788 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:21:33.789 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T22:21:33.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.112) 0:02:12.804 ******* 2020-11-16T22:21:33.838 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T22:21:33.871 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.049) 0:02:12.854 ******* 2020-11-16T22:21:33.878 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T22:21:33.880 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:33.912 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.040) 0:02:12.894 ******* 2020-11-16T22:21:33.917 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:33.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:33 +0000 (0:00:00.039) 0:02:12.934 ******* 2020-11-16T22:21:34.038 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:21:34.040 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:21:34.076 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.124) 0:02:13.059 ******* 2020-11-16T22:21:34.083 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:21:34.118 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.041) 0:02:13.100 ******* 2020-11-16T22:21:34.373 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:21:34.374 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:21:34.407 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.289) 0:02:13.390 ******* 2020-11-16T22:21:34.434 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:21:34.480 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.072) 0:02:13.462 ******* 2020-11-16T22:21:34.549 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:21:34.551 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:21:34.588 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.107) 0:02:13.570 ******* 2020-11-16T22:21:34.636 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:21:34.670 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.082) 0:02:13.653 ******* 2020-11-16T22:21:34.718 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:21:34.720 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:34.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.082) 0:02:13.735 ******* 2020-11-16T22:21:34.800 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:34.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:34 +0000 (0:00:00.082) 0:02:13.818 ******* 2020-11-16T22:21:35.040 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:21:35.041 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:39 UTC", "ActiveEnterTimestampMonotonic": "18446284", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target remote-fs.target basic.target sysinit.target systemd-journald.socket system.slice -.mount systemd-tmpfiles-setup.service network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:38 UTC", "AssertTimestampMonotonic": "18030479", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ConditionTimestampMonotonic": "18030479", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T22:21:35.042 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1178", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:39 UTC", "ExecMainStartTimestampMonotonic": "18446272", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:38 UTC", "InactiveExitTimestampMonotonic": "18031386", "InvocationID": "1b5a6e2f554647e085238c95a59a961a", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T22:21:35.044 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1178", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T22:21:35.045 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:39 UTC", "StateChangeTimestampMonotonic": "18446284", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:39 UTC", "WatchdogTimestampMonotonic": "18446282", "WatchdogUSec": "0"}} 2020-11-16T22:21:35.076 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.240) 0:02:14.058 ******* 2020-11-16T22:21:35.102 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:21:35.103 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.134 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.058) 0:02:14.117 ******* 2020-11-16T22:21:35.140 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:21:35.142 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:59 UTC", "ActiveEnterTimestampMonotonic": "18543709", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount sysinit.target nss-lookup.target network.target systemd-journald.socket system.slice basic.target systemd-tmpfiles-setup.service remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:58 UTC", "AssertTimestampMonotonic": "18154784", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ConditionTimestampMonotonic": "18154784", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T22:21:35.347 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1285", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:59 UTC", "ExecMainStartTimestampMonotonic": "18543698", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:58 UTC", "InactiveExitTimestampMonotonic": "18155417", "InvocationID": "af62b1ef1cb84e7585becece8e8a1c54", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T22:21:35.349 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1285", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T22:21:35.350 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:59 UTC", "StateChangeTimestampMonotonic": "18543709", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:59 UTC", "WatchdogTimestampMonotonic": "18543707", "WatchdogUSec": "0"}} Monday 16 November 2020 22:21:35 +0000 (0:00:00.041) 0:02:14.159 ******* TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:35 +0000 (0:00:00.041) 0:02:14.200 ******* TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:35 +0000 (0:00:00.050) 0:02:14.250 ******* TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi 2020-11-16T22:21:35.352 INFO:teuthology.task.ansible.out:181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.291 ******* TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:35 +0000 (0:00:00.041) 0:02:14.332 ******* 2020-11-16T22:21:35.355 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:35.357 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.390 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.372 ******* 2020-11-16T22:21:35.396 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:21:35.398 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.431 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.413 ******* 2020-11-16T22:21:35.438 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:35.439 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.472 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.454 ******* 2020-11-16T22:21:35.478 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:35.480 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.049) 0:02:14.504 ******* 2020-11-16T22:21:35.533 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:35.535 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.567 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.045) 0:02:14.550 ******* 2020-11-16T22:21:35.574 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:35.576 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.042) 0:02:14.592 ******* 2020-11-16T22:21:35.616 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:21:35.617 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.633 ******* 2020-11-16T22:21:35.657 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:35.659 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.051) 0:02:14.685 ******* 2020-11-16T22:21:35.709 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:21:35.710 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.743 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.040) 0:02:14.725 ******* 2020-11-16T22:21:35.748 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:21:35.750 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.039) 0:02:14.765 ******* 2020-11-16T22:21:35.789 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:35.790 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.039) 0:02:14.805 ******* 2020-11-16T22:21:35.827 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:21:35.829 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.866 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.043) 0:02:14.849 ******* 2020-11-16T22:21:35.873 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:35.874 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.043) 0:02:14.892 ******* 2020-11-16T22:21:35.917 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:35.918 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.952 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.042) 0:02:14.935 ******* 2020-11-16T22:21:35.959 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:35.961 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:35.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:35 +0000 (0:00:00.041) 0:02:14.977 ******* 2020-11-16T22:21:36.001 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:36.003 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.036 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.019 ******* 2020-11-16T22:21:36.043 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:36.045 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.078 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.060 ******* 2020-11-16T22:21:36.085 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:36.086 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.119 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.102 ******* 2020-11-16T22:21:36.127 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:21:36.128 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.161 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.144 ******* 2020-11-16T22:21:36.170 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:36.171 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.205 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.043) 0:02:15.187 ******* 2020-11-16T22:21:36.212 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:36.213 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.247 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.230 ******* 2020-11-16T22:21:36.254 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:21:36.256 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.289 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.272 ******* 2020-11-16T22:21:36.296 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:36.298 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.330 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.313 ******* 2020-11-16T22:21:36.337 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:36.339 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.371 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.354 ******* 2020-11-16T22:21:36.379 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:36.380 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.414 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.396 ******* 2020-11-16T22:21:36.422 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:36.424 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.456 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.439 ******* 2020-11-16T22:21:36.464 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:36.465 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.045) 0:02:15.485 ******* 2020-11-16T22:21:36.509 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:36.511 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.544 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.527 ******* 2020-11-16T22:21:36.552 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:21:36.554 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.589 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.045) 0:02:15.572 ******* 2020-11-16T22:21:36.596 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:36.598 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.632 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.614 ******* 2020-11-16T22:21:36.639 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:21:36.640 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.674 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.042) 0:02:15.656 ******* 2020-11-16T22:21:36.680 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:21:36.682 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.040) 0:02:15.697 ******* 2020-11-16T22:21:36.721 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:36.723 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.756 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.739 ******* 2020-11-16T22:21:36.763 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:21:36.765 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.780 ******* 2020-11-16T22:21:36.806 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:36.807 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.839 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.822 ******* 2020-11-16T22:21:36.846 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:36.847 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.881 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.863 ******* 2020-11-16T22:21:36.888 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:36.889 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.922 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.041) 0:02:15.904 ******* 2020-11-16T22:21:36.928 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:36.930 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:36.962 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.040) 0:02:15.945 ******* 2020-11-16T22:21:36.969 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:36.971 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.003 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:36 +0000 (0:00:00.040) 0:02:15.986 ******* 2020-11-16T22:21:37.010 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:37.011 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.044 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.040) 0:02:16.026 ******* 2020-11-16T22:21:37.052 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:37.054 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.085 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.041) 0:02:16.067 ******* 2020-11-16T22:21:37.091 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:37.093 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.129 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.044) 0:02:16.112 ******* 2020-11-16T22:21:37.137 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:37.138 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.170 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.040) 0:02:16.153 ******* 2020-11-16T22:21:37.177 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:37.178 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.212 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.041) 0:02:16.194 ******* 2020-11-16T22:21:37.220 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:21:37.222 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.254 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.042) 0:02:16.237 ******* 2020-11-16T22:21:37.261 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:37.263 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.296 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.041) 0:02:16.279 ******* 2020-11-16T22:21:37.323 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:21:37.325 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.356 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.060) 0:02:16.339 ******* 2020-11-16T22:21:37.384 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:21:37.386 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:21:37.419 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.062) 0:02:16.401 ******* 2020-11-16T22:21:37.488 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:21:37.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.103) 0:02:16.505 ******* 2020-11-16T22:21:37.727 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:37.729 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:21:37.762 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.239) 0:02:16.745 ******* 2020-11-16T22:21:37.788 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.060) 0:02:16.805 ******* 2020-11-16T22:21:37.839 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:37.840 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:21:37.874 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.051) 0:02:16.857 ******* 2020-11-16T22:21:37.901 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:21:37.902 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:37.934 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:37 +0000 (0:00:00.060) 0:02:16.917 ******* 2020-11-16T22:21:38.116 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:21:38.118 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:21:38.158 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.223) 0:02:17.140 ******* 2020-11-16T22:21:38.237 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:21:38.273 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.115) 0:02:17.255 ******* 2020-11-16T22:21:38.465 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T22:21:38.467 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:21:38.500 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.227) 0:02:17.483 ******* 2020-11-16T22:21:38.591 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:21:38.626 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.126) 0:02:17.609 ******* 2020-11-16T22:21:38.811 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:21:38.813 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:21:38.848 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.221) 0:02:17.831 ******* 2020-11-16T22:21:38.928 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:21:38.963 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:38 +0000 (0:00:00.114) 0:02:17.946 ******* 2020-11-16T22:21:38.979 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:21:38.981 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:39.015 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.051) 0:02:17.997 ******* 2020-11-16T22:21:39.041 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:21:39.043 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.075 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.060) 0:02:18.058 ******* 2020-11-16T22:21:39.092 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:21:39.094 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:39.126 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.050) 0:02:18.108 ******* 2020-11-16T22:21:39.131 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:21:39.133 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.165 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.039) 0:02:18.148 ******* 2020-11-16T22:21:39.172 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:21:39.173 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.206 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.041) 0:02:18.189 ******* 2020-11-16T22:21:39.213 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:21:39.415 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.043) 0:02:18.232 ******* TASK [testnode : Stop collectl] ************************************************ skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.040) 0:02:18.273 ******* TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.040) 0:02:18.313 ******* TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.040) 0:02:18.354 ******* TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.039) 0:02:18.394 ******* 2020-11-16T22:21:39.417 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.039) 0:02:18.433 ******* 2020-11-16T22:21:39.457 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:39.459 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:21:39.491 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.040) 0:02:18.474 ******* 2020-11-16T22:21:39.497 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T22:21:39.499 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.532 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.041) 0:02:18.515 ******* 2020-11-16T22:21:39.540 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:21:39.542 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.552 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.554 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.564 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:39.565 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi181.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.770 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:39 +0000 (0:00:00.065) 0:02:18.580 ******* ok: [smithi181.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi181.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was 2020-11-16T22:21:39.772 INFO:teuthology.task.ansible.out:False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.070) 0:02:18.650 ******* skipping: [smithi175.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi175.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:39.975 INFO:teuthology.task.ansible.out: skipping: [smithi175.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.112) 0:02:18.763 ******* TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi181.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003474", "end": "2020-11-16 22:21:39.831664", "rc": 0, "start": "2020-11-16 22:21:39.828190", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} Monday 16 November 2020 22:21:39 +0000 (0:00:00.117) 0:02:18.880 ******* TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:39 +0000 (0:00:00.067) 0:02:18.948 ******* 2020-11-16T22:21:39.982 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T22:21:39.984 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003043", "end": "2020-11-16 22:21:39.955018", "rc": 0, "start": "2020-11-16 22:21:39.951975", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:21:40.024 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:40 +0000 (0:00:00.059) 0:02:19.007 ******* 2020-11-16T22:21:40.030 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T22:21:40.032 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:40.066 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:40 +0000 (0:00:00.042) 0:02:19.049 ******* 2020-11-16T22:21:40.072 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T22:21:40.074 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:40.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:40 +0000 (0:00:00.043) 0:02:19.093 ******* 2020-11-16T22:21:40.116 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T22:21:40.117 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:40.150 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:40 +0000 (0:00:00.040) 0:02:19.133 ******* 2020-11-16T22:21:40.156 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T22:21:40.157 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:40.362 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:40 +0000 (0:00:00.040) 0:02:19.173 ******* TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:40 +0000 (0:00:00.060) 0:02:19.233 ******* 2020-11-16T22:21:41.047 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:21:41.048 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:41.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.830) 0:02:20.064 ******* 2020-11-16T22:21:41.109 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:21:41.111 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:41.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.061) 0:02:20.126 ******* 2020-11-16T22:21:41.149 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:21:41.151 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:21:41.184 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.040) 0:02:20.167 ******* 2020-11-16T22:21:41.191 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:21:41.193 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={'block_used': 13783688, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311420, 'size_available': 926971576320, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311420, "block_size": 4096, "block_total": 240095108, "block_used": 13783688, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971576320, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:21:41.229 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.044) 0:02:20.212 ******* 2020-11-16T22:21:41.236 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:21:41.237 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:41.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.041) 0:02:20.253 ******* 2020-11-16T22:21:41.309 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:21:41.310 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item={'block_used': 13783681, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311427, 'size_available': 926971604992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311427, "block_size": 4096, "block_total": 240095108, "block_used": 13783681, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971604992, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:21:41.342 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.071) 0:02:20.325 ******* 2020-11-16T22:21:41.359 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:21:41.361 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009722", "end": "2020-11-16 22:21:41.364729", "rc": 0, "start": "2020-11-16 22:21:41.355007", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:41.393 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.050) 0:02:20.376 ******* 2020-11-16T22:21:41.431 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:21:41.433 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'block_used': 13783688, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311420, 'size_available': 926971576320, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311420, "block_size": 4096, "block_total": 240095108, "block_used": 13783688, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971576320, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:41.467 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.073) 0:02:20.450 ******* 2020-11-16T22:21:41.473 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:21:41.475 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010117", "end": "2020-11-16 22:21:41.472106", "rc": 0, "start": "2020-11-16 22:21:41.461989", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:41.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.040) 0:02:20.490 ******* 2020-11-16T22:21:41.513 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:21:41.548 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.040) 0:02:20.530 ******* 2020-11-16T22:21:41.554 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:21:41.556 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'block_used': 13783681, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311427, 'size_available': 926971604992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311427, "block_size": 4096, "block_total": 240095108, "block_used": 13783681, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971604992, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:41.592 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.043) 0:02:20.574 ******* 2020-11-16T22:21:41.598 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:41.599 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:41.610 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:21:41.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:41 +0000 (0:00:00.052) 0:02:20.626 ******* 2020-11-16T22:21:41.680 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:41.682 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:42.940 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:21:42.942 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9283 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T22:21:43.147 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104427 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T22:21:43.149 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9283 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T22:21:43.355 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104427 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:21:42 +0000 (0:00:01.346) 0:02:21.972 ******* 2020-11-16T22:21:43.638 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:43.640 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhFEhWCKvNtF613trKWm13Id23s51ReYoM'], 'uuids': ['2278be7f-ed95-4ab6-a90b-46258221de19']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.835804", "end": "2020-11-16 22:21:43.627494", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhFEhWCKvNtF613trKWm13Id23s51ReYoM"], "labels": [], "masters": [], "uuids": ["2278be7f-ed95-4ab6-a90b-46258221de19"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:41.791690", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:43.842 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => 2020-11-16T22:21:43.844 INFO:teuthology.task.ansible.out:(item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vR7FYH9bCCm12DotNbWbRNy4tf1S3amf'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.946889", "end": "2020-11-16 22:21:43.649493", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vR7FYH9bCCm12DotNbWbRNy4tf1S3amf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": 2020-11-16T22:21:43.846 INFO:teuthology.task.ansible.out:"89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:41.702604", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_ 2020-11-16T22:21:44.051 INFO:teuthology.task.ansible.out:discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Install non aarch64 packages] ********************************* ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:44.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:44 +0000 (0:00:01.096) 0:02:23.069 ******* 2020-11-16T22:21:44.258 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:21:44.259 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T22:21:44.462 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:44 +0000 (0:00:00.214) 0:02:23.283 ******* TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:21:44 +0000 (0:00:00.064) 0:02:23.348 ******* 2020-11-16T22:21:44.547 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:21:44.548 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:21:44.587 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:44 +0000 (0:00:00.221) 0:02:23.569 ******* 2020-11-16T22:21:44.925 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:44.927 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3SGVyBPduw5APFmVKEd5ypYVxudFA5byh'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.122498", "end": "2020-11-16 22:21:44.910565", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3SGVyBPduw5APFmVKEd5ypYVxudFA5byh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:43.788067", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:44.928 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTWX', 'wwn-0x5000c5009293ee09'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1', 'wwn-0x5000c5009293ee09-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293ee09', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTWX', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX", "wwn-0x5000c5009293ee09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1", "wwn-0x5000c5009293ee09-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTWX", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293ee09"}}, "skip_reason": 2020-11-16T22:21:44.930 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T22:21:45.018 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhDWQTYdCKpvDiAOsdIIL7YWZIUjK8jqVp'], 'uuids': ['90e8212a-a0e3-4954-9816-cb0ebee16e48']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.108757", "end": "2020-11-16 22:21:45.006232", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhDWQTYdCKpvDiAOsdIIL7YWZIUjK8jqVp"], "labels": [], "masters": [], "uuids": ["90e8212a-a0e3-4954-9816-cb0ebee16e48"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:43.897475", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:45.031 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XJFL', 'wwn-0x5000c5009292c947'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XJFL-part1', 'wwn-0x5000c5009292c947-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009292c947', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XJFL', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XJFL", "wwn-0x5000c5009292c947"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XJFL-part1", "wwn-0x5000c5009292c947-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XJFL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009292c947"}}, "skip_reason": 2020-11-16T22:21:45.033 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T22:21:45.044 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:21:45.045 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:21:45.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:45 +0000 (0:00:00.505) 0:02:24.075 ******* 2020-11-16T22:21:45.309 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:21:45.310 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:21:45.444 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:45 +0000 (0:00:00.351) 0:02:24.426 ******* 2020-11-16T22:21:45.617 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:21:45.618 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:45.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:45 +0000 (0:00:00.212) 0:02:24.639 ******* 2020-11-16T22:21:45.942 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:21:45.943 INFO:teuthology.task.ansible.out:[WARNING]: The service (apache2) is actually an init script but the system is managed by systemd fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-16T22:21:45.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:45 +0000 (0:00:00.324) 0:02:24.963 ******* 2020-11-16T22:21:46.007 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:21:46.009 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.046 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.065) 0:02:25.029 ******* 2020-11-16T22:21:46.072 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:21:46.074 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.111 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.065) 0:02:25.094 ******* 2020-11-16T22:21:46.137 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:21:46.138 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.176 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.064) 0:02:25.158 ******* 2020-11-16T22:21:46.201 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:21:46.203 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.240 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.064) 0:02:25.222 ******* 2020-11-16T22:21:46.266 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:21:46.267 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.307 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.067) 0:02:25.290 ******* 2020-11-16T22:21:46.333 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:21:46.335 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.064) 0:02:25.355 ******* 2020-11-16T22:21:46.381 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:46.382 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-CTrYXl-le6Q-nxdf-p4wH-xVp9-EMil-IVj7li', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300M0400BGN', 'nvme-nvme.8086-435646543632333330304d3034303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.194933", "end": "2020-11-16 22:21:46.330382", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-CTrYXl-le6Q-nxdf-p4wH-xVp9-EMil-IVj7li", "nvme-INTEL_SSDPEDMD400G4_CVFT623300M0400BGN", "nvme-nvme.8086-435646543632333330304d3034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:45.135449", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-16T22:21:46.384 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:46.385 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.387 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.389 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-IMzlva-1nVn-RnLa-d2Fr-BWZT-HfZw-3Vekdh', 'nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN', 'nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.185884", "end": "2020-11-16 22:21:46.362942", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-IMzlva-1nVn-RnLa-d2Fr-BWZT-HfZw-3Vekdh", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN", "nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:45.177058", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-16T22:21:46.390 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:46.390 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.392 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.403 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.070) 0:02:25.425 ******* 2020-11-16T22:21:46.470 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:21:46.471 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.066) 0:02:25.491 ******* 2020-11-16T22:21:46.535 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:46.537 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.573 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.064) 0:02:25.556 ******* 2020-11-16T22:21:46.600 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:21:46.601 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.065) 0:02:25.622 ******* 2020-11-16T22:21:46.670 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.709 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.069) 0:02:25.692 ******* 2020-11-16T22:21:46.736 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:21:46.738 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.774 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.065) 0:02:25.757 ******* 2020-11-16T22:21:46.800 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.063) 0:02:25.821 ******* 2020-11-16T22:21:46.864 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:46.865 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.905 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.066) 0:02:25.888 ******* 2020-11-16T22:21:46.931 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:46.969 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:46 +0000 (0:00:00.064) 0:02:25.952 ******* 2020-11-16T22:21:46.995 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:46.996 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.033 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.063) 0:02:26.016 ******* 2020-11-16T22:21:47.059 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:47.060 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.097 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.063) 0:02:26.079 ******* 2020-11-16T22:21:47.124 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:21:47.125 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.162 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.065) 0:02:26.145 ******* 2020-11-16T22:21:47.188 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.226 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.063) 0:02:26.209 ******* 2020-11-16T22:21:47.252 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.290 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.063) 0:02:26.273 ******* 2020-11-16T22:21:47.316 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:47.317 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.354 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.063) 0:02:26.336 ******* 2020-11-16T22:21:47.380 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:47.381 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.418 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.064) 0:02:26.401 ******* 2020-11-16T22:21:47.444 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:21:47.445 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.485 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.066) 0:02:26.467 ******* 2020-11-16T22:21:47.512 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:21:47.513 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.065) 0:02:26.533 ******* 2020-11-16T22:21:47.577 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:21:47.579 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.616 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.066) 0:02:26.599 ******* 2020-11-16T22:21:47.623 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:47.624 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhPREhjgp0vR2fXiMoWUHEy6ANBLQkgqxb'], 'uuids': ['4f71a6b4-61b7-486e-abd1-1799029b51f5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.110031", "end": "2020-11-16 22:21:47.573278", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhPREhjgp0vR2fXiMoWUHEy6ANBLQkgqxb"], "labels": [], "masters": [], "uuids": ["4f71a6b4-61b7-486e-abd1-1799029b51f5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:46.463247", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:47.625 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (i 2020-11-16T22:21:47.626 INFO:teuthology.task.ansible.out:tem={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.637 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.675 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.058) 0:02:26.658 ******* 2020-11-16T22:21:47.691 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:47.693 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX391eQP00KRcmtr4XrtT1zcPmOYUnN2AJR'], 'uuids': ['a7284138-a51e-4c9f-addb-30d584a4983d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.177556", "end": "2020-11-16 22:21:47.676010", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX391eQP00KRcmtr4XrtT1zcPmOYUnN2AJR"], "labels": [], "masters": [], "uuids": ["a7284138-a51e-4c9f-addb-30d584a4983d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:46.498454", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi181.front.sepia.ceph.com] => (i 2020-11-16T22:21:47.695 INFO:teuthology.task.ansible.out:tem={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.704 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:47.840 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:21:47.841 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:21:47.878 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:47 +0000 (0:00:00.203) 0:02:26.861 ******* 2020-11-16T22:21:48.330 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:21:48.331 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:21:48.368 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:48 +0000 (0:00:00.489) 0:02:27.351 ******* 2020-11-16T22:21:48.394 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:21:48.396 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:48.433 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:48 +0000 (0:00:00.064) 0:02:27.415 ******* 2020-11-16T22:21:48.825 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:48.827 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh3GvX76H4ROMysvYsACKus2dnNu4vJVIg'], 'uuids': ['12e4e347-c886-42f0-ac68-82339a09ca75']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.112766", "end": "2020-11-16 22:21:48.823064", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh3GvX76H4ROMysvYsACKus2dnNu4vJVIg"], "labels": [], "masters": [], "uuids": ["12e4e347-c886-42f0-ac68-82339a09ca75"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:47.710298", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:48.837 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:48.911 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:21:48.912 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:21:48.949 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:48 +0000 (0:00:00.516) 0:02:27.932 ******* 2020-11-16T22:21:48.965 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:48.966 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vHjDxqRIjZMYuaAtd0jgXSnGNFicYkT6'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.135057", "end": "2020-11-16 22:21:48.953132", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vHjDxqRIjZMYuaAtd0jgXSnGNFicYkT6"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:47.818075", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:48.967 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:49.414 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T22:21:49.416 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:21:49.453 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:49 +0000 (0:00:00.503) 0:02:28.435 ******* 2020-11-16T22:21:49.896 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:21:49.898 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:21:49.935 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:49 +0000 (0:00:00.481) 0:02:28.917 ******* 2020-11-16T22:21:50.077 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:50.079 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh985MrVQCCgALFbedeAY1g7GvVEdWSF7b'], 'uuids': ['952e8d62-e71d-4a41-b0e5-4d1faab4d1de']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.120598", "end": "2020-11-16 22:21:50.065628", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh985MrVQCCgALFbedeAY1g7GvVEdWSF7b"], "labels": [], "masters": [], "uuids": ["952e8d62-e71d-4a41-b0e5-4d1faab4d1de"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:48.945030", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:50.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.186) 0:02:29.104 ******* 2020-11-16T22:21:50.129 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:21:50.130 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:50.169 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.048) 0:02:29.152 ******* 2020-11-16T22:21:50.176 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:50.178 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.198 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.239 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.070) 0:02:29.222 ******* 2020-11-16T22:21:50.246 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:50.247 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3VPa0uuxOU5Yoorg4jx72cuux9zrdJu5o'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.124570", "end": "2020-11-16 22:21:50.204242", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3VPa0uuxOU5Yoorg4jx72cuux9zrdJu5o"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:49.079672", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:50.290 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.051) 0:02:29.273 ******* 2020-11-16T22:21:50.297 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:21:50.298 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.338 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.047) 0:02:29.321 ******* 2020-11-16T22:21:50.346 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:50.346 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.358 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:21:50.360 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.398 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.059) 0:02:29.380 ******* 2020-11-16T22:21:50.424 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T22:21:50.426 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.464 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.066) 0:02:29.446 ******* 2020-11-16T22:21:50.471 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:50.472 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhFEhWCKvNtF613trKWm13Id23s51ReYoM'], 'uuids': ['2278be7f-ed95-4ab6-a90b-46258221de19']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.161731", "end": "2020-11-16 22:21:50.437343", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhFEhWCKvNtF613trKWm13Id23s51ReYoM"], "labels": [], "masters": [], "uuids": ["2278be7f-ed95-4ab6-a90b-46258221de19"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:50.275612", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.158015 s, 730 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.158015 s, 730 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:50.473 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.475 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.485 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T22:21:50.487 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.525 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.061) 0:02:29.508 ******* 2020-11-16T22:21:50.552 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:50.554 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:21:50.594 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.068) 0:02:29.576 ******* 2020-11-16T22:21:50.612 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vR7FYH9bCCm12DotNbWbRNy4tf1S3amf'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.157350", "end": "2020-11-16 22:21:50.604966", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vR7FYH9bCCm12DotNbWbRNy4tf1S3amf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:50.447616", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152677 s, 755 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152677 s, 755 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:50.624 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.625 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.646 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:21:50.648 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram14) => {"ansible_loop_var": "item", "changed": false, "item": "ram14", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=ram15) => {"ansible_loop_var": "item", "changed": false, "item": "ram15", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.657 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram12) => {"ansible_loop_var": "item", "changed": false, "item": "ram12", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.658 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram13) => {"ansible_loop_var": "item", "changed": false, "item": "ram13", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.669 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram10) => {"ansible_loop_var": "item", "changed": false, "item": "ram10", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.680 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram11) => {"ansible_loop_var": "item", "changed": false, "item": "ram11", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.682 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram8) => {"ansible_loop_var": "item", "changed": false, "item": "ram8", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.691 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram9) => {"ansible_loop_var": "item", "changed": false, "item": "ram9", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.702 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram4) => {"ansible_loop_var": "item", "changed": false, "item": "ram4", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.704 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram5) => {"ansible_loop_var": "item", "changed": false, "item": "ram5", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.713 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram6) => {"ansible_loop_var": "item", "changed": false, "item": "ram6", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.715 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram7) => {"ansible_loop_var": "item", "changed": false, "item": "ram7", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.724 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram0) => {"ansible_loop_var": "item", "changed": false, "item": "ram0", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.726 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram1) => {"ansible_loop_var": "item", "changed": false, "item": "ram1", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.735 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram2) => {"ansible_loop_var": "item", "changed": false, "item": "ram2", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.746 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=ram3) => {"ansible_loop_var": "item", "changed": false, "item": "ram3", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.748 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhDWQTYdCKpvDiAOsdIIL7YWZIUjK8jqVp'], 'uuids': ['90e8212a-a0e3-4954-9816-cb0ebee16e48']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.163455", "end": "2020-11-16 22:21:50.738235", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhDWQTYdCKpvDiAOsdIIL7YWZIUjK8jqVp"], "labels": [], "masters": [], "uuids": ["90e8212a-a0e3-4954-9816-cb0ebee16e48"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:50.574780", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.158775 s, 726 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.158775 s, 726 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:50.759 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:21:50.760 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XJFL', 'wwn-0x5000c5009292c947'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XJFL-part1', 'wwn-0x5000c5009292c947-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009292c947', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XJFL', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XJFL", "wwn-0x5000c5009292c947"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XJFL-part1", "wwn-0x5000c5009292c947-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_a 2020-11-16T22:21:50.762 INFO:teuthology.task.ansible.out:ddress": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XJFL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009292c947"}}, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.771 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.773 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.782 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.784 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.793 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.795 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.804 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.815 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.817 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.827 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.828 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.838 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:21:50.882 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:50 +0000 (0:00:00.288) 0:02:29.865 ******* 2020-11-16T22:21:50.910 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:50.912 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3SGVyBPduw5APFmVKEd5ypYVxudFA5byh'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.160189", "end": "2020-11-16 22:21:50.901208", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3SGVyBPduw5APFmVKEd5ypYVxudFA5byh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:50.741019", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155485 s, 742 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155485 s, 742 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:50.913 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTWX', 'wwn-0x5000c5009293ee09'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1', 'wwn-0x5000c5009293ee09-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293ee09', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTWX', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX", "wwn-0x5000c5009293ee09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1", "wwn-0x5000c5009293ee09-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTWX", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293ee09"}}, "skip_reason": 2020-11-16T22:21:50.915 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T22:21:50.915 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-CTrYXl-le6Q-nxdf-p4wH-xVp9-EMil-IVj7li', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300M0400BGN', 'nvme-nvme.8086-435646543632333330304d3034303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.045941", "end": "2020-11-16 22:21:50.907645", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-CTrYXl-le6Q-nxdf-p4wH-xVp9-EMil-IVj7li", "nvme-INTEL_SSDPEDMD400G4_CVFT623300M0400BGN", "nvme-nvme.8086-435646543632333330304d3034303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:50.861704", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) cop 2020-11-16T22:21:50.916 INFO:teuthology.task.ansible.out:ied, 0.0431235 s, 2.7 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0431235 s, 2.7 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:50.927 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:50.928 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.065 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T22:21:51.067 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003366", "end": "2020-11-16 22:21:51.044338", "rc": 0, "start": "2020-11-16 22:21:51.040972", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:21:51.111 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:51 +0000 (0:00:00.228) 0:02:30.093 ******* 2020-11-16T22:21:51.118 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:51.118 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-IMzlva-1nVn-RnLa-d2Fr-BWZT-HfZw-3Vekdh', 'nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN', 'nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.053789", "end": "2020-11-16 22:21:51.084000", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-IMzlva-1nVn-RnLa-d2Fr-BWZT-HfZw-3Vekdh", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN", "nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.030211", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0491445 s, 2.3 GB/s", "std 2020-11-16T22:21:51.120 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0491445 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.121 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.143 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.187 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:51 +0000 (0:00:00.076) 0:02:30.170 ******* 2020-11-16T22:21:51.204 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:51.206 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhPREhjgp0vR2fXiMoWUHEy6ANBLQkgqxb'], 'uuids': ['4f71a6b4-61b7-486e-abd1-1799029b51f5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.148964", "end": "2020-11-16 22:21:51.189153", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULhPREhjgp0vR2fXiMoWUHEy6ANBLQkgqxb"], "labels": [], "masters": [], "uuids": ["4f71a6b4-61b7-486e-abd1-1799029b51f5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.040189", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14375 s, 802 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14375 s, 802 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0. 2020-11-16T22:21:51.208 INFO:teuthology.task.ansible.out:00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.217 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.219 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:51 +0000 (0:00:00.070) 0:02:30.240 ******* 2020-11-16T22:21:51.284 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T22:21:51.286 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:51 +0000 (0:00:00.065) 0:02:30.305 ******* 2020-11-16T22:21:51.382 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:21:51.383 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX391eQP00KRcmtr4XrtT1zcPmOYUnN2AJR'], 'uuids': ['a7284138-a51e-4c9f-addb-30d584a4983d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.150793", "end": "2020-11-16 22:21:51.374027", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX391eQP00KRcmtr4XrtT1zcPmOYUnN2AJR"], "labels": [], "masters": [], "uuids": ["a7284138-a51e-4c9f-addb-30d584a4983d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.223234", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147134 s, 784 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147134 s, 784 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:51.394 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.395 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.500 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh3GvX76H4ROMysvYsACKus2dnNu4vJVIg'], 'uuids': ['12e4e347-c886-42f0-ac68-82339a09ca75']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.160015", "end": "2020-11-16 22:21:51.490154", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh3GvX76H4ROMysvYsACKus2dnNu4vJVIg"], "labels": [], "masters": [], "uuids": ["12e4e347-c886-42f0-ac68-82339a09ca75"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.330139", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155278 s, 743 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155278 s, 743 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:51.511 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.679 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vHjDxqRIjZMYuaAtd0jgXSnGNFicYkT6'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.159213", "end": "2020-11-16 22:21:51.673190", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3vHjDxqRIjZMYuaAtd0jgXSnGNFicYkT6"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.513977", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154666 s, 746 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154666 s, 746 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:51.690 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:51.785 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh985MrVQCCgALFbedeAY1g7GvVEdWSF7b'], 'uuids': ['952e8d62-e71d-4a41-b0e5-4d1faab4d1de']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.157161", "end": "2020-11-16 22:21:51.773738", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Gw0HeZTlBakvLeYC1CbrtlcWkYA7EULh985MrVQCCgALFbedeAY1g7GvVEdWSF7b"], "labels": [], "masters": [], "uuids": ["952e8d62-e71d-4a41-b0e5-4d1faab4d1de"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.616577", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152392 s, 757 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152392 s, 757 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:51.828 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:51 +0000 (0:00:00.505) 0:02:30.811 ******* 2020-11-16T22:21:51.971 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3VPa0uuxOU5Yoorg4jx72cuux9zrdJu5o'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.155350", "end": "2020-11-16 22:21:51.956956", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-wsF8ikKMg7MUaYJgFrVoKfIRFCyY4PX3VPa0uuxOU5Yoorg4jx72cuux9zrdJu5o"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:51.801606", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150726 s, 765 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150726 s, 765 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.015 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.186) 0:02:30.998 ******* 2020-11-16T22:21:52.075 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T22:21:52.077 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.112263", "end": "2020-11-16 22:21:52.066359", "rc": 0, "start": "2020-11-16 22:21:51.954096", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.095) 0:02:31.093 ******* 2020-11-16T22:21:52.283 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.131948", "end": "2020-11-16 22:21:52.274692", "rc": 0, "start": "2020-11-16 22:21:52.142744", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.318 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.207) 0:02:31.300 ******* 2020-11-16T22:21:52.325 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T22:21:52.326 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.067065", "end": "2020-11-16 22:21:52.305121", "rc": 0, "start": "2020-11-16 22:21:52.238056", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.361 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.043) 0:02:31.344 ******* 2020-11-16T22:21:52.369 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:21:52.370 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:52.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.053) 0:02:31.398 ******* 2020-11-16T22:21:52.423 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:21:52.459 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.044) 0:02:31.442 ******* 2020-11-16T22:21:52.466 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:21:52.468 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.507 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.047) 0:02:31.490 ******* 2020-11-16T22:21:52.515 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:21:52.516 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for physical volumes] *********************************** changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.057166", "end": "2020-11-16 22:21:52.502490", "rc": 0, "start": "2020-11-16 22:21:52.445324", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.553 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.046) 0:02:31.536 ******* 2020-11-16T22:21:52.585 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.031) 0:02:31.568 ******* 2020-11-16T22:21:52.594 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:21:52.595 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'block_used': 14062243, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032865, 'size_available': 925830615040, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119632, 'block_size': 4096, 'inode_available': 60935344}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032865, "block_size": 4096, "block_total": 240095108, "block_used": 14062243, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935344, "inode_total": 61054976, "inode_used": 119632, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830615040, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:21:52.596 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.049) 0:02:31.617 ******* 2020-11-16T22:21:52.666 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.031) 0:02:31.649 ******* 2020-11-16T22:21:52.674 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:21:52.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.044) 0:02:31.693 ******* 2020-11-16T22:21:52.716 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:21:52.718 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.042) 0:02:31.736 ******* 2020-11-16T22:21:52.759 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:21:52.761 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.044) 0:02:31.780 ******* 2020-11-16T22:21:52.804 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:21:52.806 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.843 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.045) 0:02:31.825 ******* 2020-11-16T22:21:52.849 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:21:52.851 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.022979", "end": "2020-11-16 22:21:52.821344", "rc": 0, "start": "2020-11-16 22:21:52.798365", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:21:52.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.049) 0:02:31.875 ******* 2020-11-16T22:21:52.930 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.037) 0:02:31.913 ******* 2020-11-16T22:21:52.939 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:21:52.939 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'block_used': 14062243, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032865, 'size_available': 925830615040, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119632, 'block_size': 4096, 'inode_available': 60935344}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032865, "block_size": 4096, "block_total": 240095108, "block_used": 14062243, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935344, "inode_total": 61054976, "inode_used": 119632, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830615040, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:52.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:52 +0000 (0:00:00.050) 0:02:31.963 ******* 2020-11-16T22:21:52.988 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:21:52.990 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:53.024 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.043) 0:02:32.007 ******* 2020-11-16T22:21:53.059 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.034) 0:02:32.042 ******* 2020-11-16T22:21:53.066 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:21:53.067 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi175.front.sepia.ceph.com] => (item={'block_used': 13783681, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311427, 'size_available': 926971604992, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311427, "block_size": 4096, "block_total": 240095108, "block_used": 13783681, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971604992, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.107 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.047) 0:02:32.090 ******* 2020-11-16T22:21:53.140 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.033) 0:02:32.123 ******* 2020-11-16T22:21:53.148 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:21:53.150 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.184 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.043) 0:02:32.167 ******* 2020-11-16T22:21:53.192 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:53.193 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.194 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.196 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:53.196 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.197 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.198 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.051) 0:02:32.218 ******* 2020-11-16T22:21:53.244 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.245 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.247 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.247 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.249 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.250 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.261 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:53.262 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.295 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.060) 0:02:32.278 ******* 2020-11-16T22:21:53.304 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.305 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.306 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.308 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:21:53.308 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:21:53.343 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.047) 0:02:32.325 ******* 2020-11-16T22:21:53.348 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:53.350 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.384 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.041) 0:02:32.367 ******* 2020-11-16T22:21:53.391 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:21:53.393 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'block_used': 13783688, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311420, 'size_available': 926971576320, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311420, "block_size": 4096, "block_total": 240095108, "block_used": 13783688, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971576320, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.427 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.043) 0:02:32.410 ******* 2020-11-16T22:21:53.434 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:21:53.436 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.469 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.041) 0:02:32.451 ******* 2020-11-16T22:21:53.476 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:21:53.477 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.515 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.045) 0:02:32.497 ******* 2020-11-16T22:21:53.522 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:21:53.523 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.556 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.041) 0:02:32.539 ******* 2020-11-16T22:21:53.565 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:53.566 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.598 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.042) 0:02:32.581 ******* 2020-11-16T22:21:53.606 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:21:53.607 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.641 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.042) 0:02:32.624 ******* 2020-11-16T22:21:53.649 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:21:53.650 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.683 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.041) 0:02:32.666 ******* 2020-11-16T22:21:53.710 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:21:53.711 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.742 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.059) 0:02:32.725 ******* 2020-11-16T22:21:53.769 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:21:53.771 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.802 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.059) 0:02:32.785 ******* 2020-11-16T22:21:53.829 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:53.862 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:53 +0000 (0:00:00.059) 0:02:32.845 ******* 2020-11-16T22:21:54.199 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T22:21:54.201 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:21:54.233 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:54 +0000 (0:00:00.370) 0:02:33.216 ******* 2020-11-16T22:21:54.240 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:21:54.275 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:54 +0000 (0:00:00.041) 0:02:33.258 ******* 2020-11-16T22:21:54.780 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:21:54.782 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:54.791 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.010 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.032 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.095 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:55.096 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0VvUfLrdg3v1YQUeyxHT3PCBzrREffdCM'], 'uuids': ['f2db210b-66dc-4ac3-90ec-dff707cc8420']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.617440", "end": "2020-11-16 22:21:55.072529", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0VvUfLrdg3v1YQUeyxHT3PCBzrREffdCM"], "labels": [], "masters": [], "uuids": ["f2db210b-66dc-4ac3-90ec-dff707cc8420"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:53.455089", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:55.241 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:21:55.242 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.252 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.473 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.494 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.714 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.716 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:21:55.750 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:55 +0000 (0:00:01.475) 0:02:34.733 ******* 2020-11-16T22:21:55.781 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:55 +0000 (0:00:00.031) 0:02:34.764 ******* 2020-11-16T22:21:55.872 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:21:55.874 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:21:55.908 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:55 +0000 (0:00:00.126) 0:02:34.891 ******* 2020-11-16T22:21:55.916 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:21:55.952 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:55 +0000 (0:00:00.044) 0:02:34.935 ******* 2020-11-16T22:21:56.053 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:21:56.055 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.096 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.170 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.213 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.287 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.319 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.393 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:56.394 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0w2hMGg7QqGj5uy5W16tKuusIXbuH3aid'], 'uuids': ['0d43a544-71bf-44c1-8bff-258ad6679cda']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.131629", "end": "2020-11-16 22:21:56.377242", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0w2hMGg7QqGj5uy5W16tKuusIXbuH3aid"], "labels": [], "masters": [], "uuids": ["0d43a544-71bf-44c1-8bff-258ad6679cda"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:55.245613", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Write /scratch_devs] ****** 2020-11-16T22:21:56.396 INFO:teuthology.task.ansible.out:************************************ changed: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.406 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:56.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.492) 0:02:35.427 ******* 2020-11-16T22:21:56.453 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:21:56.454 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:56.490 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.045) 0:02:35.473 ******* 2020-11-16T22:21:56.570 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T22:21:56.572 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:21:56.605 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.114) 0:02:35.587 ******* 2020-11-16T22:21:56.623 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:21:56.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.051) 0:02:35.639 ******* 2020-11-16T22:21:56.662 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:21:56.663 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:21:56.698 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.042) 0:02:35.681 ******* 2020-11-16T22:21:56.705 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:21:56.740 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:56 +0000 (0:00:00.041) 0:02:35.723 ******* 2020-11-16T22:21:57.163 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:21:57.165 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:21:57.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:57 +0000 (0:00:00.456) 0:02:36.180 ******* 2020-11-16T22:21:57.631 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:21:57.632 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:21:57.664 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:57 +0000 (0:00:00.467) 0:02:36.647 ******* 2020-11-16T22:21:57.691 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:57.691 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0liBTTu5Z6iCng6WAUkStA2OSLpRGJhla'], 'uuids': ['f19551fd-2608-41d7-98a5-41c4fda5e621']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.133718", "end": "2020-11-16 22:21:57.677408", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0liBTTu5Z6iCng6WAUkStA2OSLpRGJhla"], "labels": [], "masters": [], "uuids": ["f19551fd-2608-41d7-98a5-41c4fda5e621"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:56.543690", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:57.806 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:21:57.808 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:21:57.840 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:57 +0000 (0:00:00.175) 0:02:36.822 ******* 2020-11-16T22:21:57.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:21:57.971 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:21:58.003 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:57 +0000 (0:00:00.163) 0:02:36.986 ******* 2020-11-16T22:21:58.164 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:21:58.166 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:21:58.207 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:58 +0000 (0:00:00.195) 0:02:37.182 ******* 2020-11-16T22:21:58.323 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:21:58.325 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:21:58.358 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:58 +0000 (0:00:00.158) 0:02:37.340 ******* 2020-11-16T22:21:58.664 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:21:58.665 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T22:21:58.697 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:58 +0000 (0:00:00.339) 0:02:37.680 ******* 2020-11-16T22:21:58.984 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:21:58.986 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0FMGXjqFrtXfc4Gv7iNMCdqVWGDbTbIld'], 'uuids': ['14022b15-6155-4c2e-b22d-835abb5733e4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.129523", "end": "2020-11-16 22:21:58.969465", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0FMGXjqFrtXfc4Gv7iNMCdqVWGDbTbIld"], "labels": [], "masters": [], "uuids": ["14022b15-6155-4c2e-b22d-835abb5733e4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:57.839942", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:21:59.384 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:21:59.386 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:21:59.417 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.720) 0:02:38.400 ******* 2020-11-16T22:21:59.558 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:21:59.560 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:21:59.593 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.175) 0:02:38.576 ******* 2020-11-16T22:21:59.599 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:21:59.601 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:21:59.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.041) 0:02:38.617 ******* 2020-11-16T22:21:59.724 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:21:59.726 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:21:59.759 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.124) 0:02:38.741 ******* 2020-11-16T22:21:59.807 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T22:21:59.809 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045258", "end": "2020-11-16 22:21:59.807537", "rc": 0, "start": "2020-11-16 22:21:59.762279", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T22:21:59.841 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.082) 0:02:38.824 ******* 2020-11-16T22:21:59.868 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:59.901 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.059) 0:02:38.884 ******* 2020-11-16T22:21:59.928 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:21:59.929 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:21:59.960 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:21:59 +0000 (0:00:00.059) 0:02:38.943 ******* 2020-11-16T22:22:00.074 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:22:00.076 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:22:00.109 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:00 +0000 (0:00:00.148) 0:02:39.091 ******* 2020-11-16T22:22:00.323 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:22:00.324 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0b0sOYm6YfrpL6cYgeWdfbvkfF2SQQjMp'], 'uuids': ['58c83e57-cd0b-4fb4-8906-b0e7a9224865']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.172301", "end": "2020-11-16 22:22:00.304781", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0b0sOYm6YfrpL6cYgeWdfbvkfF2SQQjMp"], "labels": [], "masters": [], "uuids": ["58c83e57-cd0b-4fb4-8906-b0e7a9224865"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:21:59.132480", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi106.front.sepia.ceph.com] => 2020-11-16T22:22:00.326 INFO:teuthology.task.ansible.out:(item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ', 'wwn-0x5000c50091e1aea6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', 'wwn-0x5000c50091e1aea6-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1aea6', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QHPJ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, "skip_reason": "Conditional result was False" 2020-11-16T22:22:00.327 INFO:teuthology.task.ansible.out:} 2020-11-16T22:22:00.336 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:00.539 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 2020-11-16T22:22:00.743 INFO:teuthology.task.ansible.out: 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'ho 2020-11-16T22:22:00.947 INFO:teuthology.task.ansible.out:st': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) = 2020-11-16T22:22:00.949 INFO:teuthology.task.ansible.out:> {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} Monday 16 November 2020 22:22:00 +0000 (0:00:00.351) 0:02:39.443 ******* TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} Monday 16 November 2020 22:22:00 +0000 (0:00:00.408) 0:02:39.851 ******* TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:22:00 +0000 (0:00:00.059) 0:02:39.910 ******* 2020-11-16T22:22:01.223 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:22:01.225 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:22:01.257 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.328) 0:02:40.239 ******* 2020-11-16T22:22:01.284 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:01.317 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.060) 0:02:40.299 ******* 2020-11-16T22:22:01.353 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:22:01.354 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:22:01.387 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.070) 0:02:40.370 ******* 2020-11-16T22:22:01.519 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:22:01.520 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ActiveEnterTimestampMonotonic": "17839639", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service systemd-journald.socket sysinit.target system.slice -.mount basic.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:38 UTC", "AssertTimestampMonotonic": "17639798", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ConditionTimestampMonotonic": "17639798", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-16T22:22:01.521 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1055", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:38 UTC", "ExecMainStartTimestampMonotonic": "17839629", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:38 UTC", "InactiveExitTimestampMonotonic": "17640723", "InvocationID": "668a734f19d1416ebe8fc8e51e653754", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-16T22:22:01.523 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1055", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:38 UTC", "S 2020-11-16T22:22:01.524 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "17839639", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:38 UTC", "WatchdogTimestampMonotonic": "17839638", "WatchdogUSec": "0"}} 2020-11-16T22:22:01.726 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.166) 0:02:40.537 ******* TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047719", "end": "2020-11-16 22:22:01.561870", "rc": 0, "start": "2020-11-16 22:22:01.514151", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} Monday 16 November 2020 22:22:01 +0000 (0:00:00.050) 0:02:40.587 ******* TASK [testnode : set_fact] ***************************************************** ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} Monday 16 November 2020 22:22:01 +0000 (0:00:00.040) 0:02:40.628 ******* TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:22:01 +0000 (0:00:00.040) 0:02:40.668 ***** 2020-11-16T22:22:01.728 INFO:teuthology.task.ansible.out:** TASK [testnode : set_fact] ***************************************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-GysNms-GImg-izaL-cl78-Yr4I-aeab-jdGvwW', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', 'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.133274", "end": "2020-11-16 22:22:01.670784", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-GysNms-GImg-izaL-cl78-Yr4I-aeab-jdGvwW", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode 2020-11-16T22:22:01.730 INFO:teuthology.task.ansible.out:": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:00.537510", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:01.757 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.071) 0:02:40.740 ******* 2020-11-16T22:22:01.789 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.031) 0:02:40.771 ******* 2020-11-16T22:22:01.823 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:01 +0000 (0:00:00.033) 0:02:40.805 ******* 2020-11-16T22:22:01.829 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:22:01.830 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:01.832 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:02.034 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': 2020-11-16T22:22:02.036 INFO:teuthology.task.ansible.out:[], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:22:01 +0000 (0:00:00.056) 0:02:40.862 ******* TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was Fa 2020-11-16T22:22:02.038 INFO:teuthology.task.ansible.out:lse"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters' 2020-11-16T22:22:02.039 INFO:teuthology.task.ansible.out:: [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 2020-11-16T22:22:02.041 INFO:teuthology.task.ansible.out:'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", 2020-11-16T22:22:02.243 INFO:teuthology.task.ansible.out:"links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, 2020-11-16T22:22:02.245 INFO:teuthology.task.ansible.out:"scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:02.654 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:22:02.656 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:22:02.693 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:02 +0000 (0:00:00.813) 0:02:41.676 ******* 2020-11-16T22:22:02.720 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T22:22:02.721 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:02.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:02 +0000 (0:00:00.059) 0:02:41.736 ******* 2020-11-16T22:22:02.759 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:22:02.760 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:22:02.793 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:02 +0000 (0:00:00.040) 0:02:41.776 ******* 2020-11-16T22:22:02.820 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:22:02.822 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:02.853 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:02 +0000 (0:00:00.059) 0:02:41.836 ******* 2020-11-16T22:22:02.974 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:22:02.975 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:22:03.007 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.154) 0:02:41.990 ******* 2020-11-16T22:22:03.054 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:22:03.056 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:22:03.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.082) 0:02:42.072 ******* 2020-11-16T22:22:03.116 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:22:03.118 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:03.149 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.059) 0:02:42.132 ******* 2020-11-16T22:22:03.297 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:22:03.299 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:22:03.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.181) 0:02:42.314 ******* 2020-11-16T22:22:03.348 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:22:03.350 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ActiveEnterTimestampMonotonic": "18030843", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target -.mount systemd-journald.socket sysinit.target basic.target system.slice systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:58 UTC", "AssertTimestampMonotonic": "17881819", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ConditionTimestampMonotonic": "17881819", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-16T22:22:03.351 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1128", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:58 UTC", "ExecMainStartTimestampMonotonic": "18030827", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:58 UTC", "InactiveExitTimestampMonotonic": "17882833", "InvocationID": "9d1ebff2ccd145c4888567b4552ac5e0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-16T22:22:03.353 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1128", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:58 UTC", "S 2020-11-16T22:22:03.355 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "18030843", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:58 UTC", "WatchdogTimestampMonotonic": "18030841", "WatchdogUSec": "0"}} 2020-11-16T22:22:03.385 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.053) 0:02:42.368 ******* 2020-11-16T22:22:03.411 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:03.413 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:22:03.446 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.060) 0:02:42.429 ******* 2020-11-16T22:22:03.474 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:03.475 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:03.507 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.061) 0:02:42.490 ******* 2020-11-16T22:22:03.534 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:22:03.536 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:03.568 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:03 +0000 (0:00:00.060) 0:02:42.550 ******* 2020-11-16T22:22:03.845 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:22:03.847 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0VvUfLrdg3v1YQUeyxHT3PCBzrREffdCM'], 'uuids': ['f2db210b-66dc-4ac3-90ec-dff707cc8420']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:01.715864", "end": "2020-11-16 22:22:03.821781", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0VvUfLrdg3v1YQUeyxHT3PCBzrREffdCM"], "labels": [], "masters": [], "uuids": ["f2db210b-66dc-4ac3-90ec-dff707cc8420"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:02.105917", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.7061 s, 67.6 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.7061 s, 67.6 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:04.313 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:22:04.315 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:22:04.347 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.779) 0:02:43.330 ******* 2020-11-16T22:22:04.395 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:22:04.396 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"cache_update_time": 1605565248, "cache_updated": false, "changed": false} 2020-11-16T22:22:04.429 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.082) 0:02:43.412 ******* 2020-11-16T22:22:04.456 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:22:04.458 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:04.489 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.059) 0:02:43.472 ******* 2020-11-16T22:22:04.608 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:22:04.610 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:22:04.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.153) 0:02:43.625 ******* 2020-11-16T22:22:04.836 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:22:04.838 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:22:04.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.230) 0:02:43.856 ******* 2020-11-16T22:22:04.899 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:22:04.901 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:22:04.933 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.059) 0:02:43.915 ******* 2020-11-16T22:22:04.948 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:22:04.950 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:22:04.983 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:04 +0000 (0:00:00.050) 0:02:43.966 ******* 2020-11-16T22:22:05.063 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:22:05.065 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564899.2922418, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564899.2882419, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564899.2882419, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:22:05.098 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.114) 0:02:44.081 ******* 2020-11-16T22:22:05.466 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:22:05.468 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:22:05.500 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.401) 0:02:44.482 ******* 2020-11-16T22:22:05.525 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:22:05.527 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:22:05.558 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.058) 0:02:44.541 ******* 2020-11-16T22:22:05.584 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:22:05.585 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:05.617 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.058) 0:02:44.599 ******* 2020-11-16T22:22:05.736 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:22:05.738 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0w2hMGg7QqGj5uy5W16tKuusIXbuH3aid'], 'uuids': ['0d43a544-71bf-44c1-8bff-258ad6679cda']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:01.727047", "end": "2020-11-16 22:22:05.717700", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0w2hMGg7QqGj5uy5W16tKuusIXbuH3aid"], "labels": [], "masters": [], "uuids": ["0d43a544-71bf-44c1-8bff-258ad6679cda"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:03.990653", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.71484 s, 67.3 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.71484 s, 67.3 MB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:22:05.771 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.153) 0:02:44.753 ******* 2020-11-16T22:22:05.776 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:22:05.778 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:22:05.811 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.040) 0:02:44.794 ******* 2020-11-16T22:22:05.890 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:22:05.891 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:22:05.924 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.113) 0:02:44.907 ******* 2020-11-16T22:22:05.950 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:22:05.952 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:05.983 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:05 +0000 (0:00:00.058) 0:02:44.966 ******* 2020-11-16T22:22:06.020 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:22:06.022 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.032 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.033 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.050 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.083 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.099) 0:02:45.065 ******* 2020-11-16T22:22:06.119 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:22:06.121 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.131 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.133 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.166 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.083) 0:02:45.148 ******* 2020-11-16T22:22:06.193 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:06.195 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.228 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.061) 0:02:45.210 ******* 2020-11-16T22:22:06.255 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:22:06.256 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.288 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.060) 0:02:45.270 ******* 2020-11-16T22:22:06.305 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:22:06.306 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:22:06.338 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.050) 0:02:45.321 ******* 2020-11-16T22:22:06.345 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:22:06.346 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.042) 0:02:45.363 ******* 2020-11-16T22:22:06.386 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:22:06.387 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:22:06.420 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.039) 0:02:45.402 ******* 2020-11-16T22:22:06.425 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:22:06.427 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.460 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.040) 0:02:45.442 ******* 2020-11-16T22:22:06.486 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:22:06.488 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.521 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.060) 0:02:45.503 ******* 2020-11-16T22:22:06.546 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:06.548 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.580 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.059) 0:02:45.563 ******* 2020-11-16T22:22:06.587 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:22:06.588 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564919.6436052, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564919.6396053, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564919.6396053, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:22:06.623 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.042) 0:02:45.606 ******* 2020-11-16T22:22:06.629 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:22:06.631 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.670 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.047) 0:02:45.653 ******* 2020-11-16T22:22:06.698 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:06.699 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.060) 0:02:45.713 ******* 2020-11-16T22:22:06.748 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:22:06.749 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:22:06.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.051) 0:02:45.764 ******* 2020-11-16T22:22:06.808 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:22:06.810 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:22:06.842 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.060) 0:02:45.825 ******* 2020-11-16T22:22:06.869 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:22:06.871 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:06.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:06 +0000 (0:00:00.060) 0:02:45.885 ******* 2020-11-16T22:22:06.910 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:22:06.911 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011011", "end": "2020-11-16 22:22:06.868261", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:22:06.857250", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:06.984 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006061", "end": "2020-11-16 22:22:06.980224", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:22:06.974163", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:07.026 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:22:07.028 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:22:07.059 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.156) 0:02:46.042 ******* 2020-11-16T22:22:07.096 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:22:07.098 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005399", "end": "2020-11-16 22:22:07.091784", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:22:07.086385", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:07.180 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:22:07.182 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:22:07.213 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.153) 0:02:46.196 ******* 2020-11-16T22:22:07.229 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:22:07.231 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.025614", "end": "2020-11-16 22:22:07.219301", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:22:07.193687", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:07.264 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.050) 0:02:46.247 ******* 2020-11-16T22:22:07.269 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:22:07.271 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.304 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.039) 0:02:46.286 ******* 2020-11-16T22:22:07.312 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:22:07.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.042) 0:02:46.329 ******* 2020-11-16T22:22:07.353 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:22:07.355 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi175.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.366 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.402 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.055) 0:02:46.384 ******* 2020-11-16T22:22:07.430 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:22:07.432 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.442 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.453 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.454 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.464 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.465 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:22:07.466 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:22:07.498 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.096) 0:02:46.481 ******* 2020-11-16T22:22:07.529 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:07.531 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.565 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.066) 0:02:46.548 ******* 2020-11-16T22:22:07.596 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:22:07.598 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.628 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.062) 0:02:46.611 ******* 2020-11-16T22:22:07.634 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:22:07.635 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0liBTTu5Z6iCng6WAUkStA2OSLpRGJhla'], 'uuids': ['f19551fd-2608-41d7-98a5-41c4fda5e621']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:01.714544", "end": "2020-11-16 22:22:07.592552", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0liBTTu5Z6iCng6WAUkStA2OSLpRGJhla"], "labels": [], "masters": [], "uuids": ["f19551fd-2608-41d7-98a5-41c4fda5e621"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:05.878008", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.70309 s, 67.7 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.70309 s, 67.7 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:07.646 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:22:07.648 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.681 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.053) 0:02:46.664 ******* 2020-11-16T22:22:07.708 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:22:07.709 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.059) 0:02:46.723 ******* 2020-11-16T22:22:07.768 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:22:07.769 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.802 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.061) 0:02:46.785 ******* 2020-11-16T22:22:07.829 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:07.831 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.864 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.061) 0:02:46.846 ******* 2020-11-16T22:22:07.890 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:22:07.891 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.925 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.060) 0:02:46.907 ******* 2020-11-16T22:22:07.952 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:07.953 INFO:teuthology.task.ansible.out:skipping: [smithi175.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:07.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:07 +0000 (0:00:00.060) 0:02:46.968 ******* 2020-11-16T22:22:08.136 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:22:08.138 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010739", "end": "2020-11-16 22:22:08.122509", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:22:08.111770", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:08.250 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006069", "end": "2020-11-16 22:22:08.240984", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:22:08.234915", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:08.365 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005420", "end": "2020-11-16 22:22:08.358611", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:22:08.353191", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:08.510 INFO:teuthology.task.ansible.out:ok: [smithi175.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.033497", "end": "2020-11-16 22:22:08.496574", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:22:08.463077", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:08.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:08 +0000 (0:00:00.564) 0:02:47.532 ******* 2020-11-16T22:22:08.577 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:22:08.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:08 +0000 (0:00:00.059) 0:02:47.592 ******* 2020-11-16T22:22:08.728 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:22:08.730 INFO:teuthology.task.ansible.out:changed: [smithi175.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:22:09.483 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:22:09.485 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0FMGXjqFrtXfc4Gv7iNMCdqVWGDbTbIld'], 'uuids': ['14022b15-6155-4c2e-b22d-835abb5733e4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:01.714551", "end": "2020-11-16 22:22:09.466901", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0FMGXjqFrtXfc4Gv7iNMCdqVWGDbTbIld"], "labels": [], "masters": [], "uuids": ["14022b15-6155-4c2e-b22d-835abb5733e4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:07.752350", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.70284 s, 67.7 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.70284 s, 67.7 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:11.361 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0b0sOYm6YfrpL6cYgeWdfbvkfF2SQQjMp'], 'uuids': ['58c83e57-cd0b-4fb4-8906-b0e7a9224865']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:01.704918", "end": "2020-11-16 22:22:11.335299", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Y0Pl9c4J0d56XDTI7dFyHcQfDyL11Uk0b0sOYm6YfrpL6cYgeWdfbvkfF2SQQjMp"], "labels": [], "masters": [], "uuids": ["58c83e57-cd0b-4fb4-8906-b0e7a9224865"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:09.630381", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.69156 s, 68.2 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.69156 s, 68.2 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:11.362 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ', 'wwn-0x5000c50091e1aea6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', 'wwn-0x5000c50091e1aea6-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1aea6', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QHPJ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, 2020-11-16T22:22:11.363 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.373 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.374 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.385 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.396 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.397 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.408 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.409 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:11.420 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:12.000 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-GysNms-GImg-izaL-cl78-Yr4I-aeab-jdGvwW', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', 'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.418622", "end": "2020-11-16 22:22:11.986159", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-GysNms-GImg-izaL-cl78-Yr4I-aeab-jdGvwW", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:22:11.567537", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.407392 s, 283 MB/s", "stde 2020-11-16T22:22:12.002 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.407392 s, 283 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:12.054 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:12 +0000 (0:00:03.444) 0:02:51.037 ******* 2020-11-16T22:22:14.303 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T22:22:14.305 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:02.078378", "end": "2020-11-16 22:22:14.287381", "rc": 0, "start": "2020-11-16 22:22:12.209003", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:14.342 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:02.287) 0:02:53.325 ******* 2020-11-16T22:22:14.616 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T22:22:14.618 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.091599", "end": "2020-11-16 22:22:14.596306", "rc": 0, "start": "2020-11-16 22:22:14.504707", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:14.657 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.314) 0:02:53.639 ******* 2020-11-16T22:22:14.683 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:22:14.724 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.066) 0:02:53.706 ******* 2020-11-16T22:22:14.751 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:14.792 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.068) 0:02:53.775 ******* 2020-11-16T22:22:14.818 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:22:14.819 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:14.856 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.063) 0:02:53.839 ******* 2020-11-16T22:22:14.884 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:22:14.885 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:14.924 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.068) 0:02:53.907 ******* 2020-11-16T22:22:14.950 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:22:14.952 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:14.988 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:14 +0000 (0:00:00.064) 0:02:53.971 ******* 2020-11-16T22:22:15.159 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:22:15.161 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:22:15.198 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.209) 0:02:54.181 ******* 2020-11-16T22:22:15.235 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:22:15.236 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'block_used': 14062243, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032865, 'size_available': 925830615040, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119632, 'block_size': 4096, 'inode_available': 60935344}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032865, "block_size": 4096, "block_total": 240095108, "block_used": 14062243, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935344, "inode_total": 61054976, "inode_used": 119632, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830615040, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.274 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.075) 0:02:54.257 ******* 2020-11-16T22:22:15.311 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:22:15.313 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.353 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.078) 0:02:54.336 ******* 2020-11-16T22:22:15.380 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:15.381 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.418 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.065) 0:02:54.401 ******* 2020-11-16T22:22:15.445 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:15.447 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.065) 0:02:54.466 ******* 2020-11-16T22:22:15.510 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:22:15.512 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.549 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.064) 0:02:54.531 ******* 2020-11-16T22:22:15.575 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:22:15.577 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.065) 0:02:54.597 ******* 2020-11-16T22:22:15.642 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:22:15.643 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:15.681 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:15 +0000 (0:00:00.066) 0:02:54.663 ******* 2020-11-16T22:22:16.099 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T22:22:16.100 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:22:16.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:16 +0000 (0:00:00.457) 0:02:55.120 ******* 2020-11-16T22:22:16.560 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:22:16.562 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:22:16.942 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:22:17.329 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:22:17.733 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:22:18.168 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:22:18.208 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:18 +0000 (0:00:02.070) 0:02:57.191 ******* 2020-11-16T22:22:18.390 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:22:18.392 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:22:18.429 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:18 +0000 (0:00:00.220) 0:02:57.412 ******* 2020-11-16T22:22:18.631 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:22:18.633 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:22:18.779 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:22:18.937 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:22:19.094 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:22:19.105 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:19.146 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:19 +0000 (0:00:00.716) 0:02:58.128 ******* 2020-11-16T22:22:19.327 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T22:22:19.329 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:22:19.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:19 +0000 (0:00:00.220) 0:02:58.349 ******* 2020-11-16T22:22:19.403 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:22:19.404 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:22:19.441 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:19 +0000 (0:00:00.074) 0:02:58.424 ******* 2020-11-16T22:22:19.747 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:22:19.748 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:22:19.785 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:19 +0000 (0:00:00.344) 0:02:58.768 ******* 2020-11-16T22:22:20.112 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:22:20.114 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:22:20.151 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:20 +0000 (0:00:00.365) 0:02:59.134 ******* 2020-11-16T22:22:20.342 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:22:20.344 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:22:20.382 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:20 +0000 (0:00:00.230) 0:02:59.364 ******* 2020-11-16T22:22:20.562 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:22:20.564 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:22:20.602 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:20 +0000 (0:00:00.219) 0:02:59.584 ******* 2020-11-16T22:22:21.073 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:22:21.075 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:22:21.112 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:21 +0000 (0:00:00.510) 0:03:00.095 ******* 2020-11-16T22:22:21.573 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:22:21.574 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T22:22:21.612 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:21 +0000 (0:00:00.499) 0:03:00.594 ******* 2020-11-16T22:22:22.672 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:22:22.673 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:22:22.710 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:22 +0000 (0:00:01.098) 0:03:01.693 ******* 2020-11-16T22:22:22.944 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T22:22:22.946 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.058738", "end": "2020-11-16 22:22:22.928676", "rc": 0, "start": "2020-11-16 22:22:22.869938", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T22:22:22.984 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:22 +0000 (0:00:00.273) 0:03:01.966 ******* 2020-11-16T22:22:23.011 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:22:23.012 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:23.050 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:23 +0000 (0:00:00.065) 0:03:02.032 ******* 2020-11-16T22:22:23.075 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:22:23.077 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:23.114 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:23 +0000 (0:00:00.064) 0:03:02.096 ******* 2020-11-16T22:22:24.140 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:22:24.142 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:22:24.178 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:24 +0000 (0:00:01.064) 0:03:03.161 ******* 2020-11-16T22:22:24.209 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:24.349 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:24 +0000 (0:00:00.170) 0:03:03.332 ******* 2020-11-16T22:22:24.810 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:22:24.812 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:22:24.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:24 +0000 (0:00:00.499) 0:03:03.831 ******* 2020-11-16T22:22:24.876 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:22:24.877 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:24.914 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:24 +0000 (0:00:00.065) 0:03:03.897 ******* 2020-11-16T22:22:25.213 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:22:25.214 INFO:teuthology.task.ansible.out:[WARNING]: The service (ntp) is actually an init script but the system is managed by systemd fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-16T22:22:25.251 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:25 +0000 (0:00:00.337) 0:03:04.234 ******* 2020-11-16T22:22:25.279 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:25.280 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:22:25.317 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:25 +0000 (0:00:00.065) 0:03:04.300 ******* 2020-11-16T22:22:25.344 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:25.345 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:25.382 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:25 +0000 (0:00:00.064) 0:03:04.365 ******* 2020-11-16T22:22:25.407 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:22:25.409 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:25.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:25 +0000 (0:00:00.063) 0:03:04.428 ******* 2020-11-16T22:22:26.507 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:22:26.509 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1605565249, "cache_updated": false, "changed": false} 2020-11-16T22:22:26.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:26 +0000 (0:00:01.099) 0:03:05.528 ******* 2020-11-16T22:22:26.572 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:22:26.574 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:26.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:26 +0000 (0:00:00.064) 0:03:05.593 ******* 2020-11-16T22:22:26.780 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:22:26.782 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:22:26.820 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:26 +0000 (0:00:00.209) 0:03:05.802 ******* 2020-11-16T22:22:27.290 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:22:27.292 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:22:27.329 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:27 +0000 (0:00:00.508) 0:03:06.311 ******* 2020-11-16T22:22:28.346 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:22:28.347 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:22:28.392 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:28 +0000 (0:00:01.062) 0:03:07.374 ******* 2020-11-16T22:22:29.030 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:22:29.032 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:22:29.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.677) 0:03:08.052 ******* 2020-11-16T22:22:29.095 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:22:29.097 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:22:29.133 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.064) 0:03:08.116 ******* 2020-11-16T22:22:29.324 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:22:29.326 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564909.2195847, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564909.2155848, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564909.2155848, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:22:29.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.229) 0:03:08.346 ******* 2020-11-16T22:22:29.534 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:22:29.536 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:22:29.575 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.212) 0:03:08.558 ******* 2020-11-16T22:22:29.602 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:22:29.604 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:22:29.640 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.065) 0:03:08.623 ******* 2020-11-16T22:22:29.667 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:22:29.669 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:29.706 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.065) 0:03:08.689 ******* 2020-11-16T22:22:29.878 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:22:29.880 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:22:29.919 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:29 +0000 (0:00:00.213) 0:03:08.902 ******* 2020-11-16T22:22:30.104 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:22:30.106 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:22:30.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.223) 0:03:09.126 ******* 2020-11-16T22:22:30.170 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.208 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.065) 0:03:09.191 ******* 2020-11-16T22:22:30.246 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:22:30.247 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.258 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.259 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.088) 0:03:09.279 ******* 2020-11-16T22:22:30.334 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:22:30.336 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.346 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.347 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.357 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.397 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.099) 0:03:09.379 ******* 2020-11-16T22:22:30.423 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:22:30.425 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.066) 0:03:09.446 ******* 2020-11-16T22:22:30.490 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:22:30.492 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.528 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.065) 0:03:09.511 ******* 2020-11-16T22:22:30.555 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:22:30.557 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.595 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.066) 0:03:09.578 ******* 2020-11-16T22:22:30.622 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:22:30.624 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.064) 0:03:09.643 ******* 2020-11-16T22:22:30.687 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:22:30.689 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.070) 0:03:09.713 ******* 2020-11-16T22:22:30.756 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:30.757 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.796 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.065) 0:03:09.778 ******* 2020-11-16T22:22:30.822 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.862 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.065) 0:03:09.844 ******* 2020-11-16T22:22:30.888 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:22:30.890 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:22:30.926 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:30 +0000 (0:00:00.064) 0:03:09.909 ******* 2020-11-16T22:22:31.148 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:22:31.150 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.021466", "end": "2020-11-16 22:22:31.120362", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:22:31.098896", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:31.326 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.015099", "end": "2020-11-16 22:22:31.304720", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:22:31.289621", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:31.491 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.014938", "end": "2020-11-16 22:22:31.477241", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:22:31.462303", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:31.677 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.026170", "end": "2020-11-16 22:22:31.656772", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:22:31.630602", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:22:31.717 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:31 +0000 (0:00:00.790) 0:03:10.699 ******* 2020-11-16T22:22:31.742 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:22:31.781 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:22:31 +0000 (0:00:00.064) 0:03:10.763 ******* 2020-11-16T22:22:31.952 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:22:31.954 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:22:32.056 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2020-11-16T22:22:32.058 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-11-16T22:22:32.059 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-16T22:22:32.062 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-11-16T22:22:32.064 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi106.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 smithi175.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 smithi181.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 Monday 16 November 2020 22:22:32 +0000 (0:00:00.282) 0:03:11.046 ******* =============================================================================== Gathering Facts --------------------------------------------------------- 5.92s common : Update apt cache ----------------------------------------------- 4.97s testnode : Update apt cache. -------------------------------------------- 4.18s testnode : Ensure packages are not present. ----------------------------- 3.59s users : Determine if we can skip users and pubkeys updates -------------- 2.59s testnode : Remove all LVM data ------------------------------------------ 2.29s testnode : Create logical volume(s) ------------------------------------- 2.07s users : Remove managed_admin_users from managed_users ------------------- 1.66s testnode : Install python-pip on apt based systems. --------------------- 1.10s testnode : Ensure perl-doc is installed on apt systems. ----------------- 1.10s testnode : Install non aarch64 packages --------------------------------- 1.10s testnode : Upgrade packages --------------------------------------------- 1.09s testnode : Install python-apt ------------------------------------------- 1.08s testnode : Install ntp package on apt based systems. -------------------- 1.06s testnode : Install packages via pip ------------------------------------- 1.06s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.79s Gathering Facts --------------------------------------------------------- 0.78s test 2020-11-16T22:22:32.065 INFO:teuthology.task.ansible.out:node : Write /scratch_devs ------------------------------------------ 0.72s testnode : Purge resolvconf --------------------------------------------- 0.68s testnode : Upload weekly kernel-clean crontab. -------------------------- 0.52s 2020-11-16T22:22:32.372 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com')] 2020-11-16T22:22:32.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:22:32.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:22:32.445 INFO:teuthology.orchestra.run.smithi181:> true 2020-11-16T22:22:32.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:22:32.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:22:32.720 INFO:teuthology.orchestra.run.smithi175:> true 2020-11-16T22:22:32.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:22:32.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:22:32.964 INFO:teuthology.orchestra.run.smithi106:> true 2020-11-16T22:22:33.247 INFO:teuthology.run_tasks:Running task clock... 2020-11-16T22:22:33.258 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-16T22:22:33.258 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:22:33.259 INFO:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:22:33.312 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:22:33.312 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Command line: ntpd -gq 2020-11-16T22:22:33.313 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: proto: precision = 0.123 usec (-23) 2020-11-16T22:22:33.314 INFO:teuthology.orchestra.run.smithi181.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:33.314 INFO:teuthology.orchestra.run.smithi181.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:33.314 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:33.315 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:33.315 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:22:33.315 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:22:33.316 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:22:33.316 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen normally on 3 enp3s0f1 172.21.15.181:123 2020-11-16T22:22:33.316 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen normally on 4 lo [::1]:123 2020-11-16T22:22:33.316 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7b9d%5]:123 2020-11-16T22:22:33.316 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:33 ntpd[6303]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:22:36.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:36 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:36.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:36 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:36.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:36 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:36.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:36 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:37.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:37.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:37.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:37 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:38.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:38.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:38.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:38.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:38.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:38.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:38.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:38.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:38 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:39 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:40 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:41.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:41.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:41.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:41.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:41.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:41.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:41.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:41.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:41 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:22:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:16 Nov 22:22:42 ntpd[6303]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:22:42.315 INFO:teuthology.orchestra.run.smithi181.stdout:16 Nov 22:22:42 ntpd[6303]: ntpd: time slew -0.005093 s 2020-11-16T22:22:42.315 INFO:teuthology.orchestra.run.smithi181.stdout:ntpd: time slew -0.005093s 2020-11-16T22:22:42.365 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:22:42.365 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2020-11-16T22:22:42.365 INFO:teuthology.orchestra.run.smithi181.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:42.365 INFO:teuthology.orchestra.run.smithi181.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:42.365 INFO:teuthology.orchestra.run.smithi181.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:42.366 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:42.366 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:22:42.367 INFO:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:22:42.420 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:22:42.420 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Command line: ntpd -gq 2020-11-16T22:22:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: proto: precision = 0.087 usec (-23) 2020-11-16T22:22:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:22:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen normally on 3 enp3s0f1 172.21.15.175:123 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen normally on 4 lo [::1]:123 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:71f1%5]:123 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:42 ntpd[6336]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:22:42.423 INFO:teuthology.orchestra.run.smithi175.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:42.424 INFO:teuthology.orchestra.run.smithi175.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:43.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:43.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:43.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:43.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:43.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:43.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:43.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:43.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:43 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:44.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:44 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:44.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:44 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:44.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:44 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:44.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:44 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:45.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:45.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:45.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:45.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:45.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:45.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:45.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:45.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:45 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:46.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:46.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:46.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:46 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:47.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:47.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:47.423 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:47 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:48.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:48.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:48.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:48.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:48.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:48.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:48.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:48.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:48 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:49.421 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:49.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:49.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:49.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:49.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:22:49.422 INFO:teuthology.orchestra.run.smithi175.stderr:16 Nov 22:22:49 ntpd[6336]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:22:49.423 INFO:teuthology.orchestra.run.smithi175.stdout:16 Nov 22:22:49 ntpd[6336]: ntpd: time slew -0.006280 s 2020-11-16T22:22:49.423 INFO:teuthology.orchestra.run.smithi175.stdout:ntpd: time slew -0.006280s 2020-11-16T22:22:49.477 INFO:teuthology.orchestra.run.smithi175.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:22:49.477 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================== 2020-11-16T22:22:49.477 INFO:teuthology.orchestra.run.smithi175.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:49.477 INFO:teuthology.orchestra.run.smithi175.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:49.478 INFO:teuthology.orchestra.run.smithi175.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:49.478 INFO:teuthology.orchestra.run.smithi175.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:49.479 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:22:49.479 INFO:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:22:49.547 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:22:49.547 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Command line: ntpd -gq 2020-11-16T22:22:49.549 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: proto: precision = 0.044 usec (-24) 2020-11-16T22:22:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:49.550 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:22:49.551 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:22:49.551 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen normally on 3 enp3s0f1 172.21.15.106:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen normally on 4 lo [::1]:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce6d%5]:123 2020-11-16T22:22:49.552 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:49 ntpd[5139]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:22:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:50 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:52.548 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:52.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:52.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:52.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:52.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:52.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:52.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:52.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:52.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:52.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:52 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:54.548 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:54.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:54.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:54.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:54.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:54.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:54.551 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:54 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:56.548 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:22:56.548 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:22:56.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:22:56.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:22:56.549 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:22:56.550 INFO:teuthology.orchestra.run.smithi106.stderr:16 Nov 22:22:56 ntpd[5139]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:22:56.550 INFO:teuthology.orchestra.run.smithi106.stdout:16 Nov 22:22:56 ntpd[5139]: ntpd: time slew -0.006211 s 2020-11-16T22:22:56.550 INFO:teuthology.orchestra.run.smithi106.stdout:ntpd: time slew -0.006211s 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================== 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:56.653 INFO:teuthology.orchestra.run.smithi106.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:22:56.655 INFO:teuthology.run_tasks:Running task install... 2020-11-16T22:22:56.666 DEBUG:teuthology.task.install:project ceph 2020-11-16T22:22:56.666 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}} 2020-11-16T22:22:56.667 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'} 2020-11-16T22:22:56.667 INFO:teuthology.task.install:Using flavor: basic 2020-11-16T22:22:56.673 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-16T22:22:56.673 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-16T22:22:56.674 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2020-11-16T22:22:56.728 INFO:teuthology.orchestra.run.smithi181.stdout:Ubuntu 2020-11-16T22:22:56.733 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:56.734 INFO:teuthology.orchestra.run.smithi175:> sudo lsb_release -is 2020-11-16T22:22:56.735 INFO:teuthology.orchestra.run.smithi181:> sudo apt-key list | grep Ceph 2020-11-16T22:22:56.788 INFO:teuthology.orchestra.run.smithi175.stdout:Ubuntu 2020-11-16T22:22:56.793 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:56.794 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2020-11-16T22:22:56.794 INFO:teuthology.orchestra.run.smithi175:> sudo apt-key list | grep Ceph 2020-11-16T22:22:56.823 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:22:56.868 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2020-11-16T22:22:56.876 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:56.877 INFO:teuthology.orchestra.run.smithi106:> sudo apt-key list | grep Ceph 2020-11-16T22:22:56.884 INFO:teuthology.orchestra.run.smithi175.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:22:56.937 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:22:56.937 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:22:56.938 INFO:teuthology.orchestra.run.smithi175:> sudo lsb_release -is 2020-11-16T22:22:56.948 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:22:56.948 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:22:56.949 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2020-11-16T22:22:56.989 INFO:teuthology.orchestra.run.smithi175.stdout:Ubuntu 2020-11-16T22:22:56.994 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:56.994 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:22:56.995 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:22:56.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:22:56.999 INFO:teuthology.orchestra.run.smithi181.stdout:Ubuntu 2020-11-16T22:22:57.003 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:57.003 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:22:57.003 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:22:57.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:22:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:22:57.159 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:22:57.159 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:22:57.164 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:22:57.164 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:22:57.214 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:22:57.214 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:22:57.217 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2020-11-16T22:22:57.286 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:22:57.287 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.289 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:22:57.289 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.290 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2020-11-16T22:22:57.295 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.298 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:22:57.298 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:22:57.298 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:22:57.299 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:22:57.303 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:22:57.303 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:22:57.303 INFO:teuthology.orchestra.run.smithi175.stderr:172 bytes copied, 3.7044e-05 s, 4.6 MB/s 2020-11-16T22:22:57.304 INFO:teuthology.orchestra.run.smithi175:> sudo apt-get update 2020-11-16T22:22:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:22:57.305 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:22:57.305 INFO:teuthology.orchestra.run.smithi181.stderr:172 bytes copied, 7.4033e-05 s, 2.3 MB/s 2020-11-16T22:22:57.305 INFO:teuthology.orchestra.run.smithi181:> sudo apt-get update 2020-11-16T22:22:57.464 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:22:57.464 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:22:57.498 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:22:57.574 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:22:57.581 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:22:57.588 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:22:57.589 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.598 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:22:57.603 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:22:57.604 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:22:57.617 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:22:57.618 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:22:57.618 INFO:teuthology.orchestra.run.smithi106.stderr:172 bytes copied, 0.000220893 s, 779 kB/s 2020-11-16T22:22:57.619 INFO:teuthology.orchestra.run.smithi106:> sudo apt-get update 2020-11-16T22:22:57.621 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:22:57.623 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:22:57.629 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:22:57.632 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:22:57.633 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:22:57.648 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:22:57.652 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:22:57.683 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:22:57.686 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:22:57.711 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:22:57.712 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:22:57.791 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:22:57.792 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:22:57.888 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:22:57.976 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:22:57.988 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:22:57.999 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:22:58.026 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:22:58.050 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:22:58.071 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:22:58.086 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 26.8 kB in 1s (46.0 kB/s) 2020-11-16T22:22:58.095 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:22:58.145 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:22:58.218 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 26.8 kB in 1s (37.7 kB/s) 2020-11-16T22:22:59.175 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 26.8 kB in 1s (22.3 kB/s) 2020-11-16T22:22:59.391 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2020-11-16T22:22:59.439 INFO:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:22:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2020-11-16T22:22:59.489 INFO:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:22:59.530 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2020-11-16T22:22:59.569 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2020-11-16T22:22:59.809 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2020-11-16T22:22:59.810 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2020-11-16T22:22:59.846 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2020-11-16T22:22:59.847 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2020-11-16T22:22:59.985 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2020-11-16T22:22:59.985 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:22:59.985 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:22:59.986 INFO:teuthology.orchestra.run.smithi181.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:22:59.986 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:22:59.986 INFO:teuthology.orchestra.run.smithi181.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:22:59.987 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:22:59.988 INFO:teuthology.orchestra.run.smithi181.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:22:59.989 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:22:59.989 INFO:teuthology.orchestra.run.smithi181.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:22:59.989 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:22:59.989 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:22:59.989 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2020-11-16T22:22:59.990 INFO:teuthology.orchestra.run.smithi181.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:22:59.990 INFO:teuthology.orchestra.run.smithi181.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:22:59.990 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:22:59.990 INFO:teuthology.orchestra.run.smithi181.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:22:59.990 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:22:59.991 INFO:teuthology.orchestra.run.smithi181.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:22:59.992 INFO:teuthology.orchestra.run.smithi181.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:22:59.992 INFO:teuthology.orchestra.run.smithi181.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:22:59.992 INFO:teuthology.orchestra.run.smithi181.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:23:00.034 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2020-11-16T22:23:00.034 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:23:00.034 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:23:00.035 INFO:teuthology.orchestra.run.smithi175.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:23:00.035 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:23:00.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:23:00.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:23:00.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:23:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2020-11-16T22:23:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:23:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:23:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:23:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:23:00.040 INFO:teuthology.orchestra.run.smithi175.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:23:00.041 INFO:teuthology.orchestra.run.smithi175.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:23:00.041 INFO:teuthology.orchestra.run.smithi175.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:23:00.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:23:00.041 INFO:teuthology.orchestra.run.smithi175.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:23:00.090 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2020-11-16T22:23:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:23:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:23:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:23:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:23:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:23:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:23:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:23:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:23:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:23:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:23:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages will be upgraded: 2020-11-16T22:23:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 librbd1 2020-11-16T22:23:00.140 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2020-11-16T22:23:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:23:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:23:00.342 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:23:00.344 INFO:teuthology.orchestra.run.smithi181.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:23:00.344 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 168 MB of archives. 2020-11-16T22:23:00.344 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi175.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi175.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:23:00.345 INFO:teuthology.orchestra.run.smithi175.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:23:00.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:23:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:23:00.348 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:23:00.348 INFO:teuthology.orchestra.run.smithi175.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:23:00.348 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:23:00.348 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:23:00.348 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:23:00.550 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages will be upgraded: 2020-11-16T22:23:00.550 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 librbd1 2020-11-16T22:23:00.550 INFO:teuthology.orchestra.run.smithi175.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:23:00.551 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 168 MB of archives. 2020-11-16T22:23:00.551 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:23:00.551 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:23:00.551 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:23:00.552 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:23:00.552 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:23:00.552 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:23:00.553 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:23:00.563 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:23:00.563 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:23:00.587 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:23:00.592 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:23:00.602 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:23:00.602 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:23:00.614 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:23:00.649 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:23:00.659 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:23:00.660 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:23:00.864 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2020-11-16T22:23:00.864 INFO:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:23:00.865 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:23:00.865 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:23:00.865 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:23:00.865 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:23:00.866 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:23:00.867 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:23:00.867 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:23:01.070 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:23:01.070 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:23:01.070 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:23:01.071 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:23:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:23:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:23:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:23:01.072 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:23:01.074 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:23:01.075 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:23:01.075 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:23:01.075 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:23:01.075 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:23:01.075 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:23:01.076 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:23:01.076 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:23:01.278 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:23:01.278 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:23:01.278 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:23:01.278 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi175.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi175.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi175.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi175.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:23:01.279 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:23:01.280 INFO:teuthology.orchestra.run.smithi181.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:23:01.281 INFO:teuthology.orchestra.run.smithi181.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:23:01.281 INFO:teuthology.orchestra.run.smithi181.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:23:01.281 INFO:teuthology.orchestra.run.smithi181.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:23:01.482 INFO:teuthology.orchestra.run.smithi181.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:23:01.482 INFO:teuthology.orchestra.run.smithi181.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:23:01.482 INFO:teuthology.orchestra.run.smithi181.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:23:01.485 INFO:teuthology.orchestra.run.smithi175.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:23:01.485 INFO:teuthology.orchestra.run.smithi175.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:23:01.485 INFO:teuthology.orchestra.run.smithi175.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:23:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:23:01.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:23:01.488 INFO:teuthology.orchestra.run.smithi175.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:23:01.488 INFO:teuthology.orchestra.run.smithi175.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:23:01.488 INFO:teuthology.orchestra.run.smithi181.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:23:01.488 INFO:teuthology.orchestra.run.smithi181.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:23:01.488 INFO:teuthology.orchestra.run.smithi181.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:23:01.690 INFO:teuthology.orchestra.run.smithi181.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:23:01.690 INFO:teuthology.orchestra.run.smithi181.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:23:01.690 INFO:teuthology.orchestra.run.smithi181.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:23:01.690 INFO:teuthology.orchestra.run.smithi181.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:23:01.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:23:01.692 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2020-11-16T22:23:01.692 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2020-11-16T22:23:01.692 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2020-11-16T22:23:01.694 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2020-11-16T22:23:01.694 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:23:01.694 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:23:01.694 INFO:teuthology.orchestra.run.smithi106.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:23:01.695 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:23:01.696 INFO:teuthology.orchestra.run.smithi106.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:23:01.696 INFO:teuthology.orchestra.run.smithi106.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:23:01.696 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:23:01.696 INFO:teuthology.orchestra.run.smithi106.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:23:01.696 INFO:teuthology.orchestra.run.smithi106.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:23:01.902 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:23:01.902 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:23:01.902 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:23:01.903 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:23:01.904 INFO:teuthology.orchestra.run.smithi106.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:23:01.905 INFO:teuthology.orchestra.run.smithi106.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:23:01.906 INFO:teuthology.orchestra.run.smithi106.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:23:01.906 INFO:teuthology.orchestra.run.smithi106.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:23:01.906 INFO:teuthology.orchestra.run.smithi175.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:23:01.906 INFO:teuthology.orchestra.run.smithi181.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:23:01.906 INFO:teuthology.orchestra.run.smithi181.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:23:01.907 INFO:teuthology.orchestra.run.smithi181.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:23:02.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:23:02.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:23:02.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:23:02.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:23:02.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:23:02.111 INFO:teuthology.orchestra.run.smithi181.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:23:02.111 INFO:teuthology.orchestra.run.smithi181.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:23:02.115 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2020-11-16T22:23:02.115 INFO:teuthology.orchestra.run.smithi106.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:23:02.115 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:23:02.116 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:23:02.117 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:23:02.118 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:23:02.118 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:23:02.118 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:23:02.118 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:23:02.322 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:23:02.322 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:23:02.322 INFO:teuthology.orchestra.run.smithi106.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages will be upgraded: 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 librbd1 2020-11-16T22:23:02.323 INFO:teuthology.orchestra.run.smithi106.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 168 MB of archives. 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi175.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:23:02.324 INFO:teuthology.orchestra.run.smithi175.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:23:02.325 INFO:teuthology.orchestra.run.smithi175.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:23:02.326 INFO:teuthology.orchestra.run.smithi175.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:23:02.326 INFO:teuthology.orchestra.run.smithi175.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:23:02.326 INFO:teuthology.orchestra.run.smithi175.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:23:02.326 INFO:teuthology.orchestra.run.smithi175.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:23:02.530 INFO:teuthology.orchestra.run.smithi175.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:23:02.530 INFO:teuthology.orchestra.run.smithi175.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:23:02.530 INFO:teuthology.orchestra.run.smithi175.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:23:02.530 INFO:teuthology.orchestra.run.smithi175.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:23:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:23:02.533 INFO:teuthology.orchestra.run.smithi181.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:23:02.534 INFO:teuthology.orchestra.run.smithi181.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:23:02.738 INFO:teuthology.orchestra.run.smithi181.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:23:02.738 INFO:teuthology.orchestra.run.smithi181.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:23:02.738 INFO:teuthology.orchestra.run.smithi181.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:23:02.738 INFO:teuthology.orchestra.run.smithi181.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:23:02.739 INFO:teuthology.orchestra.run.smithi181.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:23:02.740 INFO:teuthology.orchestra.run.smithi181.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:23:02.741 INFO:teuthology.orchestra.run.smithi181.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:23:02.741 INFO:teuthology.orchestra.run.smithi181.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:23:02.744 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:23:02.744 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:23:02.745 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:23:02.745 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:23:02.745 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:23:02.950 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:23:02.950 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:23:02.950 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:23:02.950 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:23:02.951 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:23:02.952 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:23:02.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:23:02.954 INFO:teuthology.orchestra.run.smithi175.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:23:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:23:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:23:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:23:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:23:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:23:03.159 INFO:teuthology.orchestra.run.smithi175.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:23:03.159 INFO:teuthology.orchestra.run.smithi175.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:03.159 INFO:teuthology.orchestra.run.smithi175.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:23:03.159 INFO:teuthology.orchestra.run.smithi175.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:23:03.159 INFO:teuthology.orchestra.run.smithi181.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:03.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:23:03.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:23:03.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:23:03.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:23:03.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:23:03.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:23:03.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:23:03.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:23:03.364 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:23:03.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:39 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:23:03.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:23:03.367 INFO:teuthology.orchestra.run.smithi175.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:23:03.367 INFO:teuthology.orchestra.run.smithi175.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:23:03.367 INFO:teuthology.orchestra.run.smithi175.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:23:03.367 INFO:teuthology.orchestra.run.smithi181.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:23:03.570 INFO:teuthology.orchestra.run.smithi181.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:23:03.571 INFO:teuthology.orchestra.run.smithi106.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:23:03.572 INFO:teuthology.orchestra.run.smithi106.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:23:03.573 INFO:teuthology.orchestra.run.smithi106.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:23:03.574 INFO:teuthology.orchestra.run.smithi175.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:23:03.617 INFO:teuthology.orchestra.run.smithi106.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:23:03.618 INFO:teuthology.orchestra.run.smithi106.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:23:03.620 INFO:teuthology.orchestra.run.smithi106.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:23:03.624 INFO:teuthology.orchestra.run.smithi106.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:23:03.626 INFO:teuthology.orchestra.run.smithi106.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi106.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi106.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi106.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:23:03.852 INFO:teuthology.orchestra.run.smithi106.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:23:03.853 INFO:teuthology.orchestra.run.smithi106.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:23:03.854 INFO:teuthology.orchestra.run.smithi106.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:23:03.858 INFO:teuthology.orchestra.run.smithi106.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:23:03.930 INFO:teuthology.orchestra.run.smithi106.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:23:03.932 INFO:teuthology.orchestra.run.smithi106.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:23:03.968 INFO:teuthology.orchestra.run.smithi175.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi106.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:23:04.037 INFO:teuthology.orchestra.run.smithi106.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:23:04.038 INFO:teuthology.orchestra.run.smithi106.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:23:04.039 INFO:teuthology.orchestra.run.smithi106.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:23:04.045 INFO:teuthology.orchestra.run.smithi106.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:23:04.246 INFO:teuthology.orchestra.run.smithi106.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:23:04.248 INFO:teuthology.orchestra.run.smithi106.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:23:04.248 INFO:teuthology.orchestra.run.smithi106.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:23:04.269 INFO:teuthology.orchestra.run.smithi106.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:23:04.271 INFO:teuthology.orchestra.run.smithi106.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:23:04.328 INFO:teuthology.orchestra.run.smithi106.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:23:04.330 INFO:teuthology.orchestra.run.smithi106.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:23:04.395 INFO:teuthology.orchestra.run.smithi106.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:23:04.959 INFO:teuthology.orchestra.run.smithi181.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:23:04.960 INFO:teuthology.orchestra.run.smithi181.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:23:04.983 INFO:teuthology.orchestra.run.smithi181.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:23:05.133 INFO:teuthology.orchestra.run.smithi181.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:23:05.134 INFO:teuthology.orchestra.run.smithi181.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:23:05.136 INFO:teuthology.orchestra.run.smithi181.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:23:05.257 INFO:teuthology.orchestra.run.smithi106.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:23:05.261 INFO:teuthology.orchestra.run.smithi106.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:23:05.343 INFO:teuthology.orchestra.run.smithi181.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:23:05.383 INFO:teuthology.orchestra.run.smithi106.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:23:05.887 INFO:teuthology.orchestra.run.smithi175.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:23:05.887 INFO:teuthology.orchestra.run.smithi175.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:23:05.889 INFO:teuthology.orchestra.run.smithi181.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:23:05.889 INFO:teuthology.orchestra.run.smithi181.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:23:05.889 INFO:teuthology.orchestra.run.smithi181.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:23:05.931 INFO:teuthology.orchestra.run.smithi175.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:23:06.094 INFO:teuthology.orchestra.run.smithi175.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:23:06.096 INFO:teuthology.orchestra.run.smithi175.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:23:06.100 INFO:teuthology.orchestra.run.smithi175.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:23:06.334 INFO:teuthology.orchestra.run.smithi175.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:23:06.966 INFO:teuthology.orchestra.run.smithi175.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:23:06.966 INFO:teuthology.orchestra.run.smithi175.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:23:06.972 INFO:teuthology.orchestra.run.smithi175.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:23:09.410 INFO:teuthology.orchestra.run.smithi181.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:23:09.411 INFO:teuthology.orchestra.run.smithi181.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:23:09.411 INFO:teuthology.orchestra.run.smithi181.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:23:10.112 INFO:teuthology.orchestra.run.smithi181.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:23:10.336 INFO:teuthology.orchestra.run.smithi106.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:23:10.598 INFO:teuthology.orchestra.run.smithi175.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:23:10.598 INFO:teuthology.orchestra.run.smithi175.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:23:10.599 INFO:teuthology.orchestra.run.smithi175.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:23:10.814 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 168 MB in 10s (16.8 MB/s) 2020-11-16T22:23:10.935 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:23:10.974 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:23:10.977 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:23:11.017 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:23:11.029 INFO:teuthology.orchestra.run.smithi175.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:23:11.038 INFO:teuthology.orchestra.run.smithi106.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:23:11.044 INFO:teuthology.orchestra.run.smithi106.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:23:11.156 INFO:teuthology.orchestra.run.smithi106.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:23:11.319 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:23:11.331 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:23:11.369 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:23:11.658 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:23:11.669 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:23:11.671 INFO:teuthology.orchestra.run.smithi106.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:23:11.699 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:23:11.746 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 168 MB in 11s (15.5 MB/s) 2020-11-16T22:23:11.902 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:23:11.944 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:23:11.947 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:23:11.992 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:23:12.060 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:23:12.072 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:23:12.110 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:23:12.294 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:23:12.306 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:23:12.345 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:23:12.421 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:23:12.434 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:23:12.513 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:23:12.647 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:23:12.659 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:23:12.688 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:23:12.846 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:12.984 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:13.000 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:23:13.011 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:23:13.050 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:23:13.393 INFO:teuthology.orchestra.run.smithi106.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:23:13.394 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:23:13.394 INFO:teuthology.orchestra.run.smithi106.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:23:13.406 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:23:13.424 INFO:teuthology.orchestra.run.smithi106.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:23:13.486 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:23:13.508 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:13.544 INFO:teuthology.orchestra.run.smithi106.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:23:13.556 INFO:teuthology.orchestra.run.smithi106.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:23:13.558 INFO:teuthology.orchestra.run.smithi106.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:23:13.629 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:13.743 INFO:teuthology.orchestra.run.smithi106.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:23:13.826 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:13.931 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:14.222 INFO:teuthology.orchestra.run.smithi106.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:23:14.223 INFO:teuthology.orchestra.run.smithi106.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:23:14.223 INFO:teuthology.orchestra.run.smithi106.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:23:14.300 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:23:14.312 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:14.350 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:14.481 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:14.604 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:14.619 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:23:14.631 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:14.661 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.030 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:23:15.041 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:15.080 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.200 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:23:15.211 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:15.241 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.391 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:23:15.396 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:15.433 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.577 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:23:15.589 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:15.627 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.727 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:23:15.739 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:15.777 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:15.955 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:23:15.967 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:16.004 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:16.088 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:23:16.099 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:23:16.138 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:23:16.315 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:23:16.326 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:16.365 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:16.374 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:23:16.386 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:16.424 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:16.693 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:23:16.705 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:16.743 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:16.794 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:23:16.805 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:23:16.843 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:23:17.088 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:23:17.099 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:23:17.302 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:23:17.303 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:23:17.303 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:23:17.303 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:23:17.390 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:23:17.403 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:17.440 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:17.476 INFO:teuthology.orchestra.run.smithi106.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:23:17.477 INFO:teuthology.orchestra.run.smithi106.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:23:17.477 INFO:teuthology.orchestra.run.smithi106.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:23:17.533 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:23:17.543 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:17.582 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:17.793 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:23:17.805 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:23:18.060 INFO:teuthology.orchestra.run.smithi106.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:23:18.127 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:23:18.139 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:18.171 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:23:18.183 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:23:18.201 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:18.246 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:23:18.521 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:23:18.534 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:23:18.571 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:23:18.582 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:23:18.593 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:18.632 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:18.832 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:23:18.844 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:23:18.890 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:23:19.088 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 168 MB in 16s (10.4 MB/s) 2020-11-16T22:23:19.226 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:23:19.237 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:19.261 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:23:19.268 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:19.285 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:23:19.297 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:23:19.329 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104570 files and directories currently installed.) 2020-11-16T22:23:19.332 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:23:19.335 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:23:19.375 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:23:19.587 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:23:19.599 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:23:19.654 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:23:19.662 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:23:19.666 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:19.704 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:19.736 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:23:19.750 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:23:19.795 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:23:19.940 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:23:19.951 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:23:19.990 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:23:20.116 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:23:20.128 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:20.156 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:23:20.166 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:20.172 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:23:20.214 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:23:20.368 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:23:20.381 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:23:20.418 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:23:20.592 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:23:20.606 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:23:20.650 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:23:20.729 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:23:20.740 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:20.779 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:21.028 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:23:21.043 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:23:21.111 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:23:21.199 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:21.249 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:21.504 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:21.657 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:22.047 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:23:22.059 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:22.097 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:22.284 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:22.444 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:23:23.018 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:23:23.030 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:23:23.068 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:23:23.121 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:23:23.133 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:23.140 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:23:23.154 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:23.190 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:23.213 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:23.396 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:23:23.410 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:23:23.534 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:23:23.547 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:23.592 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:23.740 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:23:23.752 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:23:23.782 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:24.036 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:24.076 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:23:24.087 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:23:24.126 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:23:24.171 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:23:24.209 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:23:24.397 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:24.447 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:24.480 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:23:24.492 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:23:24.504 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:23:24.515 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:23:24.554 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:23:24.563 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:23:24.767 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:23:24.781 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:24.825 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:24.849 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:23:24.857 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:23:24.861 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:24.869 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:23:24.899 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:24.906 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:23:25.179 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:23:25.194 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:23:25.195 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:23:25.208 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:23:25.209 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:23:25.210 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:23:25.236 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:23:25.252 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:23:25.539 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:23:25.552 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:23:25.563 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:23:25.571 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:25.596 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:23:25.608 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:23:25.623 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:23:25.630 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:25.646 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:23:25.916 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:23:25.928 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:25.968 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:23:25.980 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:23:25.986 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:25.999 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:23:26.015 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:23:26.018 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:23:26.049 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:23:26.312 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:23:26.325 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:23:26.329 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:23:26.341 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:23:26.378 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:23:26.397 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:23:26.402 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:23:26.418 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:23:26.460 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:23:26.706 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:23:26.719 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:23:26.756 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:23:26.815 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:23:26.826 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:23:26.855 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:23:26.865 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:23:26.869 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:26.913 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:27.085 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:23:27.097 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:23:27.135 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:23:27.143 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:23:27.156 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:23:27.193 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:23:27.463 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:23:27.475 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:23:27.504 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:23:27.512 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:23:27.523 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:23:27.523 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:23:27.539 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:27.562 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:23:27.582 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:27.790 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:23:27.802 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:23:27.840 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:23:27.898 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:23:27.911 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:23:27.918 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:23:27.932 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:23:27.948 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:23:27.968 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:23:28.168 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:23:28.179 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:23:28.218 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:23:28.235 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:23:28.246 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:23:28.285 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:23:28.320 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:23:28.336 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:23:28.379 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:23:28.529 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:23:28.540 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:23:28.579 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:23:28.579 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:23:28.591 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:23:28.629 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:23:28.823 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:23:28.838 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:23:28.882 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:23:28.882 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:23:28.893 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:23:28.932 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:23:28.932 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:23:28.944 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:23:28.982 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:23:29.210 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:23:29.222 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:23:29.235 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:23:29.252 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:29.260 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:23:29.285 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:23:29.293 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:29.298 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:23:29.335 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:23:29.596 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:23:29.608 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:23:29.637 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:23:29.646 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:23:29.649 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:23:29.688 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:23:29.796 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:23:29.811 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:29.854 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:30.032 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:23:30.044 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:23:30.082 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:23:30.394 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:23:30.404 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:23:30.443 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:23:30.729 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:23:30.742 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:23:30.779 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:23:31.107 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:23:31.111 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:23:31.119 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:23:31.124 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:23:31.149 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:23:31.165 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:23:31.462 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:23:31.473 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:23:31.493 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:23:31.506 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:23:31.543 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:23:31.846 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:23:31.857 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:23:31.896 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:23:32.199 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:23:32.211 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:23:32.212 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:23:32.249 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:23:32.855 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:23:32.867 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:23:32.905 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:23:33.044 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:23:33.054 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:33.093 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:33.249 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:23:33.262 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:23:33.299 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:23:33.498 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:23:33.512 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:23:33.556 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:23:33.654 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:23:33.666 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:23:33.704 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:23:33.943 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:23:33.957 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:23:34.001 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:23:34.089 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:23:34.090 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:23:34.099 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:34.103 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:34.138 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:34.140 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:34.345 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:23:34.361 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:23:34.403 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:23:34.627 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:23:34.640 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:34.685 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:34.781 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:23:34.796 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:23:34.839 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:23:35.301 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:23:35.316 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:23:35.368 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:23:35.648 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:36.144 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph. 2020-11-16T22:23:36.157 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:36.216 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:23:36.227 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:36.230 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:23:36.266 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:23:36.547 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:23:36.558 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:36.596 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:36.619 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:23:36.634 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:23:36.669 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:23:36.950 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:23:36.961 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:37.441 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:23:37.456 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:23:37.503 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:23:37.514 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:37.516 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:23:37.553 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:37.736 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph. 2020-11-16T22:23:37.749 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:37.778 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:37.852 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:23:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:23:37.872 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:23:37.884 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:23:37.922 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:23:37.927 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:23:38.055 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:23:38.068 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:38.105 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:38.233 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:23:38.246 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:38.263 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:23:38.278 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:23:38.300 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:38.347 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:23:38.534 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:23:38.547 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:38.584 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:38.603 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:23:38.614 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:23:38.653 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:23:38.720 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:23:38.735 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:23:38.783 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:23:38.939 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:23:38.952 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:23:38.989 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:23:39.078 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:23:39.089 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:39.128 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:39.185 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:23:39.200 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:23:39.244 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:23:39.309 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:23:39.321 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:39.384 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:39.414 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:23:39.427 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:23:39.464 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:23:39.558 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:23:39.573 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:23:39.613 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:23:39.817 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:23:39.830 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:23:40.226 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:23:40.238 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:23:40.275 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:23:40.287 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:23:40.299 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:23:40.362 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:23:40.390 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:23:40.405 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:23:40.451 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:23:40.656 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:23:40.809 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:23:40.826 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:23:40.863 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:23:40.864 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:23:40.878 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:23:40.914 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:23:40.943 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:23:40.955 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:40.993 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:23:41.315 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:23:41.643 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:23:41.659 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:23:41.918 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:23:41.931 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:23:41.968 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:23:42.105 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:23:42.120 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:23:42.163 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:23:42.517 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:23:42.531 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:23:42.532 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:23:42.532 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:23:42.543 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:23:42.546 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:23:42.575 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:23:42.581 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:23:42.590 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:23:42.885 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:23:42.897 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:23:42.934 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:23:42.935 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:23:42.953 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:23:42.993 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:23:43.012 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:23:43.024 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:23:43.062 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:23:43.388 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:23:43.403 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:23:43.407 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:23:43.418 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:23:43.446 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:23:43.456 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:23:43.876 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:23:43.890 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:23:43.934 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:23:43.988 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:23:44.001 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:44.038 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:23:44.346 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:23:44.362 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:44.403 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:44.604 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:23:44.617 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:23:44.654 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:23:44.954 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:44.998 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:45.007 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:23:45.019 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:23:45.057 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:23:45.117 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:23:45.129 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:23:45.174 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:23:45.647 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:23:45.659 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:45.697 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:45.847 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:23:45.860 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:23:45.897 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:23:46.249 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:23:46.536 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:23:46.550 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:23:46.594 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:23:46.817 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:23:46.830 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:23:46.867 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:23:46.906 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:23:46.919 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:23:46.947 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:23:47.153 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:23:47.166 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:23:47.203 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:23:47.267 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:23:47.281 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:23:47.325 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:23:47.531 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:23:47.543 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:23:47.581 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:23:47.755 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:23:47.768 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:23:47.821 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:23:47.900 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:23:47.914 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:23:47.958 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:23:48.132 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:23:48.146 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:23:48.182 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:23:48.287 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:23:48.299 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:23:48.307 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph. 2020-11-16T22:23:48.322 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:48.337 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:23:48.365 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:48.536 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:23:48.548 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:23:48.577 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:23:48.648 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:23:48.660 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:23:48.698 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:23:48.751 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:23:48.766 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:23:49.035 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:23:49.047 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:23:49.085 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:23:49.263 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:23:49.277 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:49.321 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:23:49.559 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:23:49.597 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:23:49.745 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:23:49.759 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:23:49.803 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:23:49.840 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:23:49.855 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:49.890 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:49.933 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:23:49.945 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:23:49.983 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:23:50.252 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:23:50.267 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:23:50.289 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:23:50.310 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:23:50.395 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:23:50.409 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:23:50.445 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:23:50.572 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:23:50.586 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:50.622 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:23:50.900 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:23:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:23:50.941 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:23:51.141 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:23:51.155 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:23:51.199 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:23:51.236 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:23:51.248 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:51.285 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:51.328 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:23:51.340 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:23:51.378 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:23:51.552 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:23:51.566 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:23:51.771 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package jq. 2020-11-16T22:23:51.771 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:23:51.771 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:23:51.771 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:23:51.772 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:23:51.772 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:23:51.772 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:23:51.954 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:23:51.958 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package socat. 2020-11-16T22:23:51.969 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:51.970 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:23:52.008 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:23:52.008 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:23:52.013 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:52.020 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:23:52.139 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:23:52.379 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:23:52.391 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:23:52.428 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:23:52.472 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:23:52.484 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:52.521 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:52.690 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:23:52.703 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:23:52.748 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:23:52.849 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:23:52.862 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:23:52.899 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:23:53.077 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:23:53.090 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:53.127 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:53.219 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:23:53.422 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:23:53.423 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:23:53.423 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:23:53.423 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:23:53.423 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package jq. 2020-11-16T22:23:53.550 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:23:53.588 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:23:53.874 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package socat. 2020-11-16T22:23:53.886 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:23:53.924 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:23:54.026 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:23:54.041 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:23:54.084 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:23:54.322 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:23:54.335 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:23:54.372 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:23:54.445 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:23:54.462 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:23:54.503 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:23:54.666 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:23:54.679 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:23:54.716 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:23:55.062 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:23:55.076 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:55.153 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:55.970 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:23:55.985 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:23:56.027 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:23:56.556 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:23:56.569 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:23:56.614 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:23:56.881 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:23:56.893 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:23:56.931 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:23:57.042 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:23:57.057 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:23:57.101 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:23:57.459 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:23:57.470 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:57.509 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:57.761 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:23:57.773 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:57.811 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:57.832 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:23:57.848 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:57.890 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:58.056 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:23:58.068 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:23:58.105 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:23:58.460 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:23:58.472 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:23:58.534 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:23:58.824 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:23:58.836 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:23:58.837 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:23:58.851 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:23:58.865 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:23:58.887 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:23:59.109 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:23:59.122 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:23:59.159 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:59.160 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:23:59.173 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:23:59.176 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:23:59.185 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:23:59.215 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:23:59.218 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:23:59.428 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:23:59.442 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:23:59.487 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:23:59.501 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-py. 2020-11-16T22:23:59.512 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:23:59.521 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:23:59.536 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:23:59.739 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:23:59.751 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:23:59.789 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:23:59.905 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:23:59.917 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:23:59.954 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:23:59.956 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:23:59.972 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:24:00.015 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:24:00.118 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:24:00.132 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:24:00.168 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:24:00.283 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:24:00.294 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:24:00.333 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:24:00.384 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:24:00.400 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:24:00.443 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:24:00.471 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:24:00.483 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:24:00.521 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:24:00.660 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:24:00.671 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:24:00.710 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:24:00.846 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:24:00.863 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:24:00.904 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:24:01.014 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:24:01.025 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:24:01.063 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:24:01.090 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:24:01.103 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:24:01.140 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:24:01.282 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:24:01.297 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:24:01.325 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:24:01.338 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:24:01.340 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:24:01.383 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:01.443 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-py. 2020-11-16T22:24:01.455 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:24:01.517 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:24:01.702 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:24:01.718 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:24:01.744 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:24:01.756 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:24:01.760 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:24:01.794 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:01.854 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:24:01.868 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:24:01.912 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:24:02.231 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:24:02.243 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:24:02.272 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:24:02.281 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:24:02.282 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:24:02.287 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:24:02.295 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:24:02.330 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:24:02.332 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:24:02.610 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:24:02.623 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:24:02.625 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:02.651 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:24:02.725 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:24:02.740 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:24:02.792 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:24:02.971 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:24:02.982 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:24:03.245 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:24:03.262 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:24:03.281 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:24:03.293 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:24:03.303 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:24:03.307 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:24:03.321 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:24:03.331 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:24:03.365 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:03.633 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:24:03.645 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:24:03.673 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:24:03.683 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:04.046 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:24:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:24:04.104 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:04.238 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:24:04.251 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:04.465 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:24:04.478 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:24:04.515 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:04.615 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:24:04.629 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:24:04.673 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:04.688 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:24:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:24:04.771 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:24:04.784 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:24:04.796 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:24:04.834 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:24:05.146 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:24:05.159 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:24:05.188 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:05.190 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:24:05.207 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:24:05.304 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:24:05.318 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:24:05.353 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:24:05.354 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:24:05.490 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:24:05.502 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:24:05.540 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:24:05.673 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:24:05.679 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:24:05.686 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:24:05.695 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:24:05.737 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:05.748 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:24:05.877 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:24:05.889 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:24:05.968 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:06.090 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:24:06.093 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:24:06.106 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:24:06.110 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:24:06.143 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:06.148 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:24:06.305 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:24:06.318 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:24:06.355 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:06.513 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:24:06.518 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:24:06.526 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:24:06.534 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:24:06.563 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:06.576 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:24:06.792 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:24:06.806 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:06.866 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:24:06.878 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:24:06.887 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package jq. 2020-11-16T22:24:06.903 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:24:06.915 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:24:06.945 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:24:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package socat. 2020-11-16T22:24:07.260 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:24:07.272 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:24:07.274 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:24:07.306 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:24:07.318 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:07.605 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:24:07.618 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:24:07.663 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:24:07.862 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:24:07.875 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:07.902 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:24:07.912 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:07.915 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:24:07.959 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:24:08.032 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:24:08.047 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:24:08.091 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:08.238 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:24:08.312 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:24:08.370 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:24:08.402 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:24:08.416 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:24:08.460 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:08.534 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:24:08.833 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:24:08.850 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:08.891 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:08.902 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:24:08.914 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:24:08.928 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:08.972 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:09.020 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:24:09.656 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:10.009 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:24:10.022 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:10.040 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:24:10.058 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:24:10.634 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:24:10.663 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:24:10.811 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:24:10.945 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:24:11.023 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:24:11.149 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:24:11.280 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:24:11.641 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:24:11.752 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:11.941 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:24:12.120 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:24:12.234 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:12.388 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:24:12.544 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:24:12.680 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:24:12.837 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:24:12.907 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:24:12.963 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:24:13.033 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:24:13.087 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:24:13.103 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:24:13.145 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:24:13.359 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:24:13.472 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:24:13.483 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:24:13.522 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:13.719 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:24:13.809 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:24:13.826 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:13.867 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:13.891 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:24:13.987 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:24:14.169 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:24:14.181 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:24:14.217 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:24:14.219 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:24:14.288 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:14.476 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:24:14.581 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:24:14.640 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:24:14.655 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:24:14.678 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:14.698 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:24:14.890 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:24:15.016 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:24:15.021 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:24:15.034 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:24:15.049 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:24:15.084 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:24:15.111 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:24:15.364 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:24:15.421 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:24:15.437 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:24:15.470 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:24:15.515 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:24:15.716 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:24:15.857 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-py. 2020-11-16T22:24:15.872 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:24:15.915 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:24:15.953 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:24:16.001 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:24:16.277 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:24:16.279 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:24:16.326 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:24:16.342 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:24:16.555 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:24:16.578 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:24:16.729 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:24:16.744 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:24:16.781 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:16.787 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:24:16.854 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:24:17.132 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:24:17.141 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:24:17.148 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:24:17.190 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:24:17.222 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:24:17.230 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:24:17.475 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:24:17.576 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:24:17.591 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:24:17.602 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:24:17.607 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:24:17.819 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:24:17.908 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:18.004 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:24:18.019 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:24:18.062 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:18.104 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:24:18.405 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:24:18.473 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:24:18.489 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:24:18.531 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:18.534 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user cephadm....done 2020-11-16T22:24:18.723 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:24:18.731 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:24:19.016 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:24:19.024 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:24:19.052 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:24:19.067 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:24:19.110 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:24:19.142 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:24:19.268 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:24:19.425 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:24:19.471 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:24:19.486 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:24:19.529 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:19.603 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:24:19.751 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:19.912 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:24:20.039 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:24:20.069 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:20.327 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:24:20.342 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:24:20.373 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:20.409 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user cephadm....done 2020-11-16T22:24:20.735 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:24:20.745 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:24:20.761 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:24:20.829 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:24:20.833 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:24:21.034 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:24:21.290 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:21.294 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:24:21.420 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:21.712 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:24:21.728 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:24:21.746 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:24:21.762 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:21.855 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:24:21.944 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:24:21.977 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:24:22.006 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:24:22.106 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:24:22.132 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:24:22.148 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:22.516 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:24:22.576 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:24:22.593 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:24:22.634 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:22.751 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:24:22.817 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:24:23.029 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:24:23.035 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:24:23.045 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:24:23.087 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:24:23.161 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:24:23.185 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:24:23.487 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:24:23.487 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:23.491 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:24:23.505 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:24:23.549 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:23.671 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:24:23.814 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:24:23.886 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:24:23.919 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:24:23.969 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:24:23.985 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:24:24.026 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:24.089 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:24:24.140 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:24:24.179 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:24:24.424 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:24:24.441 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:24:24.555 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:24:24.565 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:24.601 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:24:24.606 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:24.727 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:24.751 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:24:24.978 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:24:25.088 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:24:25.094 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:24:25.418 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:24:25.481 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:24:25.607 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:24:25.629 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:24:25.734 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:24:25.749 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:24:25.792 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:25.900 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:24:26.022 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:24:26.093 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:24:26.160 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:24:26.201 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:24:26.357 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:24:26.485 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:24:26.522 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:24:26.533 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:24:26.659 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:26.778 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:24:26.911 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:24:26.965 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:24:27.037 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:24:27.091 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:24:27.130 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:24:27.396 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:24:27.431 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:27.497 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:24:27.732 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:24:27.798 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:24:27.860 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:24:28.083 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:24:28.192 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:24:28.329 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:24:28.375 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:24:28.467 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:24:28.663 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:24:28.685 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:24:28.803 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:24:29.023 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:24:29.070 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:24:29.121 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:24:29.272 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:24:29.299 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:24:29.371 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:29.425 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:24:29.663 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:24:29.835 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:24:29.956 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:24:30.230 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:24:30.241 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:24:30.435 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:30.604 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:24:30.642 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:24:30.736 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:24:30.952 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:24:30.981 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:24:31.111 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:24:31.188 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:24:31.361 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:24:31.573 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:24:31.725 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:24:31.874 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:24:31.884 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:24:32.117 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:24:32.218 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:24:32.274 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:32.328 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:32.487 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:24:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:24:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:32.935 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:24:33.030 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:24:33.057 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:33.061 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:24:33.182 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:24:33.422 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:24:33.462 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:24:33.738 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:24:33.798 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:24:33.862 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:24:33.981 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:24:34.098 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:24:34.132 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:24:34.224 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:34.247 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:34.350 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:24:34.550 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:24:34.648 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:24:34.668 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:34.794 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:34.920 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:34.935 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:24:35.029 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:24:35.061 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:24:35.175 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:24:35.304 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:24:35.416 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:24:35.664 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:24:35.717 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:24:35.996 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:24:36.023 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:24:36.043 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:36.297 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:36.424 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:24:36.440 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:36.733 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:36.805 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:24:36.850 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:36.896 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:24:37.047 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:24:37.047 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:24:37.260 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:24:37.319 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:37.437 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:37.562 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:24:37.677 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:24:37.678 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:24:37.700 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:24:37.718 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:24:38.071 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.152 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.445 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.727 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:38.757 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:24:38.758 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:24:38.827 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:24:39.047 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.094 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user cephadm....done 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.272 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.286 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:24:39.290 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:24:39.380 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:24:39.397 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.413 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:24:39.523 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:39.654 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:24:39.719 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:24:39.780 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:24:39.922 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:24:40.017 INFO:teuthology.orchestra.run.smithi181.stdout:Adding group ceph....done 2020-11-16T22:24:40.324 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:24:40.372 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:40.404 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user ceph....done 2020-11-16T22:24:40.472 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:24:40.580 INFO:teuthology.orchestra.run.smithi181.stdout:Setting system user ceph properties....done 2020-11-16T22:24:40.708 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:24:40.738 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:24:40.782 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:40.834 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:24:40.933 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:40.942 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:24:41.058 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:41.193 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:41.235 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:41.244 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:41.468 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:24:41.469 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:24:41.662 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:24:41.820 INFO:teuthology.orchestra.run.smithi175.stdout:Adding group ceph....done 2020-11-16T22:24:41.895 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:42.021 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:24:42.030 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:42.148 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:24:42.205 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user ceph....done 2020-11-16T22:24:42.367 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:24:42.423 INFO:teuthology.orchestra.run.smithi175.stdout:Setting system user ceph properties....done 2020-11-16T22:24:42.549 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:24:42.580 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:24:42.648 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:42.800 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:24:42.863 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:24:42.864 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:24:42.983 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:24:43.082 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:24:43.095 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:43.120 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:24:43.141 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:43.360 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:24:43.361 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:24:43.477 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:24:43.494 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:43.772 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:43.837 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:24:43.872 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:43.904 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:24:43.904 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:24:44.187 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:44.190 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:24:44.205 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:24:44.322 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:44.465 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:44.491 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:44.614 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:24:44.685 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:24:44.685 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:24:44.704 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:24:44.704 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:24:44.957 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:24:44.974 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:44.975 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.206 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:24:45.206 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:24:45.216 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:24:45.484 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.585 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.588 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:24:45.588 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:24:45.703 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.743 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:24:45.804 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:24:45.840 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:24:45.879 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:45.997 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:24:46.005 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:46.114 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:46.162 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:24:46.328 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:24:46.329 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:24:46.354 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:24:46.481 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:24:46.582 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:46.782 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:24:46.799 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:24:46.800 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:24:46.900 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:24:47.078 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:47.245 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:47.372 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:47.410 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:24:47.473 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:24:47.535 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:24:47.674 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:24:47.904 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:24:48.024 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:24:48.067 INFO:teuthology.orchestra.run.smithi181.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:24:48.125 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:24:48.271 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:24:48.639 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:24:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:24:48.835 INFO:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:24:48.919 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2020-11-16T22:24:49.028 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:24:49.179 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2020-11-16T22:24:49.179 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2020-11-16T22:24:49.355 INFO:teuthology.orchestra.run.smithi181.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:24:49.355 INFO:teuthology.orchestra.run.smithi181.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:24:49.355 INFO:teuthology.orchestra.run.smithi181.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:24:49.355 INFO:teuthology.orchestra.run.smithi181.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:24:49.355 INFO:teuthology.orchestra.run.smithi181.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:24:49.356 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2020-11-16T22:24:49.356 INFO:teuthology.orchestra.run.smithi181.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:24:49.356 INFO:teuthology.orchestra.run.smithi181.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:24:49.356 INFO:teuthology.orchestra.run.smithi181.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:24:49.357 INFO:teuthology.orchestra.run.smithi181.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:24:49.357 INFO:teuthology.orchestra.run.smithi181.stdout: zlib1g-dev 2020-11-16T22:24:49.358 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2020-11-16T22:24:49.358 INFO:teuthology.orchestra.run.smithi181.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:24:49.358 INFO:teuthology.orchestra.run.smithi181.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:24:49.396 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2020-11-16T22:24:49.396 INFO:teuthology.orchestra.run.smithi181.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:24:49.397 INFO:teuthology.orchestra.run.smithi181.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:24:49.397 INFO:teuthology.orchestra.run.smithi181.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:24:49.397 INFO:teuthology.orchestra.run.smithi181.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:24:49.398 INFO:teuthology.orchestra.run.smithi181.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:24:49.398 INFO:teuthology.orchestra.run.smithi181.stdout: zlib1g-dev 2020-11-16T22:24:49.483 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:24:49.597 INFO:teuthology.orchestra.run.smithi181.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:24:49.598 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:24:49.598 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:24:49.598 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:24:49.676 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:24:49.718 INFO:teuthology.orchestra.run.smithi175.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:24:49.776 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:24:49.868 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:24:50.045 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:24:50.127 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:24:50.176 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:24:50.196 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:24:50.207 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:24:50.209 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:24:50.210 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:24:50.213 INFO:teuthology.orchestra.run.smithi175.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:24:50.215 INFO:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:24:50.225 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:24:50.228 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:24:50.229 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:24:50.231 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:24:50.231 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:24:50.289 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:24:50.290 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:24:50.296 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2020-11-16T22:24:50.299 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:24:50.300 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:24:50.369 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:24:50.370 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:24:50.372 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:24:50.391 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:24:50.392 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:24:50.494 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:24:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:24:50.547 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:24:50.554 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2020-11-16T22:24:50.555 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2020-11-16T22:24:50.569 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:24:50.573 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:24:50.573 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:24:50.574 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:24:50.607 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:24:50.637 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:24:50.740 INFO:teuthology.orchestra.run.smithi175.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:24:50.740 INFO:teuthology.orchestra.run.smithi175.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:24:50.740 INFO:teuthology.orchestra.run.smithi175.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:24:50.740 INFO:teuthology.orchestra.run.smithi175.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:24:50.741 INFO:teuthology.orchestra.run.smithi175.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:24:50.741 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2020-11-16T22:24:50.741 INFO:teuthology.orchestra.run.smithi175.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:24:50.741 INFO:teuthology.orchestra.run.smithi175.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:24:50.742 INFO:teuthology.orchestra.run.smithi175.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:24:50.742 INFO:teuthology.orchestra.run.smithi175.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:24:50.742 INFO:teuthology.orchestra.run.smithi175.stdout: zlib1g-dev 2020-11-16T22:24:50.743 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2020-11-16T22:24:50.743 INFO:teuthology.orchestra.run.smithi175.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:24:50.743 INFO:teuthology.orchestra.run.smithi175.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:24:50.781 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2020-11-16T22:24:50.781 INFO:teuthology.orchestra.run.smithi175.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:24:50.782 INFO:teuthology.orchestra.run.smithi175.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:24:50.782 INFO:teuthology.orchestra.run.smithi175.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:24:50.783 INFO:teuthology.orchestra.run.smithi175.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:24:50.783 INFO:teuthology.orchestra.run.smithi175.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:24:50.783 INFO:teuthology.orchestra.run.smithi175.stdout: zlib1g-dev 2020-11-16T22:24:50.970 INFO:teuthology.orchestra.run.smithi175.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:24:50.970 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:24:50.970 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:24:50.971 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:24:51.013 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:24:51.093 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 7,861 kB in 1s (6,585 kB/s) 2020-11-16T22:24:51.215 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package flex. 2020-11-16T22:24:51.260 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 110001 files and directories currently installed.) 2020-11-16T22:24:51.263 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:24:51.306 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:24:51.406 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:24:51.432 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:24:51.501 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:24:51.560 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:24:51.590 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:24:51.597 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:24:51.599 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:24:51.600 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:24:51.614 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:24:51.618 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:24:51.619 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:24:51.620 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:24:51.662 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:24:51.701 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:24:51.705 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:24:51.712 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:24:51.713 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:24:51.715 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:24:51.725 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:24:51.726 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:24:51.727 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:24:51.749 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:24:51.749 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:24:51.760 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:24:51.832 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:24:51.833 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:24:51.841 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:24:51.868 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:24:51.882 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:24:51.883 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:24:51.915 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:24:51.924 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:24:51.925 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:24:51.926 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:24:51.981 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:24:51.992 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:24:52.045 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package bison. 2020-11-16T22:24:52.057 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:24:52.091 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:24:52.470 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 7,861 kB in 1s (6,650 kB/s) 2020-11-16T22:24:52.507 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package dump. 2020-11-16T22:24:52.519 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:24:52.557 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:24:52.608 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package flex. 2020-11-16T22:24:52.650 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 110001 files and directories currently installed.) 2020-11-16T22:24:52.653 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:24:52.698 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:24:52.853 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package indent. 2020-11-16T22:24:52.864 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:24:52.903 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:24:53.102 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:24:53.113 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:24:53.155 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:24:53.160 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:24:53.168 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:24:53.197 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:24:53.458 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:24:53.513 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package bison. 2020-11-16T22:24:53.525 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:24:53.584 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:24:53.770 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:24:53.782 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:24:53.798 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:24:53.819 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:24:53.908 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package dump. 2020-11-16T22:24:53.921 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:24:53.958 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:24:54.089 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:24:54.101 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:24:54.131 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:24:54.200 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:24:54.220 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package indent. 2020-11-16T22:24:54.233 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:24:54.303 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:24:54.326 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:24:54.442 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:24:54.455 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:24:54.534 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:24:54.572 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:24:54.586 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:24:54.630 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:24:54.769 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:24:54.812 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:24:54.825 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:24:54.862 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:24:54.874 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:24:54.886 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:24:54.924 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:24:55.120 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:24:55.157 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:24:55.171 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:55.198 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:24:55.203 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:24:55.215 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:24:55.252 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:24:55.413 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:24:55.451 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:24:55.462 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:55.501 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:24:55.522 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:24:55.533 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:24:55.539 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:24:55.572 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:24:55.665 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:55.824 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:24:55.837 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:24:55.848 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:24:55.887 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:24:55.925 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:24:55.938 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:24:56.017 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:24:56.198 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:24:56.201 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:56.209 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:24:56.240 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:24:56.277 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:24:56.290 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:24:56.327 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:56.328 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:24:56.453 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:24:56.535 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:24:56.547 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:24:56.584 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:24:56.595 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:24:56.680 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:24:56.693 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:56.730 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:24:56.958 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:24:56.970 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:56.971 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:24:56.983 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:24:57.020 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:24:57.057 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:24:57.092 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:24:57.299 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:24:57.310 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:24:57.349 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:24:57.425 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:24:57.428 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:24:57.441 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:24:57.477 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:24:57.668 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:24:57.680 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:57.718 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:24:57.838 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:24:57.843 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:24:57.852 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:24:57.897 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:24:58.037 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:24:58.050 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:58.087 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:24:58.183 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:24:58.195 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:24:58.225 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:24:58.340 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:24:58.351 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:24:58.390 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:24:58.611 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:24:58.623 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:24:58.669 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:24:58.855 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:24:59.022 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:24:59.034 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:24:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:24:59.261 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:24:59.272 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:24:59.311 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:24:59.391 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:24:59.403 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:59.441 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:24:59.647 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:24:59.659 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:24:59.698 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:24:59.710 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:24:59.724 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:24:59.768 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:24:59.892 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:25:00.038 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:25:00.050 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:25:00.088 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:25:00.252 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:00.353 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:25:00.381 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:25:00.404 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:00.532 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:25:00.542 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:00.545 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:25:00.582 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:25:00.780 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:00.905 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:00.969 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:25:00.982 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:25:01.017 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:25:01.030 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:25:01.036 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:01.086 INFO:teuthology.orchestra.run.smithi181.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:25:01.086 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:25:01.161 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:25:01.162 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:25:01.187 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:01.605 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:25:01.618 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:25:01.683 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:25:01.684 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:01.694 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:25:01.698 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:01.816 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:01.870 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:01.942 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:25:02.074 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:25:02.278 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:25:02.345 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ppp. 2020-11-16T22:25:02.356 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:25:02.506 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:25:02.518 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:25:02.566 INFO:teuthology.orchestra.run.smithi175.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:25:02.566 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:25:02.669 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:02.675 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:03.016 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:25:03.030 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:25:03.075 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:03.085 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:25:03.099 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:25:03.171 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:03.340 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:03.453 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:25:03.466 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:25:03.482 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:25:03.503 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:03.739 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:25:03.751 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:25:03.789 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:03.841 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ppp. 2020-11-16T22:25:03.855 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:25:03.926 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:04.077 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:04.132 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:04.203 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:04.329 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:04.335 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:25:04.349 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:25:04.402 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:04.471 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:25:04.484 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:25:04.513 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:04.686 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:04.815 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:04.924 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:25:04.937 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:25:04.952 INFO:teuthology.orchestra.run.smithi106.stdout:Adding group ceph....done 2020-11-16T22:25:04.983 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:05.235 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:25:05.249 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:25:05.293 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:05.424 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user ceph....done 2020-11-16T22:25:05.669 INFO:teuthology.orchestra.run.smithi106.stdout:Setting system user ceph properties....done 2020-11-16T22:25:05.700 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:25:05.713 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:25:05.726 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:25:05.772 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:05.846 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:25:05.875 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:25:05.972 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:06.072 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:06.098 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:25:06.199 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:25:06.201 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:06.243 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:25:06.259 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:06.410 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:06.704 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:06.839 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:07.056 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:25:07.069 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:07.069 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:07.126 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:25:07.126 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:25:07.340 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:07.394 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:25:07.491 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:07.617 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:07.794 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:07.826 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:07.916 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:07.928 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:07.952 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:25:08.042 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:25:08.042 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:08.054 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:25:08.104 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:08.172 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:08.230 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:25:08.281 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:25:08.356 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:08.504 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:25:08.531 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:25:08.565 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:08.593 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:25:08.719 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:25:08.845 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:25:08.965 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:08.966 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:08.970 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:25:09.097 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:25:09.215 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:09.243 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:09.295 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:09.378 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:09.487 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:09.559 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:25:09.559 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:25:09.655 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:09.762 INFO:teuthology.orchestra.run.smithi181.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:25:09.781 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:09.932 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:25:10.024 INFO:teuthology.orchestra.run.smithi181.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:25:10.058 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:10.155 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:10.155 INFO:teuthology.orchestra.run.smithi181.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:25:10.184 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:25:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:10.305 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:10.310 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:25:10.408 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:25:10.479 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:25:10.488 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:10.604 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:25:10.631 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:25:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:25:10.757 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:10.856 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:25:10.971 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:25:10.971 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:25:10.982 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:25:11.108 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:11.337 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:11.337 INFO:teuthology.orchestra.run.smithi181.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:25:11.338 INFO:teuthology.orchestra.run.smithi181.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:25:11.338 INFO:teuthology.orchestra.run.smithi181.stdout:- enp3s0f1 2020-11-16T22:25:11.338 INFO:teuthology.orchestra.run.smithi181.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:25:11.338 INFO:teuthology.orchestra.run.smithi181.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:25:11.338 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:11.470 INFO:teuthology.orchestra.run.smithi175.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:25:11.472 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:25:11.533 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:11.667 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:25:11.718 INFO:teuthology.orchestra.run.smithi175.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:25:11.835 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:11.851 INFO:teuthology.orchestra.run.smithi175.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:25:11.887 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:11.945 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:25:12.013 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:25:12.105 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:25:12.105 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:25:12.130 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:12.257 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:25:12.382 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:12.478 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:12.628 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:12.654 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:12.915 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:25:12.915 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:25:12.922 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:12.922 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:13.114 INFO:teuthology.orchestra.run.smithi175.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:25:13.114 INFO:teuthology.orchestra.run.smithi175.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:25:13.114 INFO:teuthology.orchestra.run.smithi175.stdout:- enp3s0f1 2020-11-16T22:25:13.115 INFO:teuthology.orchestra.run.smithi175.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:25:13.115 INFO:teuthology.orchestra.run.smithi175.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:25:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:13.242 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:25:13.265 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:25:13.448 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:25:13.454 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:13.458 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:13.677 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:25:13.740 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:25:13.866 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:25:14.193 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:25:14.245 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:14.405 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:14.668 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:14.668 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:15.058 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:25:15.259 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:25:15.944 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:25:16.035 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:25:16.404 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:25:16.773 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:25:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:25:17.231 DEBUG:teuthology.parallel:result is None 2020-11-16T22:25:17.770 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:25:17.807 INFO:teuthology.orchestra.run.smithi106.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:25:17.861 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:25:17.879 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:25:17.961 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:25:18.338 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:25:18.430 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:25:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:25:18.601 INFO:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:25:18.690 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:25:18.728 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2020-11-16T22:25:18.987 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2020-11-16T22:25:18.987 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2020-11-16T22:25:19.141 INFO:teuthology.orchestra.run.smithi175.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:25:19.143 DEBUG:teuthology.parallel:result is None 2020-11-16T22:25:19.159 INFO:teuthology.orchestra.run.smithi106.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:25:19.159 INFO:teuthology.orchestra.run.smithi106.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:25:19.160 INFO:teuthology.orchestra.run.smithi106.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:25:19.160 INFO:teuthology.orchestra.run.smithi106.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:25:19.160 INFO:teuthology.orchestra.run.smithi106.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:25:19.160 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2020-11-16T22:25:19.161 INFO:teuthology.orchestra.run.smithi106.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:25:19.161 INFO:teuthology.orchestra.run.smithi106.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:25:19.362 INFO:teuthology.orchestra.run.smithi106.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:25:19.362 INFO:teuthology.orchestra.run.smithi106.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:25:19.362 INFO:teuthology.orchestra.run.smithi106.stdout: zlib1g-dev 2020-11-16T22:25:19.363 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2020-11-16T22:25:19.364 INFO:teuthology.orchestra.run.smithi106.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:25:19.364 INFO:teuthology.orchestra.run.smithi106.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:25:19.364 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2020-11-16T22:25:19.364 INFO:teuthology.orchestra.run.smithi106.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:25:19.364 INFO:teuthology.orchestra.run.smithi106.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:25:19.365 INFO:teuthology.orchestra.run.smithi106.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:25:19.365 INFO:teuthology.orchestra.run.smithi106.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:25:19.365 INFO:teuthology.orchestra.run.smithi106.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:25:19.365 INFO:teuthology.orchestra.run.smithi106.stdout: zlib1g-dev 2020-11-16T22:25:19.408 INFO:teuthology.orchestra.run.smithi106.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:25:19.408 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:25:19.408 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:25:19.408 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:25:19.857 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:25:19.939 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:25:19.988 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:25:20.005 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:25:20.016 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:25:20.018 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:25:20.020 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:25:20.035 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:25:20.039 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:25:20.040 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:25:20.041 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:25:20.099 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:25:20.100 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:25:20.108 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:25:20.109 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:25:20.177 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:25:20.179 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:25:20.181 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:25:20.199 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:25:20.200 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:25:20.270 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:25:20.271 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:25:20.298 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:25:20.325 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:25:20.326 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:25:20.355 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:25:20.365 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:25:20.366 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:25:20.413 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:25:21.131 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 7,861 kB in 1s (6,722 kB/s) 2020-11-16T22:25:21.279 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package flex. 2020-11-16T22:25:21.352 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112494 files and directories currently installed.) 2020-11-16T22:25:21.554 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:25:21.555 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:25:21.888 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:25:21.905 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:25:21.947 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:22.349 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package bison. 2020-11-16T22:25:22.365 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:25:22.420 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:22.895 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package dump. 2020-11-16T22:25:22.911 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:25:22.953 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:25:23.297 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package indent. 2020-11-16T22:25:23.307 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:25:23.339 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:25:23.700 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:25:23.710 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:25:23.750 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:25:24.053 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:25:24.069 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:25:24.111 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:25:24.566 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:25:24.581 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:25:24.624 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:25:25.011 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:25:25.027 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:25:25.069 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:25:25.489 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:25:25.505 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:25:25.589 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:25:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:25:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:25:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:25:26.320 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:25:26.335 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:26.681 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:25:26.697 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:25:26.739 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:27.570 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:25:27.585 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:25:27.628 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:25:28.030 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:25:28.046 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:25:28.089 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:28.458 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:25:28.474 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:25:28.517 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:28.905 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:25:28.919 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:25:28.963 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:29.374 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:25:29.391 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:25:29.643 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:29.987 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:25:30.001 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:25:30.045 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:30.339 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:25:30.354 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:25:30.436 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:25:31.001 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:25:31.018 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:25:31.059 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:25:31.479 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:25:31.494 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:25:31.571 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:32.260 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:25:32.279 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:25:32.595 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:33.182 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:25:33.199 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:25:33.263 INFO:teuthology.orchestra.run.smithi106.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:25:33.263 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:25:33.477 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:33.979 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:25:33.995 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:25:34.094 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:35.146 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ppp. 2020-11-16T22:25:35.162 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:25:35.715 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:36.125 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:25:36.184 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:36.612 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:25:36.627 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:25:36.692 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:37.023 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:25:37.040 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:25:37.081 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:37.518 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:25:37.535 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:25:37.629 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:37.988 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:38.115 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:25:39.138 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:25:39.405 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:25:39.539 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:39.690 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:25:39.807 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:25:39.875 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:40.326 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:25:40.427 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:25:40.587 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:25:41.045 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:41.045 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:25:41.582 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:25:41.708 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:25:41.842 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:25:42.044 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:42.170 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:42.296 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:25:42.464 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:42.590 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:25:42.716 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:25:42.836 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:25:42.901 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:25:43.043 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:25:43.169 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:25:43.329 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:25:43.455 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:25:43.581 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:25:43.968 INFO:teuthology.orchestra.run.smithi106.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:25:44.513 INFO:teuthology.orchestra.run.smithi106.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:25:44.797 INFO:teuthology.orchestra.run.smithi106.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:25:44.843 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:44.994 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:25:45.120 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:25:45.246 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:25:45.389 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:25:46.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:46.329 INFO:teuthology.orchestra.run.smithi106.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:25:46.329 INFO:teuthology.orchestra.run.smithi106.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:25:46.329 INFO:teuthology.orchestra.run.smithi106.stdout:- enp3s0f1 2020-11-16T22:25:46.329 INFO:teuthology.orchestra.run.smithi106.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:25:46.329 INFO:teuthology.orchestra.run.smithi106.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:25:46.330 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:46.500 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:25:46.875 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:25:47.229 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:25:48.293 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:25:48.478 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:25:48.761 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:48.761 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:25:49.414 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:25:49.748 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:25:53.717 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:25:53.809 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:25:54.244 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:25:54.386 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:25:54.779 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:25:55.468 INFO:teuthology.orchestra.run.smithi106.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:25:55.474 DEBUG:teuthology.parallel:result is None 2020-11-16T22:25:55.475 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2020-11-16T22:25:55.529 INFO:teuthology.orchestra.run.smithi181.stdout:Ubuntu 2020-11-16T22:25:55.535 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:25:55.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:25:55.703 INFO:teuthology.orchestra.run.smithi181:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:25:55.729 INFO:teuthology.orchestra.run.smithi181.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:55.729 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:55.730 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:25:55.730 INFO:teuthology.orchestra.run.smithi175:> sudo lsb_release -is 2020-11-16T22:25:55.786 INFO:teuthology.orchestra.run.smithi175.stdout:Ubuntu 2020-11-16T22:25:55.792 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:25:55.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:25:55.954 INFO:teuthology.orchestra.run.smithi175:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:25:55.979 INFO:teuthology.orchestra.run.smithi175.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:55.980 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:55.980 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:25:55.980 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2020-11-16T22:25:56.058 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2020-11-16T22:25:56.066 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:25:56.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:25:56.231 INFO:teuthology.orchestra.run.smithi106:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:25:56.265 INFO:teuthology.orchestra.run.smithi106.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:56.265 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:25:56.266 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:25:56.266 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-16T22:25:56.267 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:56.267 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.272 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:24+1 records in 2020-11-16T22:25:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:24+1 records out 2020-11-16T22:25:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000226647 s, 55.9 MB/s 2020-11-16T22:25:56.282 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:56.282 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.287 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.296 INFO:teuthology.orchestra.run.smithi175.stderr:24+1 records in 2020-11-16T22:25:56.296 INFO:teuthology.orchestra.run.smithi175.stderr:24+1 records out 2020-11-16T22:25:56.297 INFO:teuthology.orchestra.run.smithi175.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000224047 s, 56.5 MB/s 2020-11-16T22:25:56.297 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:56.298 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:25:56.339 INFO:teuthology.orchestra.run.smithi106.stderr:24+1 records in 2020-11-16T22:25:56.339 INFO:teuthology.orchestra.run.smithi106.stderr:24+1 records out 2020-11-16T22:25:56.339 INFO:teuthology.orchestra.run.smithi106.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000775391 s, 16.3 MB/s 2020-11-16T22:25:56.341 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-16T22:25:56.342 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:56.342 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.347 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.356 INFO:teuthology.orchestra.run.smithi181.stderr:5+1 records in 2020-11-16T22:25:56.356 INFO:teuthology.orchestra.run.smithi181.stderr:5+1 records out 2020-11-16T22:25:56.356 INFO:teuthology.orchestra.run.smithi181.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.5647e-05 s, 32.0 MB/s 2020-11-16T22:25:56.357 INFO:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:25:56.411 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:56.412 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.417 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.425 INFO:teuthology.orchestra.run.smithi175.stderr:5+1 records in 2020-11-16T22:25:56.425 INFO:teuthology.orchestra.run.smithi175.stderr:5+1 records out 2020-11-16T22:25:56.426 INFO:teuthology.orchestra.run.smithi175.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.000102683 s, 29.8 MB/s 2020-11-16T22:25:56.426 INFO:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:25:56.483 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:56.484 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.498 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:25:56.511 INFO:teuthology.orchestra.run.smithi106.stderr:5+1 records in 2020-11-16T22:25:56.511 INFO:teuthology.orchestra.run.smithi106.stderr:5+1 records out 2020-11-16T22:25:56.511 INFO:teuthology.orchestra.run.smithi106.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.000287997 s, 10.6 MB/s 2020-11-16T22:25:56.513 INFO:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:25:56.583 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-16T22:25:56.583 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:56.584 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi181.stderr:299 bytes copied, 7.2331e-05 s, 4.1 MB/s 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:25:56.656 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:56.657 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.662 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.670 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:25:56.670 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:25:56.670 INFO:teuthology.orchestra.run.smithi175.stderr:299 bytes copied, 7.02e-05 s, 4.3 MB/s 2020-11-16T22:25:56.671 INFO:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:25:56.727 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:56.727 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.741 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:25:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:25:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:25:56.755 INFO:teuthology.orchestra.run.smithi106.stderr:299 bytes copied, 0.000194057 s, 1.5 MB/s 2020-11-16T22:25:56.757 INFO:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:25:56.781 INFO:teuthology.run_tasks:Running task ceph... 2020-11-16T22:25:56.841 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-16T22:25:56.842 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2020-11-16T22:25:56.848 INFO:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2020-11-16T22:25:56.851 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2020-11-16T22:25:56.871 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-16T22:25:56.872 INFO:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:25:56.877 INFO:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:25:56.907 INFO:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:25:56.921 INFO:tasks.ceph:Creating extra log directories... 2020-11-16T22:25:56.921 INFO:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:25:56.927 INFO:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:25:56.963 INFO:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:25:56.984 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-16T22:25:56.984 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-16T22:25:56.985 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-16T22:25:56.985 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:56.985 INFO:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.024 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:57.024 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:57.024 INFO:teuthology.orchestra.run.smithi181.stderr:408 bytes copied, 7.5613e-05 s, 5.4 MB/s 2020-11-16T22:25:57.024 INFO:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:25:57.101 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:57.102 INFO:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.106 INFO:teuthology.orchestra.run.smithi175.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.108 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:25:57.108 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:25:57.108 INFO:teuthology.orchestra.run.smithi175.stderr:408 bytes copied, 7.7273e-05 s, 5.3 MB/s 2020-11-16T22:25:57.108 INFO:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:25:57.180 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:57.181 INFO:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.195 INFO:teuthology.orchestra.run.smithi106.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:25:57.196 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:25:57.197 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:25:57.197 INFO:teuthology.orchestra.run.smithi106.stderr:408 bytes copied, 0.000116416 s, 3.5 MB/s 2020-11-16T22:25:57.197 INFO:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:25:57.278 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-16T22:25:57.278 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-16T22:25:57.278 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916', 'branch': 'wip-pdonnell-testing-20201116.174825', 'description': 'multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5631916', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20201116.174825'}, 'ceph': {'cephfs': {'max_mds': 9, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20201116.174825', 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'mds.d', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mgr.x', 'mds.e', 'mds.f', 'mds.g', 'mds.h', 'mds.i', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0', 'client.1']], 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604', 'sleep_before_teardown': 0, 'suite': 'multimds', 'suite_branch': 'wip-pdonnell-testing-20201116.174825', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '6959059e6aae2fb5241382a4eeeca62785867604', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (3, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, {'ceph': None}, {'kclient': None}, {'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_20:59:43', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.24832', 'targets': {'smithi181.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqruCyaTHoXvcJm3J4HRvwr51yZGpfwT7iQpOdzyr7CmAhcGqX1mfOEbBiSVxVFjwU6M9bfkREBEro9J2pDH4XaCsAzsq4wN4ZbVNy3is7FDfyTR6EDxzXd2ioTnNzkYok9bvbCg6U4t+5iQfJSuqM+jhZJ4iCB1FsbM6GWHfAe266oPF+tQOpwKcsZjxgtyhzTSQx+Rn1Uy5s/gji4TKAXPkVigaKwOAk1xH9VUo7jVMG5Hbdjqrs0SDYQnKfxhZG8wrOqp3a6Smy1EVJwTMbYiOfovK3s4fGPAj3MERC8i9Vv2KYucdmehyUqlFsXAuFfIZjjdUERE6kJXhm5xep', 'smithi175.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVOktKoMfu6uFtTEGcR66WDbENC3HFqNGgaHwkOsjpUUNHeDw4Dc58LnzQsfkIqFzXbJFzhs/GVsN4j2vPfR5K+0jfr1BoBjISJ0eyH7cKaGlx/z2Nm0LG+agurz8Db9rmcZkDC21wdDHxquGEq0k8JcpE+RQ0Rm+FSgtLgYTm8sWbbqtF1py7mJPxhg7Y4FWJfd76QosPpfe9yksBqqt4M8VoA9rhzTbWAQSRG27l6hhq5um3Z0EcbF7qL+Ojhw8A4SJpkBFegY2/H3hZpY7WZh0PJqRpEOjfJDDvqtnmUg1tQAI96gV5wpmB9XRT+25tTcgSgMj79DXLJfbAdnBX', 'smithi106.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC86IesroZTKWHbZ7ijHF7SSE6avm1jwyfyXzCcYdLVs7UIoPJnnAHZF1D0mCPr0yUxOhrmrgKZOmQDtA6Sy0zEcPukPNELacHXUlUCuL4OMhEuInKehZ+KwN3WWcQhFAvPfPdTLTep4vsk68NJwNn877kG/mq05EyCghjczKNUxCLhl41B38MTtwMH3Irg4LXGrQt3qFCgsBjQ8vr1fbybAv05W8NBQrlTDVNQKpqnHAjpw5uo+z+ionhCJJ7yYEXftceswHltbOGZQgwNIs64Rqg/L2m4Hk1aSyzsG5mqPNiTOegGoHmnJwdkhA28mihJlNxZbrppKUPA8Xl3efmj'}} 2020-11-16T22:25:57.279 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:25:57.285 INFO:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:25:57.287 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:25:57.293 INFO:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:25:57.339 INFO:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:25:57.341 INFO:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:25:57.364 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:57.364 INFO:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:25:57.402 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:25:57.403 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:25:57.403 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:25:57.404 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:25:57.404 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:25:57.404 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:57.404 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:57.404 INFO:teuthology.orchestra.run.smithi181.stderr:72 bytes copied, 0.00014886 s, 484 kB/s 2020-11-16T22:25:57.404 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:25:57.405 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:25:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:25:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.453 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-16T22:25:57.454 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.454 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2020-11-16 22:21:54.914194179 +0000 2020-11-16T22:25:57.454 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2020-11-16 22:21:54.794194182 +0000 2020-11-16T22:25:57.454 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2020-11-16 22:21:54.794194182 +0000 2020-11-16T22:25:57.454 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2020-11-16T22:25:57.462 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2020-11-16T22:25:57.463 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2020-11-16T22:25:57.463 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000384841 s, 1.3 MB/s 2020-11-16T22:25:57.467 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:25:57.517 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:25:57.517 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.517 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-11-16T22:25:57.517 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.517 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2020-11-16 22:21:55.154194174 +0000 2020-11-16T22:25:57.518 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2020-11-16 22:21:55.042194176 +0000 2020-11-16T22:25:57.518 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2020-11-16 22:21:55.042194176 +0000 2020-11-16T22:25:57.518 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2020-11-16T22:25:57.526 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2020-11-16T22:25:57.527 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2020-11-16T22:25:57.527 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000328059 s, 1.6 MB/s 2020-11-16T22:25:57.530 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:25:57.576 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:25:57.577 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.577 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-11-16T22:25:57.577 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.577 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2020-11-16 22:21:55.274194171 +0000 2020-11-16T22:25:57.577 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2020-11-16 22:21:55.270194171 +0000 2020-11-16T22:25:57.578 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2020-11-16 22:21:55.270194171 +0000 2020-11-16T22:25:57.578 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2020-11-16T22:25:57.586 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2020-11-16T22:25:57.586 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2020-11-16T22:25:57.586 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000381031 s, 1.3 MB/s 2020-11-16T22:25:57.590 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:25:57.637 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:25:57.637 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.637 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-11-16T22:25:57.638 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.638 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2020-11-16 22:21:55.490194166 +0000 2020-11-16T22:25:57.638 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2020-11-16 22:21:55.486194166 +0000 2020-11-16T22:25:57.638 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2020-11-16 22:21:55.486194166 +0000 2020-11-16T22:25:57.638 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2020-11-16T22:25:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2020-11-16T22:25:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2020-11-16T22:25:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.00034733 s, 1.5 MB/s 2020-11-16T22:25:57.650 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:25:57.655 INFO:teuthology.orchestra.run.smithi175.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:25:57.656 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:25:57.656 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:25:57.656 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:25:57.657 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:25:57.657 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:25:57.657 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:25:57.657 INFO:teuthology.orchestra.run.smithi175.stderr:72 bytes copied, 0.00017891 s, 402 kB/s 2020-11-16T22:25:57.657 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:25:57.658 INFO:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:25:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2020-11-16 22:21:54.900990704 +0000 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2020-11-16 22:21:54.792990706 +0000 2020-11-16T22:25:57.704 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2020-11-16 22:21:54.792990706 +0000 2020-11-16T22:25:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2020-11-16T22:25:57.713 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2020-11-16T22:25:57.713 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2020-11-16T22:25:57.713 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000333143 s, 1.5 MB/s 2020-11-16T22:25:57.718 INFO:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:25:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2020-11-16 22:21:55.132990698 +0000 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2020-11-16 22:21:55.032990701 +0000 2020-11-16T22:25:57.768 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2020-11-16 22:21:55.032990701 +0000 2020-11-16T22:25:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2020-11-16T22:25:57.777 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2020-11-16T22:25:57.777 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2020-11-16T22:25:57.778 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000313922 s, 1.6 MB/s 2020-11-16T22:25:57.781 INFO:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:25:57.827 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:25:57.828 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.828 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-11-16T22:25:57.828 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.828 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2020-11-16 22:21:55.364990693 +0000 2020-11-16T22:25:57.828 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2020-11-16 22:21:55.264990695 +0000 2020-11-16T22:25:57.829 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2020-11-16 22:21:55.264990695 +0000 2020-11-16T22:25:57.829 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2020-11-16T22:25:57.837 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2020-11-16T22:25:57.837 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2020-11-16T22:25:57.837 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000376363 s, 1.4 MB/s 2020-11-16T22:25:57.841 INFO:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:25:57.888 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:25:57.888 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:25:57.888 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-11-16T22:25:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:25:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2020-11-16 22:21:55.608990688 +0000 2020-11-16T22:25:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2020-11-16 22:21:55.512990690 +0000 2020-11-16T22:25:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2020-11-16 22:21:55.512990690 +0000 2020-11-16T22:25:57.889 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2020-11-16T22:25:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2020-11-16T22:25:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2020-11-16T22:25:57.897 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000408097 s, 1.3 MB/s 2020-11-16T22:25:57.901 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:25:57.902 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T22:25:57.902 INFO:tasks.ceph:Generating config... 2020-11-16T22:25:57.903 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-16T22:25:57.904 INFO:tasks.ceph:[client] debug client = 20 2020-11-16T22:25:57.904 INFO:tasks.ceph:[client] debug ms = 1 2020-11-16T22:25:57.904 INFO:tasks.ceph:[client] fuse default permissions = False 2020-11-16T22:25:57.904 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-16T22:25:57.904 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-16T22:25:57.905 INFO:tasks.ceph:[global] ms die on skipped message = False 2020-11-16T22:25:57.905 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-16T22:25:57.905 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-16T22:25:57.905 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-16T22:25:57.905 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-16T22:25:57.906 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-16T22:25:57.906 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-16T22:25:57.906 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-16T22:25:57.906 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-16T22:25:57.907 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-16T22:25:57.907 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-16T22:25:57.907 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-16T22:25:57.907 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-16T22:25:57.907 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-16T22:25:57.908 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-16T22:25:57.908 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-16T22:25:57.908 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-16T22:25:57.908 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-16T22:25:57.908 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-16T22:25:57.909 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-16T22:25:57.909 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-16T22:25:57.909 INFO:tasks.ceph:[osd] bdev async discard = True 2020-11-16T22:25:57.909 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-11-16T22:25:57.909 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2020-11-16T22:25:57.910 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-16T22:25:57.910 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-16T22:25:57.910 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-16T22:25:57.910 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-16T22:25:57.910 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-16T22:25:57.911 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-16T22:25:57.911 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-16T22:25:57.911 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-16T22:25:57.911 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-16T22:25:57.912 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-16T22:25:57.912 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-16T22:25:57.912 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-16T22:25:57.912 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-16T22:25:57.912 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-16T22:25:57.913 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-16T22:25:57.913 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2020-11-16T22:25:57.913 INFO:tasks.ceph:Setting up mon.a... 2020-11-16T22:25:57.913 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-16T22:25:57.945 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.keyring 2020-11-16T22:25:57.947 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-16T22:25:58.023 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:58.080 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.181'), ('mon.c', '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]'), ('mon.b', '172.21.15.175')] 2020-11-16T22:25:58.080 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790],172.21.15.175', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds.d': {}, 'mds.e': {}, 'mds.f': {}, 'mds.g': {}, 'mds.h': {}, 'mds.i': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}} 2020-11-16T22:25:58.081 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:58.082 INFO:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:25:58.126 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:25:58.128 INFO:teuthology.orchestra.run.smithi181.stderr:7+1 records in 2020-11-16T22:25:58.128 INFO:teuthology.orchestra.run.smithi181.stderr:7+1 records out 2020-11-16T22:25:58.128 INFO:teuthology.orchestra.run.smithi181.stderr:3662 bytes (3.7 kB, 3.6 KiB) copied, 0.00011697 s, 31.3 MB/s 2020-11-16T22:25:58.128 INFO:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.181 --addv c '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]' --add b 172.21.15.175 --print /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:58.206 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: generated fsid 0da1f45c-24e6-440b-b459-36ddfb20a11a 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 0 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:fsid 0da1f45c-24e6-440b-b459-36ddfb20a11a 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:last_changed 2020-11-16T22:25:58.225489+0000 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:created 2020-11-16T22:25:58.225489+0000 2020-11-16T22:25:58.207 INFO:teuthology.orchestra.run.smithi181.stdout:min_mon_release 0 (unknown) 2020-11-16T22:25:58.208 INFO:teuthology.orchestra.run.smithi181.stdout:election_strategy: 1 2020-11-16T22:25:58.208 INFO:teuthology.orchestra.run.smithi181.stdout:0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2020-11-16T22:25:58.208 INFO:teuthology.orchestra.run.smithi181.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2020-11-16T22:25:58.208 INFO:teuthology.orchestra.run.smithi181.stdout:2: [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] mon.c 2020-11-16T22:25:58.208 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-16T22:25:58.209 INFO:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:25:58.256 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0da1f45c-24e6-440b-b459-36ddfb20a11a... 2020-11-16T22:25:58.257 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:25:58.263 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:25:58.265 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:25:58.301 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2020-11-16T22:25:58.301 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2020-11-16T22:25:58.301 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:25:58.301 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2020-11-16T22:25:58.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2020-11-16T22:25:58.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.303 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2020-11-16T22:25:58.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:25:58.305 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.306 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790],172.21.15.175" 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on skipped message = False 2020-11-16T22:25:58.307 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 0da1f45c-24e6-440b-b459-36ddfb20a11a 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2020-11-16T22:25:58.308 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2020-11-16T22:25:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.310 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.311 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2020-11-16T22:25:58.312 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 20 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 20 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug filestore = 20 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug journal = 20 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 20 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 25 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 10 2020-11-16T22:25:58.313 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: osd shutdown pgref assert = True 2020-11-16T22:25:58.314 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:58.315 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2020-11-16T22:25:58.316 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:25:58.318 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: fuse default permissions = False 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 15m 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 15m 2020-11-16T22:25:58.319 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2020-11-16T22:25:58.320 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.g] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.h] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.i] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2020-11-16T22:25:58.321 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal frag = True 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal fragment size max = 10000 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal merge size = 5 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal split bits = 3 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal split size = 100 2020-11-16T22:25:58.322 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 15m 2020-11-16T22:25:58.323 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 15m 2020-11-16T22:25:58.331 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2020-11-16T22:25:58.331 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2020-11-16T22:25:58.331 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:25:58.331 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2020-11-16T22:25:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2020-11-16T22:25:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2020-11-16T22:25:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2020-11-16T22:25:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790],172.21.15.175" 2020-11-16T22:25:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on skipped message = False 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 0da1f45c-24e6-440b-b459-36ddfb20a11a 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2020-11-16T22:25:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2020-11-16T22:25:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2020-11-16T22:25:58.340 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.340 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2020-11-16T22:25:58.340 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2020-11-16T22:25:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:25:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2020-11-16T22:25:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2020-11-16T22:25:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 20 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 20 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug filestore = 20 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug journal = 20 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 20 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 25 2020-11-16T22:25:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 10 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: osd shutdown pgref assert = True 2020-11-16T22:25:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2020-11-16T22:25:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.548 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2020-11-16T22:25:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2020-11-16T22:25:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2020-11-16T22:25:58.750 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2020-11-16T22:25:58.750 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2020-11-16T22:25:58.750 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2020-11-16T22:25:58.751 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:25:58.752 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout: fuse default permissions = False 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 15m 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 15m 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2020-11-16T22:25:58.753 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.e] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.f] 2020-11-16T22:25:58.754 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.g] 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.h] 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.i] 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout: mds bal frag = True 2020-11-16T22:25:58.755 INFO:teuthology.orchestra.run.smithi181.stdout: mds bal fragment size max = 10000 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds bal merge size = 5 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds bal split bits = 3 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds bal split size = 100 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2020-11-16T22:25:58.756 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2020-11-16T22:25:58.757 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2020-11-16T22:25:58.757 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 15m 2020-11-16T22:25:58.757 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 15m 2020-11-16T22:25:58.758 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2020-11-16T22:25:58.758 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2020-11-16T22:25:58.759 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.760 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2020-11-16T22:25:58.761 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:25:58.762 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.763 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2020-11-16T22:25:58.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.966 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2020-11-16T22:25:58.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790],172.21.15.175" 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = 0da1f45c-24e6-440b-b459-36ddfb20a11a 2020-11-16T22:25:58.967 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.968 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.969 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.970 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: bdev async discard = True 2020-11-16T22:25:58.971 INFO:teuthology.orchestra.run.smithi106.stdout: bdev enable discard = True 2020-11-16T22:25:58.972 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore allocator = bitmap 2020-11-16T22:25:58.972 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2020-11-16T22:25:58.972 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2020-11-16T22:25:58.972 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2020-11-16T22:25:59.178 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2020-11-16T22:25:59.178 INFO:teuthology.orchestra.run.smithi106.stdout: debug filestore = 20 2020-11-16T22:25:59.178 INFO:teuthology.orchestra.run.smithi106.stdout: debug journal = 20 2020-11-16T22:25:59.178 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 20 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 25 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:25:59.179 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: osd shutdown pgref assert = True 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2020-11-16T22:25:59.180 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2020-11-16T22:25:59.181 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:25:59.182 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2020-11-16T22:25:59.183 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2020-11-16T22:25:59.184 INFO:teuthology.orchestra.run.smithi106.stdout: fuse default permissions = False 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 15m 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 15m 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2020-11-16T22:25:59.185 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.e] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.f] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.g] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.h] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.i] 2020-11-16T22:25:59.186 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal frag = True 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal fragment size max = 10000 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal merge size = 5 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal split bits = 3 2020-11-16T22:25:59.187 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal split size = 100 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 15m 2020-11-16T22:25:59.188 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 15m 2020-11-16T22:25:59.189 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-16T22:25:59.189 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-16T22:25:59.235 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-16T22:25:59.236 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:59.236 INFO:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:25:59.240 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:25:59.240 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.] 2020-11-16T22:25:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB2/LJfHARfAhAA9iaZJXlBqtlBx5tIXPBoVg== 2020-11-16T22:25:59.241 INFO:teuthology.orchestra.run.smithi181.stdout:[client.admin] 2020-11-16T22:25:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg== 2020-11-16T22:25:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow *" 2020-11-16T22:25:59.442 INFO:teuthology.orchestra.run.smithi181.stdout: caps mgr = "allow *" 2020-11-16T22:25:59.442 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow *" 2020-11-16T22:25:59.442 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow *" 2020-11-16T22:25:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:206 bytes copied, 7.2978e-05 s, 2.8 MB/s 2020-11-16T22:25:59.443 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:59.444 INFO:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:25:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:25:59.450 INFO:teuthology.orchestra.run.smithi181.stdout: � ��\$�D �Y6�� �v��_F�p v��_F�p �aXa �����bXb �����cXc �����abc 2020-11-16T22:25:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:409 bytes copied, 0.000164117 s, 2.5 MB/s 2020-11-16T22:25:59.450 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:25:59.451 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:59.451 INFO:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.451 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.494 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:59.504 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:59.504 INFO:teuthology.orchestra.run.smithi181.stderr:206 bytes copied, 0.000124987 s, 1.6 MB/s 2020-11-16T22:25:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.512 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:25:59.513 INFO:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi181.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:409 bytes copied, 0.000130944 s, 3.1 MB/s 2020-11-16T22:25:59.560 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi175.stderr:206 bytes copied, 3.3628e-05 s, 6.1 MB/s 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.629 INFO:teuthology.orchestra.run.smithi175.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.632 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:25:59.632 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:25:59.633 INFO:teuthology.orchestra.run.smithi175.stderr:409 bytes copied, 7.703e-05 s, 5.3 MB/s 2020-11-16T22:25:59.633 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:25:59.633 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:59.633 INFO:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.634 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.648 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:25:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:25:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:25:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:206 bytes copied, 0.00024591 s, 838 kB/s 2020-11-16T22:25:59.672 INFO:teuthology.orchestra.run.smithi106.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:25:59.693 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:25:59.693 INFO:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.702 INFO:teuthology.orchestra.run.smithi106.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:25:59.704 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:25:59.705 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:25:59.705 INFO:teuthology.orchestra.run.smithi106.stderr:409 bytes copied, 0.000130657 s, 3.1 MB/s 2020-11-16T22:25:59.705 INFO:tasks.ceph:Setting up mon nodes... 2020-11-16T22:25:59.706 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-16T22:25:59.706 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:25:59.746 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:25:59.748 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:25:59.791 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:25:59.793 INFO:tasks.ceph:Setting up mds nodes... 2020-11-16T22:25:59.794 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:25:59.835 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:25:59.837 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-16T22:25:59.892 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-16T22:25:59.974 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-16T22:25:59.976 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-16T22:26:00.032 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2020-11-16T22:26:00.116 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2020-11-16T22:26:00.118 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2020-11-16T22:26:00.173 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2020-11-16T22:26:00.255 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2020-11-16T22:26:00.257 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2020-11-16T22:26:00.311 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2020-11-16T22:26:00.352 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2020-11-16T22:26:00.354 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2020-11-16T22:26:00.412 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2020-11-16T22:26:00.500 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2020-11-16T22:26:00.502 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2020-11-16T22:26:00.560 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-g && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.g /var/lib/ceph/mds/ceph-g/keyring 2020-11-16T22:26:00.639 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-g/keyring 2020-11-16T22:26:00.642 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-g 2020-11-16T22:26:00.700 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-h && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.h /var/lib/ceph/mds/ceph-h/keyring 2020-11-16T22:26:00.783 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-h/keyring 2020-11-16T22:26:00.785 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-h 2020-11-16T22:26:00.846 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-i && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.i /var/lib/ceph/mds/ceph-i/keyring 2020-11-16T22:26:00.929 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-i/keyring 2020-11-16T22:26:00.931 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-i 2020-11-16T22:26:00.988 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-16T22:26:00.988 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-16T22:26:01.053 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-16T22:26:01.078 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-16T22:26:01.136 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-16T22:26:01.164 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-16T22:26:01.164 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T22:26:01.165 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-16T22:26:01.180 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:01.180 INFO:tasks.ceph:role: osd.0 2020-11-16T22:26:01.180 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:26:01.180 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:26:01.708 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:01.709 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:01.710 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:01.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2020-11-16T22:26:01.711 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-16T22:26:01.772 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-16T22:26:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:01.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:01.784 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-16T22:26:01.840 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:01.840 INFO:tasks.ceph:role: osd.1 2020-11-16T22:26:01.840 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:26:01.840 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T22:26:02.359 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:02.359 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:02.359 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:02.359 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:02.360 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:02.360 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:02.360 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:02.360 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:02.360 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:02.361 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2020-11-16T22:26:02.361 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-16T22:26:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:02.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:02.429 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-16T22:26:02.480 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:02.480 INFO:tasks.ceph:role: osd.2 2020-11-16T22:26:02.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:26:02.480 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T22:26:02.992 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:02.993 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:02.993 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:02.993 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:02.993 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:02.993 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:02.994 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:02.994 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:02.994 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:02.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2020-11-16T22:26:02.995 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-16T22:26:03.047 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-16T22:26:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:03.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:03.058 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-16T22:26:03.112 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:03.113 INFO:tasks.ceph:role: osd.3 2020-11-16T22:26:03.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:26:03.113 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T22:26:03.624 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:03.624 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:03.625 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:03.626 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:03.626 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2020-11-16T22:26:03.627 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-16T22:26:03.657 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-16T22:26:03.709 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:03.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:03.710 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.987+0000 7fce5a837d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-16T22:26:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.991+0000 7fce5a837d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-16T22:26:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.991+0000 7fce5a837d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:26:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.991+0000 7fce5a837d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:26:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.991+0000 7fce5a837d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:26:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:03.995+0000 7fce5a837d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-16T22:26:06.917 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-16T22:26:06.932 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.018+0000 7f85f2d17d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-16T22:26:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.018+0000 7f85f2d17d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-16T22:26:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.018+0000 7f85f2d17d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:26:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.018+0000 7f85f2d17d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:26:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.018+0000 7f85f2d17d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:26:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:07.022+0000 7f85f2d17d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-16T22:26:09.954 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-16T22:26:10.012 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:10.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-16T22:26:10.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-16T22:26:10.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:26:10.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:26:10.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:26:10.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:10.105+0000 7fa031763d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-16T22:26:13.034 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-16T22:26:13.091 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:13.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-16T22:26:13.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-16T22:26:13.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:26:13.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:26:13.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:26:13.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:13.185+0000 7fde5a405d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-16T22:26:16.133 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-16T22:26:16.188 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-16T22:26:16.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:16.204 INFO:tasks.ceph:role: osd.4 2020-11-16T22:26:16.204 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:26:16.204 INFO:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:26:16.733 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:16.733 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:16.734 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:16.735 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:16.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2020-11-16T22:26:16.736 INFO:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-16T22:26:16.808 INFO:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-16T22:26:16.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:16.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:16.820 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-16T22:26:16.875 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:16.875 INFO:tasks.ceph:role: osd.5 2020-11-16T22:26:16.876 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T22:26:17.391 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:17.391 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:17.391 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:17.391 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:17.391 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:17.392 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:17.392 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:17.392 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:17.392 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:17.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2020-11-16T22:26:17.393 INFO:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-16T22:26:17.465 INFO:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-16T22:26:17.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:17.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:17.478 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-16T22:26:17.536 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:17.536 INFO:tasks.ceph:role: osd.6 2020-11-16T22:26:17.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:26:17.536 INFO:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T22:26:18.030 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:18.030 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:18.031 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:18.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2020-11-16T22:26:18.033 INFO:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-16T22:26:18.079 INFO:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-16T22:26:18.089 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:18.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:18.090 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-16T22:26:18.149 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:26:18.149 INFO:tasks.ceph:role: osd.7 2020-11-16T22:26:18.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:26:18.150 INFO:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T22:26:18.691 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:26:18.692 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:26:18.693 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:26:18.693 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:26:18.694 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2020-11-16T22:26:18.694 INFO:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-16T22:26:18.731 INFO:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-16T22:26:18.786 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:26:18.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:26:18.787 INFO:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:18.882 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.896+0000 7fd4c0c74d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-16T22:26:18.885 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.900+0000 7fd4c0c74d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-16T22:26:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.900+0000 7fd4c0c74d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:26:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.900+0000 7fd4c0c74d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:26:18.886 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.900+0000 7fd4c0c74d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:26:18.890 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:18.904+0000 7fd4c0c74d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:21.933 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-16T22:26:21.933 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-16T22:26:21.934 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:26:21.934 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:26:21.934 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:26:21.934 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:21.948+0000 7ff25d74fd80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-16T22:26:24.908 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-16T22:26:24.966 INFO:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:25.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.060+0000 7fa7a61e2d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-16T22:26:25.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.064+0000 7fa7a61e2d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-16T22:26:25.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.064+0000 7fa7a61e2d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:26:25.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.064+0000 7fa7a61e2d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:26:25.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.064+0000 7fa7a61e2d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:26:25.049 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:25.064+0000 7fa7a61e2d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-16T22:26:27.281 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:27.288 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:27.290 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:28.007 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-16T22:26:28.023 INFO:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:28.104 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-16T22:26:28.105 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-16T22:26:28.105 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:26:28.105 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:26:28.105 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:26:28.106 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:28.120+0000 7f89b2d7ed80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-16T22:26:31.050 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-16T22:26:31.106 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-16T22:26:31.106 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.107 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:26:31.112 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:26:31.120 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2020-11-16T22:26:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g== 2020-11-16T22:26:31.121 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.121 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.121 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000168574 s, 332 kB/s 2020-11-16T22:26:31.122 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.122 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:26:31.170 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:26:31.178 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2020-11-16T22:26:31.179 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA== 2020-11-16T22:26:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000182253 s, 307 kB/s 2020-11-16T22:26:31.180 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.180 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-16T22:26:31.225 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-16T22:26:31.233 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2020-11-16T22:26:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg== 2020-11-16T22:26:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000171437 s, 327 kB/s 2020-11-16T22:26:31.234 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.235 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-16T22:26:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-16T22:26:31.290 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2020-11-16T22:26:31.290 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q== 2020-11-16T22:26:31.291 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.291 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.291 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.00017444 s, 321 kB/s 2020-11-16T22:26:31.292 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.292 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2020-11-16T22:26:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2020-11-16T22:26:31.345 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2020-11-16T22:26:31.346 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA== 2020-11-16T22:26:31.346 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.346 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.346 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000162154 s, 345 kB/s 2020-11-16T22:26:31.347 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.347 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:26:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:26:31.402 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2020-11-16T22:26:31.403 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q== 2020-11-16T22:26:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000165487 s, 338 kB/s 2020-11-16T22:26:31.404 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.404 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:26:31.449 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:26:31.458 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2020-11-16T22:26:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng== 2020-11-16T22:26:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000170818 s, 328 kB/s 2020-11-16T22:26:31.460 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.460 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:26:31.505 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:26:31.514 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2020-11-16T22:26:31.514 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg== 2020-11-16T22:26:31.514 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.515 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.515 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000204393 s, 274 kB/s 2020-11-16T22:26:31.515 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:31.516 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T22:26:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T22:26:31.570 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2020-11-16T22:26:31.571 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w== 2020-11-16T22:26:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:56 bytes copied, 0.000158007 s, 354 kB/s 2020-11-16T22:26:31.572 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.572 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:26:31.577 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:26:31.586 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2020-11-16T22:26:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q== 2020-11-16T22:26:31.586 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.586 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.586 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.0001778 s, 315 kB/s 2020-11-16T22:26:31.587 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.587 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2020-11-16T22:26:31.634 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2020-11-16T22:26:31.643 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e] 2020-11-16T22:26:31.643 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA== 2020-11-16T22:26:31.643 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.643 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.643 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000174436 s, 321 kB/s 2020-11-16T22:26:31.644 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.644 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2020-11-16T22:26:31.690 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2020-11-16T22:26:31.699 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f] 2020-11-16T22:26:31.699 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ== 2020-11-16T22:26:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000168957 s, 331 kB/s 2020-11-16T22:26:31.701 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.701 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-g/keyring of=/dev/stdout 2020-11-16T22:26:31.747 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-g/keyring of=/dev/stdout 2020-11-16T22:26:31.755 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.g] 2020-11-16T22:26:31.755 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg== 2020-11-16T22:26:31.756 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.756 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.756 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.00016725 s, 335 kB/s 2020-11-16T22:26:31.757 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.757 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-h/keyring of=/dev/stdout 2020-11-16T22:26:31.803 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-h/keyring of=/dev/stdout 2020-11-16T22:26:31.811 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.h] 2020-11-16T22:26:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw== 2020-11-16T22:26:31.812 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.812 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.812 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000162737 s, 344 kB/s 2020-11-16T22:26:31.813 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.813 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-i/keyring of=/dev/stdout 2020-11-16T22:26:31.858 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-i/keyring of=/dev/stdout 2020-11-16T22:26:31.867 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.i] 2020-11-16T22:26:31.867 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg== 2020-11-16T22:26:31.868 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.868 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.868 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000160844 s, 348 kB/s 2020-11-16T22:26:31.869 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.869 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T22:26:31.915 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T22:26:31.923 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2020-11-16T22:26:31.923 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g== 2020-11-16T22:26:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.923 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.924 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.00017345 s, 323 kB/s 2020-11-16T22:26:31.924 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.925 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T22:26:31.971 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T22:26:31.977 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2020-11-16T22:26:31.977 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw== 2020-11-16T22:26:31.978 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:31.978 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:31.978 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 6.8528e-05 s, 817 kB/s 2020-11-16T22:26:31.979 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:31.979 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q== 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000119396 s, 469 kB/s 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw== 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi175.stderr:56 bytes copied, 0.000158204 s, 354 kB/s 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi106.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:26:32.111 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2020-11-16T22:26:32.111 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA== 2020-11-16T22:26:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:26:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:26:32.112 INFO:teuthology.orchestra.run.smithi106.stderr:59 bytes copied, 0.000417443 s, 141 kB/s 2020-11-16T22:26:32.113 INFO:teuthology.orchestra.run.smithi106:> set -ex 2020-11-16T22:26:32.113 INFO:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T22:26:32.169 INFO:teuthology.orchestra.run.smithi106.stderr:+ dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T22:26:32.170 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2020-11-16T22:26:32.170 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw== 2020-11-16T22:26:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2020-11-16T22:26:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2020-11-16T22:26:32.171 INFO:teuthology.orchestra.run.smithi106.stderr:59 bytes copied, 0.000213618 s, 276 kB/s 2020-11-16T22:26:32.171 INFO:tasks.ceph:Adding keys to all mons... 2020-11-16T22:26:32.172 INFO:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T22:26:32.174 INFO:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T22:26:32.185 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2020-11-16T22:26:32.185 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g== 2020-11-16T22:26:32.185 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2020-11-16T22:26:32.185 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA== 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg== 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q== 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA== 2020-11-16T22:26:32.186 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q== 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng== 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg== 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2020-11-16T22:26:32.187 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w== 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q== 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e] 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA== 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f] 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ== 2020-11-16T22:26:32.188 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.g] 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg== 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.h] 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw== 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.i] 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg== 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2020-11-16T22:26:32.189 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g== 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw== 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q== 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw== 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2020-11-16T22:26:32.190 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA== 2020-11-16T22:26:32.191 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2020-11-16T22:26:32.191 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw== 2020-11-16T22:26:32.192 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2020-11-16T22:26:32.192 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g== 2020-11-16T22:26:32.192 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2020-11-16T22:26:32.192 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA== 2020-11-16T22:26:32.192 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg== 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q== 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA== 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2020-11-16T22:26:32.193 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q== 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng== 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg== 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w== 2020-11-16T22:26:32.194 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q== 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.e] 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA== 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.f] 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ== 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.g] 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg== 2020-11-16T22:26:32.195 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.h] 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw== 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.i] 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg== 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g== 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2020-11-16T22:26:32.196 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw== 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q== 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw== 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA== 2020-11-16T22:26:32.197 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2020-11-16T22:26:32.198 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw== 2020-11-16T22:26:32.198 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:26:32.236 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:26:32.279 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.366 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.404 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.453 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.455 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.538 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.540 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:32.625 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.627 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.707 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.712 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.791 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.796 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.875 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.880 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:32.959 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:26:32.964 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.214 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.g --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.216 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.g --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.301 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.h --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.304 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.h --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.391 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.i --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.393 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.i --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:26:33.435 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.480 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.522 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.564 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.606 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.648 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:26:33.950 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-16T22:26:33.950 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-16T22:26:33.959 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:26:34.072 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-16T22:26:34.087 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-16T22:26:34.144 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:26:34.241 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-16T22:26:34.256 INFO:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-16T22:26:34.271 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:26:34.386 INFO:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-16T22:26:34.399 INFO:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:34.448 INFO:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:26:34.454 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-16T22:26:34.454 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-16T22:26:34.455 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-16T22:26:34.499 INFO:tasks.ceph.mon.a:Started 2020-11-16T22:26:34.499 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-16T22:26:34.500 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-16T22:26:34.502 INFO:tasks.ceph.mon.c:Started 2020-11-16T22:26:34.502 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-16T22:26:34.502 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-16T22:26:34.504 INFO:tasks.ceph.mon.b:Started 2020-11-16T22:26:34.504 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-16T22:26:34.505 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-16T22:26:34.505 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-16T22:26:34.507 INFO:tasks.ceph.mgr.y:Started 2020-11-16T22:26:34.507 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-16T22:26:34.508 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-16T22:26:34.510 INFO:tasks.ceph.mgr.x:Started 2020-11-16T22:26:34.510 INFO:tasks.ceph:Setting crush tunables to default 2020-11-16T22:26:34.510 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd crush tunables default 2020-11-16T22:26:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.737+0000 7f394d46a700 1 Processor -- start 2020-11-16T22:26:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.737+0000 7f394d46a700 1 -- start start 2020-11-16T22:26:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f394d46a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f394d46a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f394d46a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3948061900 con 0x7f3948081890 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f394d46a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3948061bd0 con 0x7f3948163a50 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f394d46a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3948061e70 con 0x7f3948062af0 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55858/0 (socket says 172.21.15.181:55858) 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35470/0 (socket says 172.21.15.181:35470) 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39477fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3948081890 0x7f39480d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41932/0 (socket says 172.21.15.181:41932) 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 learned_addr learned my addr 172.21.15.181:0/1209815420 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 msgr2=0x7f3948163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-16T22:26:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 msgr2=0x7f3948163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 msgr2=0x7f39480813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 msgr2=0x7f39480813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-16T22:26:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.741+0000 7f39467fc700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-16T22:26:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258915621 0 0) 0x7f3948061900 con 0x7f3948081890 2020-11-16T22:26:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3928003240 con 0x7f3948081890 2020-11-16T22:26:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 550739297 0 0) 0x7f3928003240 con 0x7f3948081890 2020-11-16T22:26:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 msgr2=0x7f39480813a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-16T22:26:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39480813a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 msgr2=0x7f3948163e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 --2- 172.21.15.181:0/1209815420 >> v2:172.21.15.181:3300/0 conn(0x7f3948163a50 0x7f3948163e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948062110 con 0x7f3948081890 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f39380014e0 con 0x7f3948081890 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 >> v1:172.21.15.181:6789/0 conn(0x7f3948081890 legacy=0x7f39480d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3928003730 0x7f3928003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39280044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39280049d0 0x7f3928004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948061bd0 con 0x7f39280049d0 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948061e70 con 0x7f3928003730 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39280041d0 con 0x7f3948062af0 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3928003730 0x7f3928003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f39467fc700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39280044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f39477fe700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39280049d0 0x7f3928004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 msgr2=0x7f39280044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3948062af0 0x7f39280044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39280049d0 msgr2=0x7f3928004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39280049d0 0x7f3928004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 -- 172.21.15.181:0/1209815420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928003240 con 0x7f3928003730 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f39477fe700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39280049d0 0x7f3928004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3928003730 0x7f3928003b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3940009df0 tx=0x7f3940008040).ready entity=mon.1 client_cookie=2e6a9d88f96f5783 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.869+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000ea80 con 0x7f3928003730 2020-11-16T22:26:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000ebe0 con 0x7f3928003730 2020-11-16T22:26:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f3945ffb700 1 -- 172.21.15.181:0/1209815420 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940005a50 con 0x7f3928003730 2020-11-16T22:26:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 -- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3928003730 msgr2=0x7f3928003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 --2- 172.21.15.181:0/1209815420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3928003730 0x7f3928003b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3940009df0 tx=0x7f3940008040).stop 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f39477fe700 1 -- 172.21.15.181:0/1209815420 reap_dead start 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 -- 172.21.15.181:0/1209815420 shutdown_connections 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 -- 172.21.15.181:0/1209815420 >> 172.21.15.181:0/1209815420 conn(0x7f39480d69a0 msgr2=0x7f39480d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 -- 172.21.15.181:0/1209815420 shutdown_connections 2020-11-16T22:26:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.873+0000 7f394d46a700 1 -- 172.21.15.181:0/1209815420 wait complete. 2020-11-16T22:26:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 Processor -- start 2020-11-16T22:26:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 -- start start 2020-11-16T22:26:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39480638e0 0x7f39480b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39480b8b20 0x7f3948176b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948177800 con 0x7f39480638e0 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948160f40 con 0x7f39480b8b20 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f394d46a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39481611e0 con 0x7f3928003730 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f39477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39480b8b20 0x7f3948176b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35492/0 (socket says 172.21.15.181:35492) 2020-11-16T22:26:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 -- 172.21.15.181:0/1486031877 learned_addr learned my addr 172.21.15.181:0/1486031877 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f39467fc700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39480638e0 0x7f39480b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 -- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39480b8b20 msgr2=0x7f3948176b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39480b8b20 0x7f3948176b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 -- 172.21.15.181:0/1486031877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39480638e0 msgr2=0x7f39480b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39480638e0 0x7f39480b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 -- 172.21.15.181:0/1486031877 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948161480 con 0x7f3928003730 2020-11-16T22:26:35.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.877+0000 7f3946ffd700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f394000ebb0 tx=0x7f3940008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:35.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940006490 con 0x7f3928003730 2020-11-16T22:26:35.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39400065f0 con 0x7f3928003730 2020-11-16T22:26:35.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3948161720 con 0x7f3928003730 2020-11-16T22:26:35.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3948162ad0 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940006970 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f394001c040 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3940029760 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.881+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39480b8f00 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.885+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f394002e060 con 0x7f3928003730 2020-11-16T22:26:35.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.961+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3948069eb0 con 0x7f3928003730 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f393effd700 1 -- 172.21.15.181:0/1486031877 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f394000c070 con 0x7f3928003730 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 msgr2=0x7f39480633f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f394000ebb0 tx=0x7f3940008040).stop 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 shutdown_connections 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39480b8b20 0x7f3948176b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3928003730 0x7f39480633f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 --2- 172.21.15.181:0/1486031877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39480638e0 0x7f39480b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 >> 172.21.15.181:0/1486031877 conn(0x7f39480d69a0 msgr2=0x7f39480d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:35.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 shutdown_connections 2020-11-16T22:26:35.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:34.969+0000 7f394d46a700 1 -- 172.21.15.181:0/1486031877 wait complete. 2020-11-16T22:26:35.080 INFO:teuthology.orchestra.run.smithi181.stderr:adjusted tunables profile to default 2020-11-16T22:26:35.080 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-16T22:26:35.081 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:35.081 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:26:35.085 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:26:35.092 INFO:teuthology.orchestra.run.smithi181.stdout:ddae4f96-c4ce-4f57-920e-c80b7e795739 2020-11-16T22:26:35.092 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:35.092 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:35.093 INFO:teuthology.orchestra.run.smithi181.stderr:37 bytes copied, 0.000203487 s, 182 kB/s 2020-11-16T22:26:35.093 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:35.094 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:26:35.302 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:26:35.311 INFO:teuthology.orchestra.run.smithi181.stdout:47600a3e-3e36-414c-af17-37a175ef1fa5 2020-11-16T22:26:35.311 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:35.311 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:35.312 INFO:teuthology.orchestra.run.smithi181.stderr:37 bytes copied, 0.000165994 s, 223 kB/s 2020-11-16T22:26:35.312 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:35.313 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:26:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:26:35.370 INFO:teuthology.orchestra.run.smithi181.stdout:0a826a69-584a-40e6-8be5-9dad35970e19 2020-11-16T22:26:35.370 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:35.370 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:35.370 INFO:teuthology.orchestra.run.smithi181.stderr:37 bytes copied, 0.000191227 s, 193 kB/s 2020-11-16T22:26:35.371 INFO:teuthology.orchestra.run.smithi181:> set -ex 2020-11-16T22:26:35.371 INFO:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T22:26:35.417 INFO:teuthology.orchestra.run.smithi181.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T22:26:35.426 INFO:teuthology.orchestra.run.smithi181.stdout:dcbcd29f-2431-4d31-9f4b-84cb393eb95e 2020-11-16T22:26:35.426 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2020-11-16T22:26:35.426 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2020-11-16T22:26:35.426 INFO:teuthology.orchestra.run.smithi181.stderr:37 bytes copied, 0.00023025 s, 161 kB/s 2020-11-16T22:26:35.427 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:35.427 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T22:26:35.432 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T22:26:35.441 INFO:teuthology.orchestra.run.smithi175.stdout:e6eb7f9b-c36a-46bb-b1b5-756336fffc81 2020-11-16T22:26:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:35.441 INFO:teuthology.orchestra.run.smithi175.stderr:37 bytes copied, 0.000159537 s, 232 kB/s 2020-11-16T22:26:35.442 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:35.442 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T22:26:35.486 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T22:26:35.494 INFO:teuthology.orchestra.run.smithi175.stdout:210445e9-3a4c-4b4a-af5d-b0399ac33558 2020-11-16T22:26:35.494 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:35.494 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:35.494 INFO:teuthology.orchestra.run.smithi175.stderr:37 bytes copied, 0.000172094 s, 215 kB/s 2020-11-16T22:26:35.495 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:35.496 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T22:26:35.543 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T22:26:35.551 INFO:teuthology.orchestra.run.smithi175.stdout:a4d749e2-3789-4d8b-ab3f-56d1b413d88a 2020-11-16T22:26:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:37 bytes copied, 0.000163206 s, 227 kB/s 2020-11-16T22:26:35.552 INFO:teuthology.orchestra.run.smithi175:> set -ex 2020-11-16T22:26:35.552 INFO:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T22:26:35.599 INFO:teuthology.orchestra.run.smithi175.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T22:26:35.607 INFO:teuthology.orchestra.run.smithi175.stdout:af0ccec7-e3da-4514-8621-b26a9adc9245 2020-11-16T22:26:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2020-11-16T22:26:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2020-11-16T22:26:35.608 INFO:teuthology.orchestra.run.smithi175.stderr:37 bytes copied, 0.000153631 s, 241 kB/s 2020-11-16T22:26:35.609 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new ddae4f96-c4ce-4f57-920e-c80b7e795739 0 2020-11-16T22:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 Processor -- start 2020-11-16T22:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 -- start start 2020-11-16T22:26:35.806 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4498145a30 0x7f4498145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:35.806 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f4498080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:35.806 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4498061b50 con 0x7f44980814c0 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44980618b0 con 0x7f4498145a30 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449e98e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44980cb930 con 0x7f44981492c0 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4498145a30 0x7f4498145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4498145a30 0x7f4498145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38388/0 (socket says 172.21.15.175:38388) 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 -- 172.21.15.175:0/423778154 learned_addr learned my addr 172.21.15.175:0/423778154 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:35.807 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d18b700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f4498080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 -- 172.21.15.175:0/423778154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 msgr2=0x7f4498080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f4498080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 -- 172.21.15.175:0/423778154 >> v1:172.21.15.181:6789/0 conn(0x7f44980814c0 legacy=0x7f44980cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 -- 172.21.15.175:0/423778154 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980cbbd0 con 0x7f4498145a30 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.820+0000 7f449d98c700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4498145a30 0x7f4498145e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4494006fd0 tx=0x7f4494006eb0).ready entity=mon.0 client_cookie=4449170623ea0660 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:35.809 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449c98a700 1 -- 172.21.15.175:0/423778154 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494009810 con 0x7f4498145a30 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449c98a700 1 -- 172.21.15.175:0/423778154 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494009970 con 0x7f4498145a30 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449c98a700 1 -- 172.21.15.175:0/423778154 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494009cf0 con 0x7f4498145a30 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- 172.21.15.175:0/423778154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4498145a30 msgr2=0x7f4498145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4498145a30 0x7f4498145e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f4494006fd0 tx=0x7f4494006eb0).stop 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- 172.21.15.175:0/423778154 shutdown_connections 2020-11-16T22:26:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4498145a30 0x7f4498145e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.811 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- 172.21.15.175:0/423778154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f4498080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.811 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- 172.21.15.175:0/423778154 >> 172.21.15.175:0/423778154 conn(0x7f44980d68c0 msgr2=0x7f44980d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:35.811 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- 172.21.15.175:0/423778154 shutdown_connections 2020-11-16T22:26:35.811 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- 172.21.15.175:0/423778154 wait complete. 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 Processor -- start 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- start start 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4498145a30 0x7f44980b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f44980bb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:35.813 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44980ba3d0 con 0x7f44980814c0 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449813f360 con 0x7f4498145a30 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e98e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44980d1610 con 0x7f44981492c0 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449d98c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449d98c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38392/0 (socket says 172.21.15.175:38392) 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449d98c700 1 -- 172.21.15.175:0/2273657925 learned_addr learned my addr 172.21.15.175:0/2273657925 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449d18b700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4498145a30 0x7f44980b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:35.814 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.824+0000 7f449e18d700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f44980bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 -- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 msgr2=0x7f44980bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f44980bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 -- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4498145a30 msgr2=0x7f44980b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4498145a30 0x7f44980b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 -- 172.21.15.175:0/2273657925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980d18b0 con 0x7f44980814c0 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449e18d700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f44980bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:35.815 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449d98c700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4494009e10 tx=0x7f4494015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494011d50 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449401e460 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449401e7e0 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44980d1bb0 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44980d2330 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f4494025030 con 0x7f44980814c0 2020-11-16T22:26:35.816 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f44940288a0 con 0x7f44980814c0 2020-11-16T22:26:35.817 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.828+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4498044f60 con 0x7f44980814c0 2020-11-16T22:26:35.819 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.832+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f449402d060 con 0x7f44980814c0 2020-11-16T22:26:35.900 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.916+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ddae4f96-c4ce-4f57-920e-c80b7e795739", "id": 0} v 0) v1 -- 0x7f449815dde0 con 0x7f44980814c0 2020-11-16T22:26:35.905 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f4489ffb700 1 -- 172.21.15.175:0/2273657925 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddae4f96-c4ce-4f57-920e-c80b7e795739", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44980d1bb0 con 0x7f44980814c0 2020-11-16T22:26:35.905 INFO:teuthology.orchestra.run.smithi175.stdout:0 2020-11-16T22:26:35.907 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 msgr2=0x7f44980b99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:35.907 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4494009e10 tx=0x7f4494015000).stop 2020-11-16T22:26:35.907 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 shutdown_connections 2020-11-16T22:26:35.907 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44981492c0 0x7f44980bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.908 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44980814c0 0x7f44980b99f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.908 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 --2- 172.21.15.175:0/2273657925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4498145a30 0x7f44980b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:35.908 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 >> 172.21.15.175:0/2273657925 conn(0x7f44980d68c0 msgr2=0x7f449815e000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:35.908 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.920+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 shutdown_connections 2020-11-16T22:26:35.908 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:35.924+0000 7f449e98e700 1 -- 172.21.15.175:0/2273657925 wait complete. 2020-11-16T22:26:35.922 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 47600a3e-3e36-414c-af17-37a175ef1fa5 1 2020-11-16T22:26:36.041 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 Processor -- start 2020-11-16T22:26:36.041 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 -- start start 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 0x7fdd04159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd04162510 0x7fdd041629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd04161350 con 0x7fdd040b21f0 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd041615f0 con 0x7fdd04162510 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd0ad87700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04161890 con 0x7fdd04159b40 2020-11-16T22:26:36.043 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd09584700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fdd040b21f0 0x7fdd040b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37684/0 (socket says 172.21.15.175:37684) 2020-11-16T22:26:36.044 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd09584700 1 -- 172.21.15.175:0/130506985 learned_addr learned my addr 172.21.15.175:0/130506985 (peer_addr_for_me v1:172.21.15.175:0/0) 2020-11-16T22:26:36.044 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.056+0000 7fdd09d85700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 0x7fdd04159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.044 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 --2- 172.21.15.175:0/130506985 >> v2:172.21.15.181:3300/0 conn(0x7fdd04162510 0x7fdd041629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.045 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd08d83700 1 -- 172.21.15.175:0/130506985 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073830816 0 0) 0x7fdd04161350 con 0x7fdd040b21f0 2020-11-16T22:26:36.045 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd08d83700 1 -- 172.21.15.175:0/130506985 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf0003240 con 0x7fdd040b21f0 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 -- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 msgr2=0x7fdd04159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 0x7fdd04159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 -- 172.21.15.175:0/130506985 >> v1:172.21.15.175:6789/0 conn(0x7fdd040b21f0 legacy=0x7fdd040b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 -- 172.21.15.175:0/130506985 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04161b30 con 0x7fdd04162510 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd09d85700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 0x7fdd04159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0a586700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04162510 0x7fdd041629c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdcfc00b9a0 tx=0x7fdcfc006df0).ready entity=mon.1 client_cookie=b8f712f4e621a4bd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.046 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd08d83700 1 -- 172.21.15.175:0/130506985 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc009680 con 0x7fdd04162510 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd08d83700 1 -- 172.21.15.175:0/130506985 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc0097e0 con 0x7fdd04162510 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd08d83700 1 -- 172.21.15.175:0/130506985 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc009b60 con 0x7fdd04162510 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04162510 msgr2=0x7fdd041629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04162510 0x7fdd041629c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdcfc00b9a0 tx=0x7fdcfc006df0).stop 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- 172.21.15.175:0/130506985 shutdown_connections 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04162510 0x7fdd041629c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/130506985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04159b40 0x7fdd04159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- 172.21.15.175:0/130506985 >> 172.21.15.175:0/130506985 conn(0x7fdd040d4570 msgr2=0x7fdd040d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- 172.21.15.175:0/130506985 shutdown_connections 2020-11-16T22:26:36.048 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- 172.21.15.175:0/130506985 wait complete. 2020-11-16T22:26:36.050 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 Processor -- start 2020-11-16T22:26:36.050 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.060+0000 7fdd0ad87700 1 -- start start 2020-11-16T22:26:36.050 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.050 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 0x7fdd04151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.050 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 0x7fdd040ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0416fee0 con 0x7fdd040b21f0 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04171350 con 0x7fdd04159b40 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041715f0 con 0x7fdd04162510 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38406/0 (socket says 172.21.15.175:38406) 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09584700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 0x7fdd04151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.051 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 -- 172.21.15.175:0/1394102294 learned_addr learned my addr 172.21.15.175:0/1394102294 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0a586700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 0x7fdd040ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 -- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 msgr2=0x7fdd040ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 0x7fdd040ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 -- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 msgr2=0x7fdd04151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 0x7fdd04151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 -- 172.21.15.175:0/1394102294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04171890 con 0x7fdd040b21f0 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09584700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 0x7fdd04151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0a586700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 0x7fdd040ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd09d85700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fdcf400efd0 tx=0x7fdcf40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4009870 con 0x7fdd040b21f0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400cdc0 con 0x7fdd040b21f0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4012600 con 0x7fdd040b21f0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd04171b30 con 0x7fdd040b21f0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd04172070 con 0x7fdd040b21f0 2020-11-16T22:26:36.053 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fdcf4022070 con 0x7fdd040b21f0 2020-11-16T22:26:36.054 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fdcf401b700 con 0x7fdd040b21f0 2020-11-16T22:26:36.054 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.064+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04172230 con 0x7fdd040b21f0 2020-11-16T22:26:36.054 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.068+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fdcf4016020 con 0x7fdd040b21f0 2020-11-16T22:26:36.133 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.148+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "47600a3e-3e36-414c-af17-37a175ef1fa5", "id": 1} v 0) v1 -- 0x7fdd04044f60 con 0x7fdd040b21f0 2020-11-16T22:26:36.137 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd01ffb700 1 -- 172.21.15.175:0/1394102294 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "47600a3e-3e36-414c-af17-37a175ef1fa5", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdcf4012db0 con 0x7fdd040b21f0 2020-11-16T22:26:36.137 INFO:teuthology.orchestra.run.smithi175.stdout:1 2020-11-16T22:26:36.139 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 msgr2=0x7fdd0416f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.139 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fdcf400efd0 tx=0x7fdcf40091d0).stop 2020-11-16T22:26:36.139 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 shutdown_connections 2020-11-16T22:26:36.139 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd04162510 0x7fdd040ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.140 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd040b21f0 0x7fdd0416f9f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.140 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 --2- 172.21.15.175:0/1394102294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd04159b40 0x7fdd04151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.140 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.152+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 >> 172.21.15.175:0/1394102294 conn(0x7fdd040d4570 msgr2=0x7fdd04152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.140 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.156+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 shutdown_connections 2020-11-16T22:26:36.140 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.156+0000 7fdd0ad87700 1 -- 172.21.15.175:0/1394102294 wait complete. 2020-11-16T22:26:36.155 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 0a826a69-584a-40e6-8be5-9dad35970e19 2 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.292+0000 7f3fe7fff700 1 Processor -- start 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.292+0000 7f3fe7fff700 1 -- start start 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fe80816e0 0x7f3fe8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fe81492c0 0x7f3fe80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe8061b50 con 0x7f3fe8145a30 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80618b0 con 0x7f3fe81492c0 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80cb930 con 0x7f3fe80816e0 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fe81492c0 0x7f3fe80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fe81492c0 0x7f3fe80cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38414/0 (socket says 172.21.15.175:38414) 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 -- 172.21.15.175:0/606302915 learned_addr learned my addr 172.21.15.175:0/606302915 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe67fc700 1 --2- 172.21.15.175:0/606302915 >> v2:172.21.15.181:3300/0 conn(0x7f3fe80816e0 0x7f3fe8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 -- 172.21.15.175:0/606302915 >> v2:172.21.15.181:3300/0 conn(0x7f3fe80816e0 msgr2=0x7f3fe8145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 --2- 172.21.15.175:0/606302915 >> v2:172.21.15.181:3300/0 conn(0x7f3fe80816e0 0x7f3fe8145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 -- 172.21.15.175:0/606302915 >> v1:172.21.15.181:6789/0 conn(0x7f3fe8145a30 legacy=0x7f3fe8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 -- 172.21.15.175:0/606302915 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80cbbd0 con 0x7f3fe81492c0 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe77fe700 1 --2- 172.21.15.175:0/606302915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe81492c0 0x7f3fe80cc5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3fe80cd910 tx=0x7f3fd8006eb0).ready entity=mon.0 client_cookie=7a616735b7fa3d63 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe5ffb700 1 -- 172.21.15.175:0/606302915 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8009810 con 0x7f3fe81492c0 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe5ffb700 1 -- 172.21.15.175:0/606302915 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8009970 con 0x7f3fe81492c0 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe5ffb700 1 -- 172.21.15.175:0/606302915 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8009cf0 con 0x7f3fe81492c0 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- 172.21.15.175:0/606302915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe81492c0 msgr2=0x7f3fe80cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/606302915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe81492c0 0x7f3fe80cc5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3fe80cd910 tx=0x7f3fd8006eb0).stop 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- 172.21.15.175:0/606302915 shutdown_connections 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/606302915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe81492c0 0x7f3fe80cc5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/606302915 >> v2:172.21.15.181:3300/0 conn(0x7f3fe80816e0 0x7f3fe8145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- 172.21.15.175:0/606302915 >> 172.21.15.175:0/606302915 conn(0x7f3fe80d68c0 msgr2=0x7f3fe80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- 172.21.15.175:0/606302915 shutdown_connections 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.296+0000 7f3fe7fff700 1 -- 172.21.15.175:0/606302915 wait complete. 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 Processor -- start 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- start start 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe8145a30 0x7f3fe8142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 0x7f3fe8143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8170e30 con 0x7f3fe80816e0 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81710d0 con 0x7f3fe8145a30 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8171370 con 0x7f3fe81492c0 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38418/0 (socket says 172.21.15.175:38418) 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 -- 172.21.15.175:0/4281998171 learned_addr learned my addr 172.21.15.175:0/4281998171 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe77fe700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 0x7f3fe8143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe67fc700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe8145a30 0x7f3fe8142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 -- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 msgr2=0x7f3fe8143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 0x7f3fe8143180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 -- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe8145a30 msgr2=0x7f3fe8142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe8145a30 0x7f3fe8142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 -- 172.21.15.175:0/4281998171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8171610 con 0x7f3fe80816e0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe77fe700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 0x7f3fe8143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe6ffd700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3fe000c6e0 tx=0x7f3fe0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007020 con 0x7f3fe80816e0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe0007180 con 0x7f3fe80816e0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe81718b0 con 0x7f3fe80816e0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000e560 con 0x7f3fe80816e0 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe8143670 con 0x7f3fe80816e0 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f3fe0010070 con 0x7f3fe80816e0 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.300+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f3fe00065e0 con 0x7f3fe80816e0 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.304+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8171d50 con 0x7f3fe80816e0 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.308+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f3fe8171d50 con 0x7f3fe80816e0 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.392+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0a826a69-584a-40e6-8be5-9dad35970e19", "id": 2} v 0) v1 -- 0x7f3fe8044f60 con 0x7f3fe80816e0 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fdeffd700 1 -- 172.21.15.175:0/4281998171 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0a826a69-584a-40e6-8be5-9dad35970e19", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3fe00290b0 con 0x7f3fe80816e0 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi175.stdout:2 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 msgr2=0x7f3fe815df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3fe000c6e0 tx=0x7f3fe0007c50).stop 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 shutdown_connections 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe81492c0 0x7f3fe8143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80816e0 0x7f3fe815df20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 --2- 172.21.15.175:0/4281998171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe8145a30 0x7f3fe8142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 >> 172.21.15.175:0/4281998171 conn(0x7f3fe80d68c0 msgr2=0x7f3fe80ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 shutdown_connections 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.400+0000 7f3fe7fff700 1 -- 172.21.15.175:0/4281998171 wait complete. 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new dcbcd29f-2431-4d31-9f4b-84cb393eb95e 3 2020-11-16T22:26:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 Processor -- start 2020-11-16T22:26:36.515 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 -- start start 2020-11-16T22:26:36.517 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 0x7f98c00600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.517 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c015e090 con 0x7f98c0147270 2020-11-16T22:26:36.517 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c015e330 con 0x7f98c015d4e0 2020-11-16T22:26:36.518 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98c5d66700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c015edb0 con 0x7f98c0146e10 2020-11-16T22:26:36.518 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98bffff700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f98c0147270 0x7f98c00605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37710/0 (socket says 172.21.15.175:37710) 2020-11-16T22:26:36.518 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98bffff700 1 -- 172.21.15.175:0/696730921 learned_addr learned my addr 172.21.15.175:0/696730921 (peer_addr_for_me v1:172.21.15.175:0/0) 2020-11-16T22:26:36.518 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98b6ffd700 1 --2- 172.21.15.175:0/696730921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 0x7f98c00600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.519 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495943224 0 0) 0x7f98c015e090 con 0x7f98c0147270 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a0003240 con 0x7f98c0147270 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959121238 0 0) 0x7f98c015e330 con 0x7f98c015d4e0 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c015e090 con 0x7f98c015d4e0 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98b6ffd700 1 -- 172.21.15.175:0/696730921 >> v1:172.21.15.181:6789/0 conn(0x7f98c015d4e0 legacy=0x7f98c015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98b6ffd700 1 -- 172.21.15.175:0/696730921 >> v1:172.21.15.175:6789/0 conn(0x7f98c0147270 legacy=0x7f98c00605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.520 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98b6ffd700 1 -- 172.21.15.175:0/696730921 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c015e5d0 con 0x7f98c0146e10 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98b6ffd700 1 --2- 172.21.15.175:0/696730921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 0x7f98c00600f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f98a800b9d0 tx=0x7f98a8006df0).ready entity=mon.2 client_cookie=8f5eec968483c59c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8009680 con 0x7f98c0146e10 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a80097e0 con 0x7f98c0146e10 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.532+0000 7f98beffd700 1 -- 172.21.15.175:0/696730921 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8009b60 con 0x7f98c0146e10 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/696730921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 msgr2=0x7f98c00600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 --2- 172.21.15.175:0/696730921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 0x7f98c00600f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f98a800b9d0 tx=0x7f98a8006df0).stop 2020-11-16T22:26:36.521 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/696730921 shutdown_connections 2020-11-16T22:26:36.522 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 --2- 172.21.15.175:0/696730921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c0146e10 0x7f98c00600f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.522 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/696730921 >> 172.21.15.175:0/696730921 conn(0x7f98c00d3680 msgr2=0x7f98c00d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.522 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/696730921 shutdown_connections 2020-11-16T22:26:36.522 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/696730921 wait complete. 2020-11-16T22:26:36.524 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 Processor -- start 2020-11-16T22:26:36.524 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- start start 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 0x7f98c015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c015d4e0 0x7f98c00629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0140780 con 0x7f98c0147270 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0140a20 con 0x7f98c0146e10 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.525 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38430/0 (socket says 172.21.15.175:38430) 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 -- 172.21.15.175:0/3294748007 learned_addr learned my addr 172.21.15.175:0/3294748007 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0140cc0 con 0x7f98c015d4e0 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98bf7fe700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 0x7f98c015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98bffff700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c015d4e0 0x7f98c00629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 -- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c015d4e0 msgr2=0x7f98c00629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c015d4e0 0x7f98c00629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.526 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 -- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 msgr2=0x7f98c015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 0x7f98c015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98b6ffd700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0140f60 con 0x7f98c0147270 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.536+0000 7f98bf7fe700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 0x7f98c015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b6ffd700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f98a8009380 tx=0x7f98a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8011bc0 con 0x7f98c0147270 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8011d20 con 0x7f98c0147270 2020-11-16T22:26:36.527 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c0141200 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a801e670 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00d2690 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f98a8025030 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f98a801ec80 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.540+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0069eb0 con 0x7f98c0147270 2020-11-16T22:26:36.528 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.544+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f98a8023070 con 0x7f98c0147270 2020-11-16T22:26:36.609 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.624+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dcbcd29f-2431-4d31-9f4b-84cb393eb95e", "id": 3} v 0) v1 -- 0x7f98c0062210 con 0x7f98c0147270 2020-11-16T22:26:36.616 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98b7fff700 1 -- 172.21.15.175:0/3294748007 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dcbcd29f-2431-4d31-9f4b-84cb393eb95e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98a8006020 con 0x7f98c0147270 2020-11-16T22:26:36.617 INFO:teuthology.orchestra.run.smithi175.stdout:3 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 msgr2=0x7f98c00d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f98a8009380 tx=0x7f98a8015000).stop 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 shutdown_connections 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98c015d4e0 0x7f98c00629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98c0146e10 0x7f98c015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.618 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 --2- 172.21.15.175:0/3294748007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0147270 0x7f98c00d21a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.619 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 >> 172.21.15.175:0/3294748007 conn(0x7f98c00d3680 msgr2=0x7f98c0143250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.619 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 shutdown_connections 2020-11-16T22:26:36.619 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.632+0000 7f98c5d66700 1 -- 172.21.15.175:0/3294748007 wait complete. 2020-11-16T22:26:36.630 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new e6eb7f9b-c36a-46bb-b1b5-756336fffc81 4 2020-11-16T22:26:36.745 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.760+0000 7f9e8cd3a700 1 Processor -- start 2020-11-16T22:26:36.745 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.760+0000 7f9e8cd3a700 1 -- start start 2020-11-16T22:26:36.748 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e8815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.748 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e8815ff80 0x7f9e88160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.748 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e880ce460 con 0x7f9e880b2570 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880ce700 con 0x7f9e8815ff80 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880ce9a0 con 0x7f9e880b2190 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e8815ff80 0x7f9e88160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e8815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e8815ff80 0x7f9e88160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.175:37972/0 (socket says 172.21.15.175:37972) 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 -- 172.21.15.175:0/476637507 learned_addr learned my addr 172.21.15.175:0/476637507 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 -- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 msgr2=0x7f9e8815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 --2- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e8815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 -- 172.21.15.175:0/476637507 >> v1:172.21.15.181:6789/0 conn(0x7f9e880b2570 legacy=0x7f9e880cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 -- 172.21.15.175:0/476637507 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880cec40 con 0x7f9e8815ff80 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e87fff700 1 --2- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e88160410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9e80009d50 tx=0x7f9e80009290).ready entity=mon.1 client_cookie=1d4435825d1c2f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e86ffd700 1 -- 172.21.15.175:0/476637507 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000bbc0 con 0x7f9e8815ff80 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e86ffd700 1 -- 172.21.15.175:0/476637507 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000bd20 con 0x7f9e8815ff80 2020-11-16T22:26:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e86ffd700 1 -- 172.21.15.175:0/476637507 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80011610 con 0x7f9e8815ff80 2020-11-16T22:26:36.751 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 msgr2=0x7f9e88160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.751 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e88160410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9e80009d50 tx=0x7f9e80009290).stop 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/476637507 shutdown_connections 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e88160410 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/476637507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e8815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/476637507 >> 172.21.15.175:0/476637507 conn(0x7f9e880cfe90 msgr2=0x7f9e880d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/476637507 shutdown_connections 2020-11-16T22:26:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.764+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/476637507 wait complete. 2020-11-16T22:26:36.753 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 Processor -- start 2020-11-16T22:26:36.753 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- start start 2020-11-16T22:26:36.754 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e88166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e880b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815b7d0 con 0x7f9e880b2570 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815e480 con 0x7f9e8815ff80 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815e720 con 0x7f9e880b2190 2020-11-16T22:26:36.755 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e88166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38442/0 (socket says 172.21.15.175:38442) 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7effd700 1 -- 172.21.15.175:0/1627223858 learned_addr learned my addr 172.21.15.175:0/1627223858 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e880b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 -- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 msgr2=0x7f9e88166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e88166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.756 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 -- 172.21.15.175:0/1627223858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 msgr2=0x7f9e88166c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 -- 172.21.15.175:0/1627223858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815e9c0 con 0x7f9e8815ff80 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7effd700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e87fff700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e880b6580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9e8805ffe0 tx=0x7f9e80016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000bbc0 con 0x7f9e8815ff80 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000be90 con 0x7f9e8815ff80 2020-11-16T22:26:36.757 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8815ec60 con 0x7f9e8815ff80 2020-11-16T22:26:36.758 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80010c70 con 0x7f9e8815ff80 2020-11-16T22:26:36.758 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8815f1a0 con 0x7f9e8815ff80 2020-11-16T22:26:36.758 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f9e80017070 con 0x7f9e8815ff80 2020-11-16T22:26:36.758 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f9e80006b20 con 0x7f9e8815ff80 2020-11-16T22:26:36.758 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.768+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8815e2c0 con 0x7f9e8815ff80 2020-11-16T22:26:36.761 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.776+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f9e80015020 con 0x7f9e8815ff80 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.860+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6eb7f9b-c36a-46bb-b1b5-756336fffc81", "id": 4} v 0) v1 -- 0x7f9e8815b910 con 0x7f9e8815ff80 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e7ffff700 1 -- 172.21.15.175:0/1627223858 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6eb7f9b-c36a-46bb-b1b5-756336fffc81", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e80010dd0 con 0x7f9e8815ff80 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi175.stdout:4 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 msgr2=0x7f9e880b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e880b6580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9e8805ffe0 tx=0x7f9e80016000).stop 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 shutdown_connections 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e8815ff80 0x7f9e880b6580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e880b2190 0x7f9e88166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 --2- 172.21.15.175:0/1627223858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e880b2570 0x7f9e88166c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 >> 172.21.15.175:0/1627223858 conn(0x7f9e880cfe90 msgr2=0x7f9e88060140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 shutdown_connections 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:36.872+0000 7f9e8cd3a700 1 -- 172.21.15.175:0/1627223858 wait complete. 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 210445e9-3a4c-4b4a-af5d-b0399ac33558 5 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 Processor -- start 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 -- start start 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12700600f0 0x7f12700b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700cf030 con 0x7f12700b44d0 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700b3310 con 0x7f1270165880 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f12770b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12700b3120 con 0x7f12700600f0 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f1275654700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f12700b44d0 0x7f1270149a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37734/0 (socket says 172.21.15.175:37734) 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f1275654700 1 -- 172.21.15.175:0/367435370 learned_addr learned my addr 172.21.15.175:0/367435370 (peer_addr_for_me v1:172.21.15.175:0/0) 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f1274e53700 1 --2- 172.21.15.175:0/367435370 >> v2:172.21.15.181:3300/0 conn(0x7f12700600f0 0x7f12700b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918493906 0 0) 0x7f12700cf030 con 0x7f12700b44d0 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.004+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1254003240 con 0x7f12700b44d0 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2647495654 0 0) 0x7f1254003240 con 0x7f12700b44d0 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 >> v2:172.21.15.181:3300/0 conn(0x7f12700600f0 msgr2=0x7f12700b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 --2- 172.21.15.175:0/367435370 >> v2:172.21.15.181:3300/0 conn(0x7f12700600f0 0x7f12700b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 >> v1:172.21.15.181:6789/0 conn(0x7f1270165880 legacy=0x7f12700604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700b35b0 con 0x7f12700b44d0 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f1268001860 con 0x7f12700b44d0 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 >> v1:172.21.15.175:6789/0 conn(0x7f12700b44d0 legacy=0x7f1270149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254003760 0x7f1254003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12540044e0 0x7f1254004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 0x7f125400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1254004350 con 0x7f1254004e50 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125400bc20 con 0x7f1254003760 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125400bdb0 con 0x7f12540044e0 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 0x7f125400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1274e53700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254003760 0x7f1254003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126ffff700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12540044e0 0x7f1254004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 -- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12540044e0 msgr2=0x7f1254004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12540044e0 0x7f1254004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 -- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254003760 msgr2=0x7f1254003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254003760 0x7f1254003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 -- 172.21.15.175:0/367435370 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254003240 con 0x7f1254004e50 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 -- 172.21.15.175:0/367435370 reap_dead start 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f1275654700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 0x7f125400b730 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f126800c870 tx=0x7f12700b3740).ready entity=mon.0 client_cookie=3e219a8a27cc4e91 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268005b80 con 0x7f1254004e50 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268005ce0 con 0x7f1254004e50 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f126f7fe700 1 -- 172.21.15.175:0/367435370 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680072e0 con 0x7f1254004e50 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 -- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 msgr2=0x7f125400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 0x7f125400b730 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f126800c870 tx=0x7f12700b3740).stop 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 -- 172.21.15.175:0/367435370 shutdown_connections 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 --2- 172.21.15.175:0/367435370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1254004e50 0x7f125400b730 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 -- 172.21.15.175:0/367435370 >> 172.21.15.175:0/367435370 conn(0x7f12700d3680 msgr2=0x7f12700d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 -- 172.21.15.175:0/367435370 shutdown_connections 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 -- 172.21.15.175:0/367435370 wait complete. 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.008+0000 7f12770b7700 1 Processor -- start 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- start start 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254004e50 0x7f12700aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12700af0a0 0x7f12700b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270174280 con 0x7f12700b87c0 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270174a90 con 0x7f1254004e50 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1270174d30 con 0x7f12700af0a0 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38462/0 (socket says 172.21.15.175:38462) 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 -- 172.21.15.175:0/669654258 learned_addr learned my addr 172.21.15.175:0/669654258 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12700af0a0 0x7f12700b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1274e53700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254004e50 0x7f12700aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 -- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12700af0a0 msgr2=0x7f12700b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12700af0a0 0x7f12700b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 -- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254004e50 msgr2=0x7f12700aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254004e50 0x7f12700aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 -- 172.21.15.175:0/669654258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270174fd0 con 0x7f12700b87c0 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f1275654700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f1268000f40 tx=0x7f1268013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268007860 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12680138b0 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270175270 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268013c50 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12700b8c40 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f1268026070 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f1268001f20 con 0x7f12700b87c0 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.012+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1270174450 con 0x7f12700b87c0 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.016+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f1268024060 con 0x7f12700b87c0 2020-11-16T22:26:37.083 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.096+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "210445e9-3a4c-4b4a-af5d-b0399ac33558", "id": 5} v 0) v1 -- 0x7f1270069eb0 con 0x7f12700b87c0 2020-11-16T22:26:37.091 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.104+0000 7f126cff9700 1 -- 172.21.15.175:0/669654258 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "210445e9-3a4c-4b4a-af5d-b0399ac33558", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1268001510 con 0x7f12700b87c0 2020-11-16T22:26:37.091 INFO:teuthology.orchestra.run.smithi175.stdout:5 2020-11-16T22:26:37.092 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 msgr2=0x7f12700cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.092 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f1268000f40 tx=0x7f1268013000).stop 2020-11-16T22:26:37.092 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 shutdown_connections 2020-11-16T22:26:37.092 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12700b87c0 0x7f12700cf920 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.093 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1254004e50 0x7f12700aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.093 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 --2- 172.21.15.175:0/669654258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12700af0a0 0x7f12700b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.093 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 >> 172.21.15.175:0/669654258 conn(0x7f12700d3680 msgr2=0x7f12700d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:37.093 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 shutdown_connections 2020-11-16T22:26:37.093 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.108+0000 7f12770b7700 1 -- 172.21.15.175:0/669654258 wait complete. 2020-11-16T22:26:37.105 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new a4d749e2-3789-4d8b-ab3f-56d1b413d88a 6 2020-11-16T22:26:37.227 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 Processor -- start 2020-11-16T22:26:37.227 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- start start 2020-11-16T22:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f869413ec50 0x7f8694061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869415d180 con 0x7f869413e870 2020-11-16T22:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869415d810 con 0x7f86940615f0 2020-11-16T22:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869415dab0 con 0x7f869413ec50 2020-11-16T22:26:37.229 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f86927fc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f869413ec50 0x7f8694061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8692ffd700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f869413e870 0x7f869415fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37748/0 (socket says 172.21.15.175:37748) 2020-11-16T22:26:37.230 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8692ffd700 1 -- 172.21.15.175:0/1505700747 learned_addr learned my addr 172.21.15.175:0/1505700747 (peer_addr_for_me v1:172.21.15.175:0/0) 2020-11-16T22:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f86927fc700 1 -- 172.21.15.175:0/1505700747 >> v1:172.21.15.181:6789/0 conn(0x7f86940615f0 legacy=0x7f869415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f86927fc700 1 -- 172.21.15.175:0/1505700747 >> v1:172.21.15.175:6789/0 conn(0x7f869413e870 legacy=0x7f869415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.231 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f86927fc700 1 -- 172.21.15.175:0/1505700747 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869415dd50 con 0x7f869413ec50 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8691ffb700 1 -- 172.21.15.175:0/1505700747 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903592041 0 0) 0x7f869415d180 con 0x7f869413e870 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f86927fc700 1 --2- 172.21.15.175:0/1505700747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413ec50 0x7f8694061100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f868800b9a0 tx=0x7f8688006df0).ready entity=mon.0 client_cookie=eb02f4ccf2236d75 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8691ffb700 1 -- 172.21.15.175:0/1505700747 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86880097b0 con 0x7f869413ec50 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8691ffb700 1 -- 172.21.15.175:0/1505700747 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8688009910 con 0x7f869413ec50 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8691ffb700 1 -- 172.21.15.175:0/1505700747 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688009c90 con 0x7f869413ec50 2020-11-16T22:26:37.232 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- 172.21.15.175:0/1505700747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413ec50 msgr2=0x7f8694061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 --2- 172.21.15.175:0/1505700747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413ec50 0x7f8694061100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f868800b9a0 tx=0x7f8688006df0).stop 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- 172.21.15.175:0/1505700747 shutdown_connections 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 --2- 172.21.15.175:0/1505700747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413ec50 0x7f8694061100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- 172.21.15.175:0/1505700747 >> 172.21.15.175:0/1505700747 conn(0x7f86940cf630 msgr2=0x7f86940cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- 172.21.15.175:0/1505700747 shutdown_connections 2020-11-16T22:26:37.233 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.244+0000 7f8693fff700 1 -- 172.21.15.175:0/1505700747 wait complete. 2020-11-16T22:26:37.235 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 Processor -- start 2020-11-16T22:26:37.235 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 -- start start 2020-11-16T22:26:37.235 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 0x7f8694161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869413ec50 0x7f869415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869415bd60 con 0x7f869413e870 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694161d70 con 0x7f869413ec50 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8693fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869416d750 con 0x7f86940615f0 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f86927fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 0x7f8694161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f86927fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38472/0 (socket says 172.21.15.175:38472) 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f86927fc700 1 -- 172.21.15.175:0/1620445347 learned_addr learned my addr 172.21.15.175:0/1620445347 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f86937fe700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869413ec50 0x7f869415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 -- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869413ec50 msgr2=0x7f869415fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869413ec50 0x7f869415fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 -- 172.21.15.175:0/1620445347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 msgr2=0x7f869416ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.237 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 -- 172.21.15.175:0/1620445347 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869416d9f0 con 0x7f86940615f0 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f86927fc700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f8692ffd700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 0x7f8694161880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f868400c6e0 tx=0x7f86840079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840072e0 con 0x7f86940615f0 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.248+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684007460 con 0x7f86940615f0 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684005840 con 0x7f86940615f0 2020-11-16T22:26:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869416dc90 con 0x7f86940615f0 2020-11-16T22:26:37.239 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869416e1d0 con 0x7f86940615f0 2020-11-16T22:26:37.239 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f8684010070 con 0x7f86940615f0 2020-11-16T22:26:37.239 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f868401da10 con 0x7f86940615f0 2020-11-16T22:26:37.239 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.252+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869416e8e0 con 0x7f86940615f0 2020-11-16T22:26:37.241 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.256+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f8684034880 con 0x7f86940615f0 2020-11-16T22:26:37.324 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.340+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4d749e2-3789-4d8b-ab3f-56d1b413d88a", "id": 6} v 0) v1 -- 0x7f8694044f60 con 0x7f86940615f0 2020-11-16T22:26:37.335 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.348+0000 7f867affd700 1 -- 172.21.15.175:0/1620445347 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4d749e2-3789-4d8b-ab3f-56d1b413d88a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8684023030 con 0x7f86940615f0 2020-11-16T22:26:37.335 INFO:teuthology.orchestra.run.smithi175.stdout:6 2020-11-16T22:26:37.336 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 msgr2=0x7f8694161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.336 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 0x7f8694161880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f868400c6e0 tx=0x7f86840079e0).stop 2020-11-16T22:26:37.336 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 shutdown_connections 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869413ec50 0x7f869415fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86940615f0 0x7f8694161880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 --2- 172.21.15.175:0/1620445347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869413e870 0x7f869416ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 >> 172.21.15.175:0/1620445347 conn(0x7f86940cf630 msgr2=0x7f869415e650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 shutdown_connections 2020-11-16T22:26:37.337 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.352+0000 7f8693fff700 1 -- 172.21.15.175:0/1620445347 wait complete. 2020-11-16T22:26:37.350 INFO:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new af0ccec7-e3da-4514-8621-b26a9adc9245 7 2020-11-16T22:26:37.466 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.480+0000 7f13bd5f4700 1 Processor -- start 2020-11-16T22:26:37.466 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.480+0000 7f13bd5f4700 1 -- start start 2020-11-16T22:26:37.468 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b813e870 0x7f13b813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13b815fdc0 0x7f13b8160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b815d180 con 0x7f13b8061100 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b8160730 con 0x7f13b815fdc0 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b813f140 con 0x7f13b813e870 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13b815fdc0 0x7f13b8160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.469 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13b815fdc0 0x7f13b8160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38478/0 (socket says 172.21.15.175:38478) 2020-11-16T22:26:37.470 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 -- 172.21.15.175:0/4287949851 learned_addr learned my addr 172.21.15.175:0/4287949851 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:37.470 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b7fff700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b813e870 0x7f13b813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.471 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 -- 172.21.15.175:0/4287949851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b813e870 msgr2=0x7f13b813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:37.471 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b813e870 0x7f13b813ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.471 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 -- 172.21.15.175:0/4287949851 >> v1:172.21.15.181:6789/0 conn(0x7f13b8061100 legacy=0x7f13b80615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 -- 172.21.15.175:0/4287949851 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b815d810 con 0x7f13b815fdc0 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b77fe700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b815fdc0 0x7f13b8160240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f13ac006fd0 tx=0x7f13ac006eb0).ready entity=mon.0 client_cookie=dec7410bdbe56679 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b6ffd700 1 -- 172.21.15.175:0/4287949851 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac014070 con 0x7f13b815fdc0 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b6ffd700 1 -- 172.21.15.175:0/4287949851 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac009810 con 0x7f13b815fdc0 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13b6ffd700 1 -- 172.21.15.175:0/4287949851 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac009b90 con 0x7f13b815fdc0 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- 172.21.15.175:0/4287949851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b815fdc0 msgr2=0x7f13b8160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.472 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b815fdc0 0x7f13b8160240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f13ac006fd0 tx=0x7f13ac006eb0).stop 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- 172.21.15.175:0/4287949851 shutdown_connections 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b813e870 0x7f13b813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/4287949851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b815fdc0 0x7f13b8160240 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- 172.21.15.175:0/4287949851 >> 172.21.15.175:0/4287949851 conn(0x7f13b80cf630 msgr2=0x7f13b80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- 172.21.15.175:0/4287949851 shutdown_connections 2020-11-16T22:26:37.473 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.484+0000 7f13bd5f4700 1 -- 172.21.15.175:0/4287949851 wait complete. 2020-11-16T22:26:37.474 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 Processor -- start 2020-11-16T22:26:37.474 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 -- start start 2020-11-16T22:26:37.475 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13b8061100 0x7f13b815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.475 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.475 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b815fdc0 0x7f13b813cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:37.475 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b816e810 con 0x7f13b813e870 2020-11-16T22:26:37.475 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b813db30 con 0x7f13b8061100 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13bd5f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b813ddd0 con 0x7f13b815fdc0 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13b77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13b77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:38484/0 (socket says 172.21.15.175:38484) 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13b77fe700 1 -- 172.21.15.175:0/3122835843 learned_addr learned my addr 172.21.15.175:0/3122835843 (peer_addr_for_me v2:172.21.15.175:0/0) 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.488+0000 7f13b7fff700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13b8061100 0x7f13b815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.476 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13bcdf3700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b815fdc0 0x7f13b813cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:37.477 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 -- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b815fdc0 msgr2=0x7f13b813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b815fdc0 0x7f13b813cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 -- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13b8061100 msgr2=0x7f13b815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13b8061100 0x7f13b815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 -- 172.21.15.175:0/3122835843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b813e070 con 0x7f13b813e870 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13b77fe700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f13ac009fd0 tx=0x7f13ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac010cd0 con 0x7f13b813e870 2020-11-16T22:26:37.478 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac010e30 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac01c670 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b816ea70 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b813d3d0 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f13ac00e040 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f13ac01ce40 con 0x7f13b813e870 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.492+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b8044f60 con 0x7f13b813e870 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.496+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f13ac01c9e0 con 0x7f13b813e870 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.580+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af0ccec7-e3da-4514-8621-b26a9adc9245", "id": 7} v 0) v1 -- 0x7f13b815e9f0 con 0x7f13b813e870 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.588+0000 7f1397fff700 1 -- 172.21.15.175:0/3122835843 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af0ccec7-e3da-4514-8621-b26a9adc9245", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13ac0330b0 con 0x7f13b813e870 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi175.stdout:7 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.588+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 msgr2=0x7f13b815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.588+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f13ac009fd0 tx=0x7f13ac016000).stop 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.588+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 shutdown_connections 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.588+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13b815fdc0 0x7f13b813cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.592+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13b8061100 0x7f13b815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.592+0000 7f13bd5f4700 1 --2- 172.21.15.175:0/3122835843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13b813e870 0x7f13b815b280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.592+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 >> 172.21.15.175:0/3122835843 conn(0x7f13b80cf630 msgr2=0x7f13b80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.592+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 shutdown_connections 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi175.stderr:2020-11-16T22:26:37.592+0000 7f13bd5f4700 1 -- 172.21.15.175:0/3122835843 wait complete. 2020-11-16T22:26:37.590 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-16T22:26:37.592 INFO:tasks.ceph.osd.0:Started 2020-11-16T22:26:37.592 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-16T22:26:37.594 INFO:tasks.ceph.osd.1:Started 2020-11-16T22:26:37.595 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T22:26:37.597 INFO:tasks.ceph.osd.2:Started 2020-11-16T22:26:37.597 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-16T22:26:37.599 INFO:tasks.ceph.osd.3:Started 2020-11-16T22:26:37.600 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-16T22:26:37.602 INFO:tasks.ceph.osd.4:Started 2020-11-16T22:26:37.602 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-16T22:26:37.604 INFO:tasks.ceph.osd.5:Started 2020-11-16T22:26:37.604 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-16T22:26:37.606 INFO:tasks.ceph.osd.6:Started 2020-11-16T22:26:37.607 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-16T22:26:37.609 INFO:tasks.ceph.osd.7:Started 2020-11-16T22:26:37.610 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-16T22:26:37.961 INFO:tasks.ceph.osd.4.smithi175.stderr:2020-11-16T22:26:37.976+0000 7fb6bb636d80 -1 Falling back to public interface 2020-11-16T22:26:38.209 INFO:tasks.ceph.osd.6.smithi175.stderr:2020-11-16T22:26:38.224+0000 7fd5bc2acd80 -1 Falling back to public interface 2020-11-16T22:26:38.215 INFO:tasks.ceph.osd.2.smithi181.stderr:2020-11-16T22:26:38.229+0000 7fb882a36d80 -1 Falling back to public interface 2020-11-16T22:26:38.228 INFO:tasks.ceph.osd.7.smithi175.stderr:2020-11-16T22:26:38.244+0000 7fb01480bd80 -1 Falling back to public interface 2020-11-16T22:26:38.231 INFO:tasks.ceph.osd.0.smithi181.stderr:2020-11-16T22:26:38.245+0000 7f2511838d80 -1 Falling back to public interface 2020-11-16T22:26:38.232 INFO:tasks.ceph.osd.1.smithi181.stderr:2020-11-16T22:26:38.245+0000 7f67aac97d80 -1 Falling back to public interface 2020-11-16T22:26:38.232 INFO:tasks.ceph.osd.5.smithi175.stderr:2020-11-16T22:26:38.248+0000 7f324df29d80 -1 Falling back to public interface 2020-11-16T22:26:38.259 INFO:tasks.ceph.osd.3.smithi181.stderr:2020-11-16T22:26:38.273+0000 7f9756806d80 -1 Falling back to public interface 2020-11-16T22:26:38.417 INFO:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:38.523 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 Processor -- start 2020-11-16T22:26:38.524 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 -- start start 2020-11-16T22:26:38.525 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fedcc156a90 0x7fedcc061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:38.525 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:38.525 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0cccd0 con 0x7fedcc15f1a0 2020-11-16T22:26:38.525 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0ccf70 con 0x7fedcc156a90 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15e030 con 0x7fedcc0615f0 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35530/0 (socket says 172.21.15.181:35530) 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 -- 172.21.15.181:0/1584175975 learned_addr learned my addr 172.21.15.181:0/1584175975 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedcaffd700 1 --2- 172.21.15.181:0/1584175975 >> v2:172.21.15.175:3300/0 conn(0x7fedcc156a90 0x7fedcc061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 -- 172.21.15.181:0/1584175975 >> v2:172.21.15.175:3300/0 conn(0x7fedcc156a90 msgr2=0x7fedcc061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:38.526 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 --2- 172.21.15.181:0/1584175975 >> v2:172.21.15.175:3300/0 conn(0x7fedcc156a90 0x7fedcc061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 -- 172.21.15.181:0/1584175975 >> v1:172.21.15.175:6789/0 conn(0x7fedcc15f1a0 legacy=0x7fedcc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 -- 172.21.15.181:0/1584175975 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15e2d0 con 0x7fedcc0615f0 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedca7fc700 1 --2- 172.21.15.181:0/1584175975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fedc0006fd0 tx=0x7fedc0006eb0).ready entity=mon.2 client_cookie=66fa691b682d2653 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedc9ffb700 1 -- 172.21.15.181:0/1584175975 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0009810 con 0x7fedcc0615f0 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedc9ffb700 1 -- 172.21.15.181:0/1584175975 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0009970 con 0x7fedcc0615f0 2020-11-16T22:26:38.527 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.537+0000 7fedc9ffb700 1 -- 172.21.15.181:0/1584175975 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0009cf0 con 0x7fedcc0615f0 2020-11-16T22:26:38.528 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- 172.21.15.181:0/1584175975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 msgr2=0x7fedcc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:38.528 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1584175975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fedc0006fd0 tx=0x7fedc0006eb0).stop 2020-11-16T22:26:38.528 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- 172.21.15.181:0/1584175975 shutdown_connections 2020-11-16T22:26:38.528 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1584175975 >> v2:172.21.15.175:3300/0 conn(0x7fedcc156a90 0x7fedcc061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.529 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1584175975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc0615f0 0x7fedcc15ecb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.529 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- 172.21.15.181:0/1584175975 >> 172.21.15.181:0/1584175975 conn(0x7fedcc0cf630 msgr2=0x7fedcc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:38.529 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- 172.21.15.181:0/1584175975 shutdown_connections 2020-11-16T22:26:38.529 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- 172.21.15.181:0/1584175975 wait complete. 2020-11-16T22:26:38.529 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 Processor -- start 2020-11-16T22:26:38.530 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- start start 2020-11-16T22:26:38.531 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0615f0 0x7fedcc12da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:38.531 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:38.531 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 0x7fedcc170f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:38.531 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0b8930 con 0x7fedcc0615f0 2020-11-16T22:26:38.531 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc16e8a0 con 0x7fedcc15f1a0 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc16eb40 con 0x7fedcc156a90 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35534/0 (socket says 172.21.15.181:35534) 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 -- 172.21.15.181:0/1009046672 learned_addr learned my addr 172.21.15.181:0/1009046672 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcaffd700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0615f0 0x7fedcc12da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcb7fe700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 0x7fedcc170f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:38.532 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 -- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 msgr2=0x7fedcc170f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 0x7fedcc170f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 -- 172.21.15.181:0/1009046672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0615f0 msgr2=0x7fedcc12da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0615f0 0x7fedcc12da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 -- 172.21.15.181:0/1009046672 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc16ede0 con 0x7fedcc156a90 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedcb7fe700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 0x7fedcc170f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.541+0000 7fedca7fc700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fedcc15e170 tx=0x7fedc0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:38.533 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011d50 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc001e460 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc16f080 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc001e7e0 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc171410 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fedc0025030 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7fedc0028be0 con 0x7fedcc156a90 2020-11-16T22:26:38.534 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc044f60 con 0x7fedcc156a90 2020-11-16T22:26:38.535 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.545+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fedc002d060 con 0x7fedcc156a90 2020-11-16T22:26:38.607 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedcc0b8bf0 con 0x7fedcc156a90 2020-11-16T22:26:38.607 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedb2ffd700 1 -- 172.21.15.181:0/1009046672 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7001 (secure 0 0 0) 0x7fedc0023070 con 0x7fedcc156a90 2020-11-16T22:26:38.608 INFO:teuthology.misc.health.smithi181.stdout: 2020-11-16T22:26:38.608 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":10,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:37.583849+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:38.609 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 msgr2=0x7fedcc12df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:38.609 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fedcc15e170 tx=0x7fedc0015000).stop 2020-11-16T22:26:38.609 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 shutdown_connections 2020-11-16T22:26:38.609 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc15f1a0 0x7fedcc170f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.609 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0615f0 0x7fedcc12da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.610 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 --2- 172.21.15.181:0/1009046672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc156a90 0x7fedcc12df40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:38.610 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 >> 172.21.15.181:0/1009046672 conn(0x7fedcc0cf630 msgr2=0x7fedcc167bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:38.610 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 shutdown_connections 2020-11-16T22:26:38.610 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:38.621+0000 7fedcbfff700 1 -- 172.21.15.181:0/1009046672 wait complete. 2020-11-16T22:26:38.624 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-16T22:26:39.395 INFO:tasks.ceph.osd.4.smithi175.stderr:2020-11-16T22:26:39.412+0000 7fb6bb636d80 -1 osd.4 0 log_to_monitors {default=true} 2020-11-16T22:26:39.621 INFO:tasks.ceph.osd.6.smithi175.stderr:2020-11-16T22:26:39.636+0000 7fd5bc2acd80 -1 osd.6 0 log_to_monitors {default=true} 2020-11-16T22:26:39.646 INFO:tasks.ceph.osd.7.smithi175.stderr:2020-11-16T22:26:39.660+0000 7fb01480bd80 -1 osd.7 0 log_to_monitors {default=true} 2020-11-16T22:26:39.664 INFO:tasks.ceph.osd.5.smithi175.stderr:2020-11-16T22:26:39.680+0000 7f324df29d80 -1 osd.5 0 log_to_monitors {default=true} 2020-11-16T22:26:39.671 INFO:tasks.ceph.osd.0.smithi181.stderr:2020-11-16T22:26:39.685+0000 7f2511838d80 -1 osd.0 0 log_to_monitors {default=true} 2020-11-16T22:26:39.677 INFO:tasks.ceph.osd.2.smithi181.stderr:2020-11-16T22:26:39.689+0000 7fb882a36d80 -1 osd.2 0 log_to_monitors {default=true} 2020-11-16T22:26:39.697 INFO:tasks.ceph.osd.1.smithi181.stderr:2020-11-16T22:26:39.709+0000 7f67aac97d80 -1 osd.1 0 log_to_monitors {default=true} 2020-11-16T22:26:39.715 INFO:tasks.ceph.osd.3.smithi181.stderr:2020-11-16T22:26:39.729+0000 7f9756806d80 -1 osd.3 0 log_to_monitors {default=true} 2020-11-16T22:26:40.676 INFO:tasks.ceph.osd.6.smithi175.stderr:2020-11-16T22:26:40.692+0000 7fd5b4f79700 -1 osd.6 0 waiting for initial osdmap 2020-11-16T22:26:40.678 INFO:tasks.ceph.osd.4.smithi175.stderr:2020-11-16T22:26:40.692+0000 7fb6b4303700 -1 osd.4 0 waiting for initial osdmap 2020-11-16T22:26:41.680 INFO:tasks.ceph.osd.7.smithi175.stderr:2020-11-16T22:26:41.696+0000 7fb00d4d8700 -1 osd.7 0 waiting for initial osdmap 2020-11-16T22:26:41.681 INFO:tasks.ceph.osd.5.smithi175.stderr:2020-11-16T22:26:41.696+0000 7f32453f3700 -1 osd.5 0 waiting for initial osdmap 2020-11-16T22:26:41.683 INFO:tasks.ceph.osd.0.smithi181.stderr:2020-11-16T22:26:41.697+0000 7f2508d02700 -1 osd.0 0 waiting for initial osdmap 2020-11-16T22:26:41.683 INFO:tasks.ceph.osd.3.smithi181.stderr:2020-11-16T22:26:41.697+0000 7f974dcd0700 -1 osd.3 0 waiting for initial osdmap 2020-11-16T22:26:41.686 INFO:tasks.ceph.osd.1.smithi181.stderr:2020-11-16T22:26:41.701+0000 7f67a2161700 -1 osd.1 0 waiting for initial osdmap 2020-11-16T22:26:41.689 INFO:tasks.ceph.osd.2.smithi181.stderr:2020-11-16T22:26:41.701+0000 7fb87b703700 -1 osd.2 0 waiting for initial osdmap 2020-11-16T22:26:45.429 INFO:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:45.553 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.565+0000 7fbfdd33f700 1 Processor -- start 2020-11-16T22:26:45.553 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.565+0000 7fbfdd33f700 1 -- start start 2020-11-16T22:26:45.554 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.554 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 0x7fbfd8160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd80ce460 con 0x7fbfd80b2570 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd80ce700 con 0x7fbfd815ff80 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80ce9a0 con 0x7fbfd80b2190 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35820/0 (socket says 172.21.15.181:35820) 2020-11-16T22:26:45.555 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 -- 172.21.15.181:0/284643446 learned_addr learned my addr 172.21.15.181:0/284643446 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:45.556 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfd80b2570 0x7fbfd80cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42284/0 (socket says 172.21.15.181:42284) 2020-11-16T22:26:45.557 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdcb3e700 1 --2- 172.21.15.181:0/284643446 >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 0x7fbfd8160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.557 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452843261 0 0) 0x7fbfd80ce460 con 0x7fbfd80b2570 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb8003240 con 0x7fbfd80b2570 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 -- 172.21.15.181:0/284643446 >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 msgr2=0x7fbfd8160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/284643446 >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 0x7fbfd8160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 -- 172.21.15.181:0/284643446 >> v1:172.21.15.181:6789/0 conn(0x7fbfd80b2570 legacy=0x7fbfd80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3274478294 0 0) 0x7fbfb8003240 con 0x7fbfd80b2570 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 -- 172.21.15.181:0/284643446 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80cec40 con 0x7fbfd80b2190 2020-11-16T22:26:45.558 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdcb3e700 1 --2- 172.21.15.181:0/284643446 >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 0x7fbfd8160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/284643446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbfc8006fd0 tx=0x7fbfc8006eb0).ready entity=mon.2 client_cookie=2c11305e88b90751 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0001860 con 0x7fbfd80b2190 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8009410 con 0x7fbfd80b2190 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfd6ffd700 1 -- 172.21.15.181:0/284643446 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8009790 con 0x7fbfd80b2190 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- 172.21.15.181:0/284643446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 msgr2=0x7fbfd815de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/284643446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbfc8006fd0 tx=0x7fbfc8006eb0).stop 2020-11-16T22:26:45.559 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- 172.21.15.181:0/284643446 shutdown_connections 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/284643446 >> v2:172.21.15.175:3300/0 conn(0x7fbfd815ff80 0x7fbfd8160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/284643446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2190 0x7fbfd815de80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- 172.21.15.181:0/284643446 >> 172.21.15.181:0/284643446 conn(0x7fbfd80cfe90 msgr2=0x7fbfd80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- 172.21.15.181:0/284643446 shutdown_connections 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.569+0000 7fbfdd33f700 1 -- 172.21.15.181:0/284643446 wait complete. 2020-11-16T22:26:45.560 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.573+0000 7fbfdd33f700 1 Processor -- start 2020-11-16T22:26:45.561 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.573+0000 7fbfdd33f700 1 -- start start 2020-11-16T22:26:45.563 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 0x7fbfd80b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.564 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.564 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd815ff80 0x7fbfd8060b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.564 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80d2650 con 0x7fbfd80b2190 2020-11-16T22:26:45.564 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80d28f0 con 0x7fbfd815ff80 2020-11-16T22:26:45.564 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80d2b90 con 0x7fbfd80b2570 2020-11-16T22:26:45.766 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.766 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdcb3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd815ff80 0x7fbfd8060b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.766 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35828/0 (socket says 172.21.15.181:35828) 2020-11-16T22:26:45.767 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 -- 172.21.15.181:0/4230390796 learned_addr learned my addr 172.21.15.181:0/4230390796 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:45.767 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 0x7fbfd80b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.769 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 -- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd815ff80 msgr2=0x7fbfd8060b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd815ff80 0x7fbfd8060b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 -- 172.21.15.181:0/4230390796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 msgr2=0x7fbfd80b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 0x7fbfd80b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 -- 172.21.15.181:0/4230390796 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8170530 con 0x7fbfd80b2570 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 0x7fbfd80b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfd77fe700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbfc000ce60 tx=0x7fbfc0001570).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:45.770 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000ab00 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000ac60 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd81707d0 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00058b0 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.577+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd8159aa0 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd8159cf0 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbfc0010030 con 0x7fbfd80b2570 2020-11-16T22:26:45.771 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfcffff700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 0x7fbfb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fbfc006dc30 con 0x7fbfd80b2570 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 0x7fbfb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.581+0000 7fbfd7fff700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 0x7fbfb4057520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbfc8009320 tx=0x7fbfc8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.589+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbfc00442c0 con 0x7fbfd80b2570 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.741+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfd8044f60 con 0x7fbfd80b2570 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.741+0000 7fbfcffff700 1 -- 172.21.15.181:0/4230390796 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12201 (secure 0 0 0) 0x7fbfc00478e0 con 0x7fbfd80b2570 2020-11-16T22:26:45.772 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 msgr2=0x7fbfb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.974 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 0x7fbfb4057520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbfc8009320 tx=0x7fbfc8015000).stop 2020-11-16T22:26:45.974 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 msgr2=0x7fbfd81595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:45.974 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbfc000ce60 tx=0x7fbfc0001570).stop 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 shutdown_connections 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfb4055060 0x7fbfb4057520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd815ff80 0x7fbfd8060b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfd80b2190 0x7fbfd80b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 --2- 172.21.15.181:0/4230390796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfd80b2570 0x7fbfd81595b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 >> 172.21.15.181:0/4230390796 conn(0x7fbfd80cfe90 msgr2=0x7fbfd80600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:45.975 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 shutdown_connections 2020-11-16T22:26:45.976 INFO:teuthology.misc.health.smithi181.stderr:2020-11-16T22:26:45.745+0000 7fbfdd33f700 1 -- 172.21.15.181:0/4230390796 wait complete. 2020-11-16T22:26:45.976 INFO:teuthology.misc.health.smithi181.stdout: 2020-11-16T22:26:45.976 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":16,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:44.720647+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:45.977 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-16T22:26:45.977 INFO:tasks.ceph:Creating RBD pool 2020-11-16T22:26:45.977 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-16T22:26:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.125+0000 7f0fd6f64700 1 Processor -- start 2020-11-16T22:26:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.125+0000 7f0fd6f64700 1 -- start start 2020-11-16T22:26:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.125+0000 7f0fd6f64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 0x7f0fd0145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 0x7f0fd01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd0081f20 con 0x7f0fd0160450 2020-11-16T22:26:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd0060470 con 0x7f0fd0145770 2020-11-16T22:26:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0060710 con 0x7f0fd00d2a30 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 0x7f0fd01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd5761700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0fd0160450 0x7f0fd00d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42296/0 (socket says 172.21.15.181:42296) 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd5761700 1 -- 172.21.15.181:0/3827435207 learned_addr learned my addr 172.21.15.181:0/3827435207 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd5f62700 1 --2- 172.21.15.181:0/3827435207 >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 0x7f0fd0145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd4f60700 1 -- 172.21.15.181:0/3827435207 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712729396 0 0) 0x7f0fd0081f20 con 0x7f0fd0160450 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd4f60700 1 -- 172.21.15.181:0/3827435207 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc003240 con 0x7f0fd0160450 2020-11-16T22:26:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 -- 172.21.15.181:0/3827435207 >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 msgr2=0x7f0fd0145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 --2- 172.21.15.181:0/3827435207 >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 0x7f0fd0145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 -- 172.21.15.181:0/3827435207 >> v1:172.21.15.181:6789/0 conn(0x7f0fd0160450 legacy=0x7f0fd00d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 -- 172.21.15.181:0/3827435207 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00609b0 con 0x7f0fd00d2a30 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd5f62700 1 --2- 172.21.15.181:0/3827435207 >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 0x7f0fd0145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6763700 1 --2- 172.21.15.181:0/3827435207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 0x7f0fd01608b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0fc8009c90 tx=0x7f0fc80091d0).ready entity=mon.2 client_cookie=955af38c36edb0dc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd4f60700 1 -- 172.21.15.181:0/3827435207 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800b9c0 con 0x7f0fd00d2a30 2020-11-16T22:26:46.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd4f60700 1 -- 172.21.15.181:0/3827435207 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800bb20 con 0x7f0fd00d2a30 2020-11-16T22:26:46.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd4f60700 1 -- 172.21.15.181:0/3827435207 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800bea0 con 0x7f0fd00d2a30 2020-11-16T22:26:46.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3827435207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 msgr2=0x7f0fd01608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:46.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3827435207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 0x7f0fd01608b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0fc8009c90 tx=0x7f0fc80091d0).stop 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3827435207 shutdown_connections 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3827435207 >> v2:172.21.15.181:3300/0 conn(0x7f0fd0145770 0x7f0fd0145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3827435207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd00d2a30 0x7f0fd01608b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3827435207 >> 172.21.15.181:0/3827435207 conn(0x7f0fd00d6760 msgr2=0x7f0fd00d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3827435207 shutdown_connections 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3827435207 wait complete. 2020-11-16T22:26:46.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 Processor -- start 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.129+0000 7f0fd6f64700 1 -- start start 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd00d2a30 0x7f0fd0165340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 0x7f0fd0144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0143530 con 0x7f0fd0145770 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd016fd80 con 0x7f0fd00d2a30 2020-11-16T22:26:46.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0143780 con 0x7f0fd0160450 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd5f62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd00d2a30 0x7f0fd0165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35844/0 (socket says 172.21.15.181:35844) 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 -- 172.21.15.181:0/3219148276 learned_addr learned my addr 172.21.15.181:0/3219148276 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd5761700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 0x7f0fd0144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 -- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd00d2a30 msgr2=0x7f0fd0165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:46.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd00d2a30 0x7f0fd0165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:46.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 -- 172.21.15.181:0/3219148276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 msgr2=0x7f0fd0144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:46.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 0x7f0fd0144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:46.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 -- 172.21.15.181:0/3219148276 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd0143a20 con 0x7f0fd0160450 2020-11-16T22:26:46.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd5761700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 0x7f0fd0144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:46.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6763700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0fc8003340 tx=0x7f0fc8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:46.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc801c6c0 con 0x7f0fd0160450 2020-11-16T22:26:46.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc801c820 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd0143cc0 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc801cba0 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd0162720 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.133+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0fc8024030 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.137+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0044f60 con 0x7f0fd0160450 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.137+0000 7f0fc5ffb700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 0x7f0fa8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:46.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.137+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f0fc806f0a0 con 0x7f0fd0160450 2020-11-16T22:26:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.137+0000 7f0fd5f62700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 0x7f0fa8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.137+0000 7f0fd5f62700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 0x7f0fa8057520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0fc0006fd0 tx=0x7f0fc0006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:46.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.145+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0fc8045790 con 0x7f0fd0160450 2020-11-16T22:26:46.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:46.293+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f0fd005d400 con 0x7f0fd0160450 2020-11-16T22:26:47.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.129+0000 7f0fc5ffb700 1 -- 172.21.15.181:0/3219148276 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f0fc80220a0 con 0x7f0fd0160450 2020-11-16T22:26:47.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 msgr2=0x7f0fa8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 0x7f0fa8057520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0fc0006fd0 tx=0x7f0fc0006df0).stop 2020-11-16T22:26:47.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 msgr2=0x7f0fd01644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0fc8003340 tx=0x7f0fc8016000).stop 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 shutdown_connections 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0fa8055060 0x7f0fa8057520 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd0160450 0x7f0fd01644f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd00d2a30 0x7f0fd0165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 --2- 172.21.15.181:0/3219148276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fd0145770 0x7f0fd0144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 >> 172.21.15.181:0/3219148276 conn(0x7f0fd00d6760 msgr2=0x7f0fd015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:47.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 shutdown_connections 2020-11-16T22:26:47.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.133+0000 7f0fd6f64700 1 -- 172.21.15.181:0/3219148276 wait complete. 2020-11-16T22:26:47.121 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'rbd' created 2020-11-16T22:26:47.133 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-16T22:26:47.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.269+0000 7fcbb6187700 1 Processor -- start 2020-11-16T22:26:47.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.269+0000 7fcbb6187700 1 -- start start 2020-11-16T22:26:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 0x7fcbb0163de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb0064620 con 0x7fcbb0161590 2020-11-16T22:26:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb00600f0 con 0x7fcbb0064240 2020-11-16T22:26:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0060390 con 0x7fcbb01642d0 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcbb0161590 0x7fcbb0065a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42314/0 (socket says 172.21.15.181:42314) 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaffff700 1 -- 172.21.15.181:0/720996864 learned_addr learned my addr 172.21.15.181:0/720996864 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaf7fe700 1 --2- 172.21.15.181:0/720996864 >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 0x7fcbb0163de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaeffd700 1 -- 172.21.15.181:0/720996864 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081299903 0 0) 0x7fcbb0064620 con 0x7fcbb0161590 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 -- 172.21.15.181:0/720996864 >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 msgr2=0x7fcbb0163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 --2- 172.21.15.181:0/720996864 >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 0x7fcbb0163de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 -- 172.21.15.181:0/720996864 >> v1:172.21.15.181:6789/0 conn(0x7fcbb0161590 legacy=0x7fcbb0065a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 -- 172.21.15.181:0/720996864 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0060630 con 0x7fcbb01642d0 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaf7fe700 1 --2- 172.21.15.181:0/720996864 >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 0x7fcbb0163de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcba6ffd700 1 --2- 172.21.15.181:0/720996864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00612b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcb9800b9a0 tx=0x7fcb98006df0).ready entity=mon.2 client_cookie=5da6bb0ae3ea18d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaeffd700 1 -- 172.21.15.181:0/720996864 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98009840 con 0x7fcbb01642d0 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaeffd700 1 -- 172.21.15.181:0/720996864 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb980099a0 con 0x7fcbb01642d0 2020-11-16T22:26:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbaeffd700 1 -- 172.21.15.181:0/720996864 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98009d20 con 0x7fcbb01642d0 2020-11-16T22:26:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- 172.21.15.181:0/720996864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 msgr2=0x7fcbb00612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 --2- 172.21.15.181:0/720996864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00612b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcb9800b9a0 tx=0x7fcb98006df0).stop 2020-11-16T22:26:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- 172.21.15.181:0/720996864 shutdown_connections 2020-11-16T22:26:47.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 --2- 172.21.15.181:0/720996864 >> v2:172.21.15.175:3300/0 conn(0x7fcbb0064240 0x7fcbb0163de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 --2- 172.21.15.181:0/720996864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00612b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- 172.21.15.181:0/720996864 >> 172.21.15.181:0/720996864 conn(0x7fcbb00cf770 msgr2=0x7fcbb00cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:47.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- 172.21.15.181:0/720996864 shutdown_connections 2020-11-16T22:26:47.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.273+0000 7fcbb6187700 1 -- 172.21.15.181:0/720996864 wait complete. 2020-11-16T22:26:47.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.277+0000 7fcbb6187700 1 Processor -- start 2020-11-16T22:26:47.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.277+0000 7fcbb6187700 1 -- start start 2020-11-16T22:26:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb0064240 0x7fcbb00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 0x7fcbb00b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0173700 con 0x7fcbb0064240 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01739a0 con 0x7fcbb0161590 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0173c40 con 0x7fcbb01642d0 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 0x7fcbb00b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35858/0 (socket says 172.21.15.181:35858) 2020-11-16T22:26:47.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 -- 172.21.15.181:0/4179651596 learned_addr learned my addr 172.21.15.181:0/4179651596 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:47.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaf7fe700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb0064240 0x7fcbb00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 -- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 msgr2=0x7fcbb00b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 0x7fcbb00b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 -- 172.21.15.181:0/4179651596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb0064240 msgr2=0x7fcbb00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb0064240 0x7fcbb00b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 -- 172.21.15.181:0/4179651596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0173ee0 con 0x7fcbb01642d0 2020-11-16T22:26:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba6ffd700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 0x7fcbb00b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:47.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbaffff700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fcba800cab0 tx=0x7fcba8007d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:47.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800ce80 con 0x7fcbb01642d0 2020-11-16T22:26:47.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8005b80 con 0x7fcbb01642d0 2020-11-16T22:26:47.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb0174180 con 0x7fcbb01642d0 2020-11-16T22:26:47.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007110 con 0x7fcbb01642d0 2020-11-16T22:26:47.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb01733d0 con 0x7fcbb01642d0 2020-11-16T22:26:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcba8010030 con 0x7fcbb01642d0 2020-11-16T22:26:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.281+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0069eb0 con 0x7fcbb01642d0 2020-11-16T22:26:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.285+0000 7fcba7fff700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 0x7fcb8c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.285+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fcba806d8c0 con 0x7fcbb01642d0 2020-11-16T22:26:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.285+0000 7fcbaf7fe700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 0x7fcb8c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.285+0000 7fcbaf7fe700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 0x7fcb8c057520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcba000b710 tx=0x7fcba0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.289+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcba8043e60 con 0x7fcbb01642d0 2020-11-16T22:26:47.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:47.441+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcbb0163bc0 con 0x7fcbb01642d0 2020-11-16T22:26:48.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.133+0000 7fcba7fff700 1 -- 172.21.15.181:0/4179651596 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fcba8047480 con 0x7fcbb01642d0 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 msgr2=0x7fcb8c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 0x7fcb8c057520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcba000b710 tx=0x7fcba0008040).stop 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 msgr2=0x7fcbb00cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fcba800cab0 tx=0x7fcba8007d00).stop 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 shutdown_connections 2020-11-16T22:26:48.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb8c055060 0x7fcb8c057520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb01642d0 0x7fcbb00cc1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb0064240 0x7fcbb00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 --2- 172.21.15.181:0/4179651596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0161590 0x7fcbb00b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 >> 172.21.15.181:0/4179651596 conn(0x7fcbb00cf770 msgr2=0x7fcbb0162b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 shutdown_connections 2020-11-16T22:26:48.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.137+0000 7fcbb6187700 1 -- 172.21.15.181:0/4179651596 wait complete. 2020-11-16T22:26:48.126 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-16T22:26:48.138 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-16T22:26:48.139 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:26:48.139 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:26:48.141 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:26:48.142 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:26:48.142 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:26:48.144 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:26:48.144 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:26:48.144 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:26:48.146 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:26:48.146 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:26:48.147 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:26:48.149 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:26:48.149 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:26:48.149 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:26:48.152 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:26:48.152 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:26:48.153 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:26:48.155 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:26:48.155 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:26:48.155 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:26:48.157 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:26:48.158 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:26:48.158 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:26:48.160 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:26:48.160 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:26:48.160 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:26:48.162 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:26:48.163 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-16T22:26:48.163 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:48.219 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:26:48.225 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:26:48.229 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:26:48.233 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:26:48.235 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:26:48.236 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:26:48.243 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:26:48.244 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:26:48.253 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 Processor -- start 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 -- start start 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 0x7f8a8c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c0c85e0 con 0x7f8a8c0cf4a0 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c0c8840 con 0x7f8a8c0cf920 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a90c88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c15ff70 con 0x7f8a8c15d140 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 0x7f8a8c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a8c0cf920 0x7f8a8c160360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42378/0 (socket says 172.21.15.181:42378) 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8bfff700 1 -- 172.21.15.181:0/1431290524 learned_addr learned my addr 172.21.15.181:0/1431290524 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8b7fe700 1 -- 172.21.15.181:0/1431290524 >> v1:172.21.15.181:6789/0 conn(0x7f8a8c0cf920 legacy=0x7f8a8c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8b7fe700 1 -- 172.21.15.181:0/1431290524 >> v1:172.21.15.175:6789/0 conn(0x7f8a8c0cf4a0 legacy=0x7f8a8c064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8b7fe700 1 -- 172.21.15.181:0/1431290524 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0c8ae0 con 0x7f8a8c15d140 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8a7fc700 1 -- 172.21.15.181:0/1431290524 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854488902 0 0) 0x7f8a8c0c8840 con 0x7f8a8c0cf920 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8b7fe700 1 --2- 172.21.15.181:0/1431290524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 0x7f8a8c15d520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8a7c00b9d0 tx=0x7f8a7c006df0).ready entity=mon.2 client_cookie=51709d5eab1632f3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8a7fc700 1 -- 172.21.15.181:0/1431290524 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c009870 con 0x7f8a8c15d140 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8a7fc700 1 -- 172.21.15.181:0/1431290524 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c0099d0 con 0x7f8a8c15d140 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.317+0000 7f8a8a7fc700 1 -- 172.21.15.181:0/1431290524 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c009d50 con 0x7f8a8c15d140 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- 172.21.15.181:0/1431290524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 msgr2=0x7f8a8c15d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 --2- 172.21.15.181:0/1431290524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 0x7f8a8c15d520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8a7c00b9d0 tx=0x7f8a7c006df0).stop 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- 172.21.15.181:0/1431290524 shutdown_connections 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 --2- 172.21.15.181:0/1431290524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c15d140 0x7f8a8c15d520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- 172.21.15.181:0/1431290524 >> 172.21.15.181:0/1431290524 conn(0x7f8a8c0d2dc0 msgr2=0x7f8a8c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- 172.21.15.181:0/1431290524 shutdown_connections 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- 172.21.15.181:0/1431290524 wait complete. 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 Processor -- start 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- start start 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c0cf4a0 0x7f8a8c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 0x7f8a8c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c13e3b0 con 0x7f8a8c0cf920 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0ccc40 con 0x7f8a8c15d140 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a90c88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0ccee0 con 0x7f8a8c0cf4a0 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c0cf4a0 0x7f8a8c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56312/0 (socket says 172.21.15.181:56312) 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 -- 172.21.15.181:0/2596506733 learned_addr learned my addr 172.21.15.181:0/2596506733 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8affd700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 0x7f8a8c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 -- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c0cf4a0 msgr2=0x7f8a8c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c0cf4a0 0x7f8a8c15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 -- 172.21.15.181:0/2596506733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 msgr2=0x7f8a8c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 0x7f8a8c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8bfff700 1 -- 172.21.15.181:0/2596506733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0cd180 con 0x7f8a8c15d140 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.321+0000 7f8a8affd700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 0x7f8a8c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a8bfff700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8a8400cae0 tx=0x7f8a84007ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005b80 con 0x7f8a8c15d140 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a84005ce0 con 0x7f8a8c15d140 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84007360 con 0x7f8a8c15d140 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c0cd420 con 0x7f8a8c15d140 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c0cdb60 con 0x7f8a8c15d140 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c044f60 con 0x7f8a8c15d140 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8a84010030 con 0x7f8a8c15d140 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 0x7f8a68057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a8b7fe700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 0x7f8a68057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8a8406db30 con 0x7f8a8c15d140 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.325+0000 7f8a8b7fe700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 0x7f8a68057520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8a7c00b150 tx=0x7f8a7c017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.333+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8a84043f50 con 0x7f8a8c15d140 2020-11-16T22:26:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.489+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a8c05d550 con 0x7f8a8c15d140 2020-11-16T22:26:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a837fe700 1 -- 172.21.15.181:0/2596506733 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13734 (secure 0 0 0) 0x7f8a84047570 con 0x7f8a8c15d140 2020-11-16T22:26:48.478 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:48.479 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:48.122549+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 msgr2=0x7f8a68057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 0x7f8a68057520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8a7c00b150 tx=0x7f8a7c017000).stop 2020-11-16T22:26:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 msgr2=0x7f8a8c16b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8a8400cae0 tx=0x7f8a84007ce0).stop 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 shutdown_connections 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8a68055060 0x7f8a68057520 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a8c15d140 0x7f8a8c16b1c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a8c0cf4a0 0x7f8a8c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 --2- 172.21.15.181:0/2596506733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c0cf920 0x7f8a8c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 >> 172.21.15.181:0/2596506733 conn(0x7f8a8c0d2dc0 msgr2=0x7f8a8c0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 shutdown_connections 2020-11-16T22:26:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.493+0000 7f8a90c88700 1 -- 172.21.15.181:0/2596506733 wait complete. 2020-11-16T22:26:48.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:26:48.493 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:26:48.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 Processor -- start 2020-11-16T22:26:48.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 -- start start 2020-11-16T22:26:48.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 0x7f6124140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61240cef10 0x7f61240d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61240cf390 con 0x7f61240d07e0 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612415ebb0 con 0x7f61240cef10 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6123fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240cf5d0 con 0x7f6124140550 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f6122ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 0x7f6124140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f61227fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61240cef10 0x7f61240d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f61237fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f61240d07e0 0x7f6124063ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42392/0 (socket says 172.21.15.181:42392) 2020-11-16T22:26:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.637+0000 7f61237fe700 1 -- 172.21.15.181:0/2422726090 learned_addr learned my addr 172.21.15.181:0/2422726090 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f61227fc700 1 -- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 msgr2=0x7f6124140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f61227fc700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 0x7f6124140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f61227fc700 1 -- 172.21.15.181:0/2422726090 >> v1:172.21.15.181:6789/0 conn(0x7f61240d07e0 legacy=0x7f6124063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6121ffb700 1 -- 172.21.15.181:0/2422726090 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421028881 0 0) 0x7f611c001450 con 0x7f61240d07e0 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f61227fc700 1 -- 172.21.15.181:0/2422726090 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61240cf870 con 0x7f61240cef10 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6122ffd700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 0x7f6124140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f61227fc700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61240cef10 0x7f61240d02f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f611800b9a0 tx=0x7f6118006df0).ready entity=mon.1 client_cookie=87f710da3b0cc406 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6121ffb700 1 -- 172.21.15.181:0/2422726090 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180097b0 con 0x7f61240cef10 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6121ffb700 1 -- 172.21.15.181:0/2422726090 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118009910 con 0x7f61240cef10 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6121ffb700 1 -- 172.21.15.181:0/2422726090 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118009c90 con 0x7f61240cef10 2020-11-16T22:26:48.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61240cef10 msgr2=0x7f61240d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61240cef10 0x7f61240d02f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f611800b9a0 tx=0x7f6118006df0).stop 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- 172.21.15.181:0/2422726090 shutdown_connections 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124140550 0x7f6124140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 --2- 172.21.15.181:0/2422726090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61240cef10 0x7f61240d02f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- 172.21.15.181:0/2422726090 >> 172.21.15.181:0/2422726090 conn(0x7f61240d4570 msgr2=0x7f61240d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- 172.21.15.181:0/2422726090 shutdown_connections 2020-11-16T22:26:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- 172.21.15.181:0/2422726090 wait complete. 2020-11-16T22:26:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 Processor -- start 2020-11-16T22:26:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.641+0000 7f6123fff700 1 -- start start 2020-11-16T22:26:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61240d07e0 0x7f612413e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124155480 0x7f61241558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124156900 con 0x7f61240d07e0 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124156120 con 0x7f6124140550 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61241563c0 con 0x7f6124155480 2020-11-16T22:26:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56324/0 (socket says 172.21.15.181:56324) 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 -- 172.21.15.181:0/1150032935 learned_addr learned my addr 172.21.15.181:0/1150032935 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61237fe700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124155480 0x7f61241558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6122ffd700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61240d07e0 0x7f612413e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 -- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124155480 msgr2=0x7f61241558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124155480 0x7f61241558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 -- 172.21.15.181:0/1150032935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61240d07e0 msgr2=0x7f612413e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61240d07e0 0x7f612413e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 -- 172.21.15.181:0/1150032935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6124156660 con 0x7f6124140550 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f61227fc700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6118006e30 tx=0x7f6118015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118011c80 con 0x7f6124140550 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118011de0 con 0x7f6124140550 2020-11-16T22:26:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6124151fa0 con 0x7f6124140550 2020-11-16T22:26:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611801e670 con 0x7f6124140550 2020-11-16T22:26:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6124155db0 con 0x7f6124140550 2020-11-16T22:26:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6124155f40 con 0x7f6124140550 2020-11-16T22:26:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.645+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6118025030 con 0x7f6124140550 2020-11-16T22:26:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.649+0000 7f610affd700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 0x7f60f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.649+0000 7f6122ffd700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 0x7f60f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.649+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f611806ef10 con 0x7f6124140550 2020-11-16T22:26:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.649+0000 7f6122ffd700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 0x7f60f4057520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6114006fd0 tx=0x7f6114008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.653+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6118045550 con 0x7f6124140550 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.809+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6124044f60 con 0x7f6124140550 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.809+0000 7f610affd700 1 -- 172.21.15.181:0/1150032935 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6118023030 con 0x7f6124140550 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 msgr2=0x7f60f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 0x7f60f4057520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6114006fd0 tx=0x7f6114008040).stop 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 msgr2=0x7f612413e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6118006e30 tx=0x7f6118015000).stop 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 shutdown_connections 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f60f4055060 0x7f60f4057520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6124155480 0x7f61241558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61240d07e0 0x7f612413e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 --2- 172.21.15.181:0/1150032935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6124140550 0x7f612413e7f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 >> 172.21.15.181:0/1150032935 conn(0x7f61240d4570 msgr2=0x7f612413d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 shutdown_connections 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.813+0000 7f6123fff700 1 -- 172.21.15.181:0/1150032935 wait complete. 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.953+0000 7fdafe140700 1 Processor -- start 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.953+0000 7fdafe140700 1 -- start start 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdaf8145b50 0x7fdaf80d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf8081f20 con 0x7fdaf8145770 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf8060470 con 0x7fdaf80d2810 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf8145fb0 con 0x7fdaf8145b50 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafc93d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdaf8145b50 0x7fdaf80d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafd93f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdaf80d2810 0x7fdaf81612b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42406/0 (socket says 172.21.15.181:42406) 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafd93f700 1 -- 172.21.15.181:0/2357493804 learned_addr learned my addr 172.21.15.181:0/2357493804 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafc93d700 1 -- 172.21.15.181:0/2357493804 >> v1:172.21.15.181:6789/0 conn(0x7fdaf80d2810 legacy=0x7fdaf81612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdae7fff700 1 -- 172.21.15.181:0/2357493804 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634852486 0 0) 0x7fdaf8060470 con 0x7fdaf80d2810 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafc93d700 1 -- 172.21.15.181:0/2357493804 >> v1:172.21.15.175:6789/0 conn(0x7fdaf8145770 legacy=0x7fdaf8160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafc93d700 1 -- 172.21.15.181:0/2357493804 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8060710 con 0x7fdaf8145b50 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafc93d700 1 --2- 172.21.15.181:0/2357493804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf8145b50 0x7fdaf80d2320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fdaf400b9a0 tx=0x7fdaf4006df0).ready entity=mon.1 client_cookie=861f3e6e41fcaaea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdae7fff700 1 -- 172.21.15.181:0/2357493804 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40097b0 con 0x7fdaf8145b50 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdae7fff700 1 -- 172.21.15.181:0/2357493804 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf4009910 con 0x7fdaf8145b50 2020-11-16T22:26:48.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdae7fff700 1 -- 172.21.15.181:0/2357493804 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4009c90 con 0x7fdaf8145b50 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- 172.21.15.181:0/2357493804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf8145b50 msgr2=0x7fdaf80d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 --2- 172.21.15.181:0/2357493804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf8145b50 0x7fdaf80d2320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fdaf400b9a0 tx=0x7fdaf4006df0).stop 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- 172.21.15.181:0/2357493804 shutdown_connections 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 --2- 172.21.15.181:0/2357493804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf8145b50 0x7fdaf80d2320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- 172.21.15.181:0/2357493804 >> 172.21.15.181:0/2357493804 conn(0x7fdaf80d6760 msgr2=0x7fdaf80d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- 172.21.15.181:0/2357493804 shutdown_connections 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.957+0000 7fdafe140700 1 -- 172.21.15.181:0/2357493804 wait complete. 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 Processor -- start 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 -- start start 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf80d2810 0x7fdaf81725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf8145770 0x7fdaf8166af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8164280 con 0x7fdaf8145770 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8172e80 con 0x7fdaf80d2810 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafe140700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8173120 con 0x7fdaf8145b50 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd13e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf80d2810 0x7fdaf81725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35950/0 (socket says 172.21.15.181:35950) 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 -- 172.21.15.181:0/3936345836 learned_addr learned my addr 172.21.15.181:0/3936345836 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafc93d700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf8145770 0x7fdaf8166af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 -- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf80d2810 msgr2=0x7fdaf81725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf80d2810 0x7fdaf81725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 -- 172.21.15.181:0/3936345836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf8145770 msgr2=0x7fdaf8166af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf8145770 0x7fdaf8166af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 -- 172.21.15.181:0/3936345836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf81733c0 con 0x7fdaf8145b50 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdafd93f700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdaf000cae0 tx=0x7fdaf0007ce0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.961+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005b80 con 0x7fdaf8145b50 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0005ce0 con 0x7fdaf8145b50 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf8164470 con 0x7fdaf8145b50 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0007360 con 0x7fdaf8145b50 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf8172ab0 con 0x7fdaf8145b50 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8044f60 con 0x7fdaf8145b50 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdaf0010030 con 0x7fdaf8145b50 2020-11-16T22:26:48.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdae57fa700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 0x7fdad0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:48.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdaf006d930 con 0x7fdaf8145b50 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdafd13e700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 0x7fdad0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.965+0000 7fdafd13e700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 0x7fdad0057550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdae8003300 tx=0x7fdae8007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:48.973+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdaf0043e60 con 0x7fdaf8145b50 2020-11-16T22:26:49.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.117+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdaf805d400 con 0x7fdaf8145b50 2020-11-16T22:26:49.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdae57fa700 1 -- 172.21.15.181:0/3936345836 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdaf0047480 con 0x7fdaf8145b50 2020-11-16T22:26:49.106 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:26:49.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 msgr2=0x7fdad0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 0x7fdad0057550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdae8003300 tx=0x7fdae8007b00).stop 2020-11-16T22:26:49.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 msgr2=0x7fdaf8166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdaf000cae0 tx=0x7fdaf0007ce0).stop 2020-11-16T22:26:49.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 shutdown_connections 2020-11-16T22:26:49.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdad0055090 0x7fdad0057550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf8145b50 0x7fdaf8166fe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf80d2810 0x7fdaf81725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 --2- 172.21.15.181:0/3936345836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf8145770 0x7fdaf8166af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 >> 172.21.15.181:0/3936345836 conn(0x7fdaf80d6760 msgr2=0x7fdaf80d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 shutdown_connections 2020-11-16T22:26:49.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.121+0000 7fdafe140700 1 -- 172.21.15.181:0/3936345836 wait complete. 2020-11-16T22:26:49.121 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:26:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 Processor -- start 2020-11-16T22:26:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 -- start start 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 0x7f4d94145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d94061b50 con 0x7f4d941492c0 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d940618b0 con 0x7f4d94145a30 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d99773700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940cb930 con 0x7f4d940816e0 2020-11-16T22:26:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d98f72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d93fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 0x7f4d94145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d98f72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35956/0 (socket says 172.21.15.181:35956) 2020-11-16T22:26:49.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d98f72700 1 -- 172.21.15.181:0/1923052336 learned_addr learned my addr 172.21.15.181:0/1923052336 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:49.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.249+0000 7f4d92ffd700 1 -- 172.21.15.181:0/1923052336 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636698717 0 0) 0x7f4d94061b50 con 0x7f4d941492c0 2020-11-16T22:26:49.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d98f72700 1 -- 172.21.15.181:0/1923052336 >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 msgr2=0x7f4d94145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:49.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d98f72700 1 --2- 172.21.15.181:0/1923052336 >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 0x7f4d94145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d98f72700 1 -- 172.21.15.181:0/1923052336 >> v1:172.21.15.175:6789/0 conn(0x7f4d941492c0 legacy=0x7f4d94080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d98f72700 1 -- 172.21.15.181:0/1923052336 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940cbbd0 con 0x7f4d940816e0 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d93fff700 1 --2- 172.21.15.181:0/1923052336 >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 0x7f4d94145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d98f72700 1 --2- 172.21.15.181:0/1923052336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4d8c009d50 tx=0x7f4d8c009290).ready entity=mon.2 client_cookie=658d276a95fa94b2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d92ffd700 1 -- 172.21.15.181:0/1923052336 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00bbc0 con 0x7f4d940816e0 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d92ffd700 1 -- 172.21.15.181:0/1923052336 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00bd20 con 0x7f4d940816e0 2020-11-16T22:26:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d92ffd700 1 -- 172.21.15.181:0/1923052336 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c011610 con 0x7f4d940816e0 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- 172.21.15.181:0/1923052336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 msgr2=0x7f4d940cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 --2- 172.21.15.181:0/1923052336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4d8c009d50 tx=0x7f4d8c009290).stop 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- 172.21.15.181:0/1923052336 shutdown_connections 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 --2- 172.21.15.181:0/1923052336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d940cc5b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 --2- 172.21.15.181:0/1923052336 >> v2:172.21.15.181:3300/0 conn(0x7f4d94145a30 0x7f4d94145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- 172.21.15.181:0/1923052336 >> 172.21.15.181:0/1923052336 conn(0x7f4d940d68c0 msgr2=0x7f4d940d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- 172.21.15.181:0/1923052336 shutdown_connections 2020-11-16T22:26:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- 172.21.15.181:0/1923052336 wait complete. 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 Processor -- start 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.253+0000 7f4d99773700 1 -- start start 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d94145a30 0x7f4d940d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 0x7f4d94165d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940d1920 con 0x7f4d941492c0 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940d1bc0 con 0x7f4d94145a30 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940d1e60 con 0x7f4d940816e0 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35960/0 (socket says 172.21.15.181:35960) 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 -- 172.21.15.181:0/254094799 learned_addr learned my addr 172.21.15.181:0/254094799 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d937fe700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d94145a30 0x7f4d940d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d98f72700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 0x7f4d94165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 -- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d94145a30 msgr2=0x7f4d940d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d94145a30 0x7f4d940d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 -- 172.21.15.181:0/254094799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 msgr2=0x7f4d94165d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 0x7f4d94165d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 -- 172.21.15.181:0/254094799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940d2100 con 0x7f4d940816e0 2020-11-16T22:26:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d98f72700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 0x7f4d94165d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:49.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d93fff700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4d8800ef50 tx=0x7f4d88006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800cb30 con 0x7f4d940816e0 2020-11-16T22:26:49.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800cc90 con 0x7f4d940816e0 2020-11-16T22:26:49.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88012600 con 0x7f4d940816e0 2020-11-16T22:26:49.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d940d23a0 con 0x7f4d940816e0 2020-11-16T22:26:49.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d94166390 con 0x7f4d940816e0 2020-11-16T22:26:49.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.257+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d940d1750 con 0x7f4d940816e0 2020-11-16T22:26:49.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.261+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4d88012970 con 0x7f4d940816e0 2020-11-16T22:26:49.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.261+0000 7f4d73fff700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 0x7f4d6c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.261+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4d8806ecb0 con 0x7f4d940816e0 2020-11-16T22:26:49.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.261+0000 7f4d937fe700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 0x7f4d6c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.261+0000 7f4d937fe700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 0x7f4d6c057520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4d940bcab0 tx=0x7f4d840079b0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.265+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4d88045130 con 0x7f4d940816e0 2020-11-16T22:26:49.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.421+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4d94044f60 con 0x7f4d940816e0 2020-11-16T22:26:49.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.421+0000 7f4d73fff700 1 -- 172.21.15.181:0/254094799 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4d88048750 con 0x7f4d940816e0 2020-11-16T22:26:49.409 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:49.409 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:26:49.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:26:49.409 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:26:49.409 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:26:49.410 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:26:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 msgr2=0x7f4d6c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 0x7f4d6c057520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4d940bcab0 tx=0x7f4d840079b0).stop 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 msgr2=0x7f4d94166890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4d8800ef50 tx=0x7f4d88006df0).stop 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 shutdown_connections 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d6c055060 0x7f4d6c057520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d941492c0 0x7f4d94165d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d940816e0 0x7f4d94166890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 --2- 172.21.15.181:0/254094799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d94145a30 0x7f4d940d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 >> 172.21.15.181:0/254094799 conn(0x7f4d940d68c0 msgr2=0x7f4d941410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:49.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 shutdown_connections 2020-11-16T22:26:49.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.425+0000 7f4d99773700 1 -- 172.21.15.181:0/254094799 wait complete. 2020-11-16T22:26:49.423 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:26:49.424 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 Processor -- start 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 -- start start 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ea015de80 0x7f6ea00b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 0x7f6ea0160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea015e2e0 con 0x7f6ea00b2190 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea00b2a60 con 0x7f6ea015ff80 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea00ce460 con 0x7f6ea015de80 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ea015de80 0x7f6ea00b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea485f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6ea00b2190 0x7f6ea00cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42434/0 (socket says 172.21.15.181:42434) 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ea015de80 0x7f6ea00b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56364/0 (socket says 172.21.15.181:56364) 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea485f700 1 -- 172.21.15.181:0/1525041092 learned_addr learned my addr 172.21.15.181:0/1525041092 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5060700 1 --2- 172.21.15.181:0/1525041092 >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 0x7f6ea0160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 -- 172.21.15.181:0/1525041092 >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 msgr2=0x7f6ea0160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 --2- 172.21.15.181:0/1525041092 >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 0x7f6ea0160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 -- 172.21.15.181:0/1525041092 >> v1:172.21.15.181:6789/0 conn(0x7f6ea00b2190 legacy=0x7f6ea00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9ffff700 1 -- 172.21.15.181:0/1525041092 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea00ce700 con 0x7f6ea015de80 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6e9f7fe700 1 -- 172.21.15.181:0/1525041092 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960614058 0 0) 0x7f6ea015e2e0 con 0x7f6ea00b2190 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.565+0000 7f6ea5060700 1 --2- 172.21.15.181:0/1525041092 >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 0x7f6ea0160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6e9ffff700 1 --2- 172.21.15.181:0/1525041092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea015de80 0x7f6ea00b2570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6ea0064d90 tx=0x7f6e900091d0).ready entity=mon.1 client_cookie=d89d15b2ff864d6c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6e9f7fe700 1 -- 172.21.15.181:0/1525041092 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000bbc0 con 0x7f6ea015de80 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6e9f7fe700 1 -- 172.21.15.181:0/1525041092 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9000bd20 con 0x7f6ea015de80 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6e9f7fe700 1 -- 172.21.15.181:0/1525041092 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90011610 con 0x7f6ea015de80 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- 172.21.15.181:0/1525041092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea015de80 msgr2=0x7f6ea00b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 --2- 172.21.15.181:0/1525041092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea015de80 0x7f6ea00b2570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6ea0064d90 tx=0x7f6e900091d0).stop 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- 172.21.15.181:0/1525041092 shutdown_connections 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 --2- 172.21.15.181:0/1525041092 >> v2:172.21.15.175:3300/0 conn(0x7f6ea015ff80 0x7f6ea0160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 --2- 172.21.15.181:0/1525041092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea015de80 0x7f6ea00b2570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- 172.21.15.181:0/1525041092 >> 172.21.15.181:0/1525041092 conn(0x7f6ea00cfe90 msgr2=0x7f6ea00d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- 172.21.15.181:0/1525041092 shutdown_connections 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- 172.21.15.181:0/1525041092 wait complete. 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 Processor -- start 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.569+0000 7f6ea5861700 1 -- start start 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea00b2190 0x7f6ea015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 0x7f6ea01709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015e7e0 con 0x7f6ea015ff80 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015ea80 con 0x7f6ea00b2190 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015ed20 con 0x7f6ea015de80 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea485f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea00b2190 0x7f6ea015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35980/0 (socket says 172.21.15.181:35980) 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 -- 172.21.15.181:0/365318475 learned_addr learned my addr 172.21.15.181:0/365318475 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5060700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 0x7f6ea01709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 -- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea00b2190 msgr2=0x7f6ea015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea00b2190 0x7f6ea015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 -- 172.21.15.181:0/365318475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 msgr2=0x7f6ea01709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 0x7f6ea01709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 -- 172.21.15.181:0/365318475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea015efc0 con 0x7f6ea015de80 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5060700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 0x7f6ea01709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9ffff700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6e90009fd0 tx=0x7f6e90016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000be90 con 0x7f6ea015de80 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9001c5f0 con 0x7f6ea015de80 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea015f260 con 0x7f6ea015de80 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9001c990 con 0x7f6ea015de80 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea015e4a0 con 0x7f6ea015de80 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.573+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0044f60 con 0x7f6ea015de80 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.577+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6e90017070 con 0x7f6ea015de80 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.577+0000 7f6e9cff9700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 0x7f6e70057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.577+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6e9006fd50 con 0x7f6ea015de80 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.577+0000 7f6ea485f700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 0x7f6e70057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.577+0000 7f6ea485f700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 0x7f6e70057520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6e98007160 tx=0x7f6e98007960).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.581+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6e900462e0 con 0x7f6ea015de80 2020-11-16T22:26:49.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:49.737+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6ea015d540 con 0x7f6ea015de80 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.625+0000 7f6e9cff9700 1 -- 172.21.15.181:0/365318475 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6e90015070 con 0x7f6ea015de80 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 msgr2=0x7f6e70057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 0x7f6e70057520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6e98007160 tx=0x7f6e98007960).stop 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 msgr2=0x7f6ea0061bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6e90009fd0 tx=0x7f6e90016000).stop 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 shutdown_connections 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e70055060 0x7f6e70057520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea015ff80 0x7f6ea01709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea00b2190 0x7f6ea015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 --2- 172.21.15.181:0/365318475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ea015de80 0x7f6ea0061bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 >> 172.21.15.181:0/365318475 conn(0x7f6ea00cfe90 msgr2=0x7f6ea00600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 shutdown_connections 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.629+0000 7f6ea5861700 1 -- 172.21.15.181:0/365318475 wait complete. 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:26:50.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.777+0000 7fd501168700 1 Processor -- start 2020-11-16T22:26:50.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.777+0000 7fd501168700 1 -- start start 2020-11-16T22:26:50.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:50.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc062b70 con 0x7fd4fc066420 2020-11-16T22:26:50.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc061b70 con 0x7fd4fc065fa0 2020-11-16T22:26:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc15db10 con 0x7fd4fc0612a0 2020-11-16T22:26:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fa59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd4fc065fa0 0x7fd4fc15f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42450/0 (socket says 172.21.15.181:42450) 2020-11-16T22:26:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fa59c700 1 -- 172.21.15.181:0/3982771412 learned_addr learned my addr 172.21.15.181:0/3982771412 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:50.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4f9d9b700 1 -- 172.21.15.181:0/3982771412 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241396210 0 0) 0x7fd4fc061b70 con 0x7fd4fc065fa0 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4f9d9b700 1 -- 172.21.15.181:0/3982771412 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e8003240 con 0x7fd4fc065fa0 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3982771412 >> v1:172.21.15.181:6789/0 conn(0x7fd4fc065fa0 legacy=0x7fd4fc15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3982771412 >> v1:172.21.15.175:6789/0 conn(0x7fd4fc066420 legacy=0x7fd4fc0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3982771412 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0cd790 con 0x7fd4fc0612a0 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4fad9d700 1 --2- 172.21.15.181:0/3982771412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc061680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd4f4009cc0 tx=0x7fd4f40091d0).ready entity=mon.2 client_cookie=7a23dbbc27d0bf85 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4f9d9b700 1 -- 172.21.15.181:0/3982771412 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400bad0 con 0x7fd4fc0612a0 2020-11-16T22:26:50.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4f9d9b700 1 -- 172.21.15.181:0/3982771412 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f400bc30 con 0x7fd4fc0612a0 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd4f9d9b700 1 -- 172.21.15.181:0/3982771412 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4011400 con 0x7fd4fc0612a0 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- 172.21.15.181:0/3982771412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 msgr2=0x7fd4fc061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 --2- 172.21.15.181:0/3982771412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc061680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd4f4009cc0 tx=0x7fd4f40091d0).stop 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- 172.21.15.181:0/3982771412 shutdown_connections 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 --2- 172.21.15.181:0/3982771412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc061680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- 172.21.15.181:0/3982771412 >> 172.21.15.181:0/3982771412 conn(0x7fd4fc0cf890 msgr2=0x7fd4fc0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:50.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- 172.21.15.181:0/3982771412 shutdown_connections 2020-11-16T22:26:50.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.781+0000 7fd501168700 1 -- 172.21.15.181:0/3982771412 wait complete. 2020-11-16T22:26:50.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 Processor -- start 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- start start 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4fc066420 0x7fd4fc1626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc163290 con 0x7fd4fc066420 2020-11-16T22:26:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc163530 con 0x7fd4fc065fa0 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc157510 con 0x7fd4fc0612a0 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:35994/0 (socket says 172.21.15.181:35994) 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56384/0 (socket says 172.21.15.181:56384) 2020-11-16T22:26:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3563421823 learned_addr learned my addr 172.21.15.181:0/3563421823 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fb59e700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4fc066420 0x7fd4fc1626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 msgr2=0x7fd4fc1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3563421823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4fc066420 msgr2=0x7fd4fc1626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4fc066420 0x7fd4fc1626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 -- 172.21.15.181:0/3563421823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc1577b0 con 0x7fd4fc0612a0 2020-11-16T22:26:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fa59c700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4fad9d700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd4f400b7d0 tx=0x7fd4f4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4021070 con 0x7fd4fc0612a0 2020-11-16T22:26:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc157a50 con 0x7fd4fc0612a0 2020-11-16T22:26:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4011e20 con 0x7fd4fc0612a0 2020-11-16T22:26:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f401c820 con 0x7fd4fc0612a0 2020-11-16T22:26:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc162bb0 con 0x7fd4fc0612a0 2020-11-16T22:26:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.785+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc069eb0 con 0x7fd4fc0612a0 2020-11-16T22:26:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.789+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd4f4017070 con 0x7fd4fc0612a0 2020-11-16T22:26:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.789+0000 7fd4e2ffd700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 0x7fd4d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.789+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fd4f4070730 con 0x7fd4fc0612a0 2020-11-16T22:26:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.789+0000 7fd4fa59c700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 0x7fd4d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.789+0000 7fd4fa59c700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 0x7fd4d8057520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd4fc061cd0 tx=0x7fd4f0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.797+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd4f404a590 con 0x7fd4fc0612a0 2020-11-16T22:26:50.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:50.949+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd4fc05d4e0 con 0x7fd4fc0612a0 2020-11-16T22:26:51.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.633+0000 7fd4e2ffd700 1 -- 172.21.15.181:0/3563421823 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v21) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd4f4015070 con 0x7fd4fc0612a0 2020-11-16T22:26:51.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 msgr2=0x7fd4d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 0x7fd4d8057520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd4fc061cd0 tx=0x7fd4f0009040).stop 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 msgr2=0x7fd4fc153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd4f400b7d0 tx=0x7fd4f4016000).stop 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 shutdown_connections 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd4d8055060 0x7fd4d8057520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4fc066420 0x7fd4fc1626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0612a0 0x7fd4fc153bc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 --2- 172.21.15.181:0/3563421823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc065fa0 0x7fd4fc1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 >> 172.21.15.181:0/3563421823 conn(0x7fd4fc0cf890 msgr2=0x7fd4fc064970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:51.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 shutdown_connections 2020-11-16T22:26:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.637+0000 7fd501168700 1 -- 172.21.15.181:0/3563421823 wait complete. 2020-11-16T22:26:51.629 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:26:51.640 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:26:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.777+0000 7fba2a217700 1 Processor -- start 2020-11-16T22:26:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.777+0000 7fba2a217700 1 -- start start 2020-11-16T22:26:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba2415f390 0x7fba24062b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240c94d0 con 0x7fba240cee70 2020-11-16T22:26:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240c9770 con 0x7fba240b6150 2020-11-16T22:26:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240ca690 con 0x7fba2415f390 2020-11-16T22:26:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29a16700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fba240b6150 0x7fba240b65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42464/0 (socket says 172.21.15.181:42464) 2020-11-16T22:26:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29a16700 1 -- 172.21.15.181:0/4212881593 learned_addr learned my addr 172.21.15.181:0/4212881593 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29215700 1 --2- 172.21.15.181:0/4212881593 >> v2:172.21.15.175:3300/0 conn(0x7fba2415f390 0x7fba24062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba1bfff700 1 -- 172.21.15.181:0/4212881593 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524748058 0 0) 0x7fba240c9770 con 0x7fba240b6150 2020-11-16T22:26:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba1bfff700 1 -- 172.21.15.181:0/4212881593 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba08003240 con 0x7fba240b6150 2020-11-16T22:26:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29215700 1 -- 172.21.15.181:0/4212881593 >> v1:172.21.15.181:6789/0 conn(0x7fba240b6150 legacy=0x7fba240b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29215700 1 -- 172.21.15.181:0/4212881593 >> v1:172.21.15.175:6789/0 conn(0x7fba240cee70 legacy=0x7fba240cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29215700 1 -- 172.21.15.181:0/4212881593 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240c9a10 con 0x7fba2415f390 2020-11-16T22:26:51.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba29215700 1 --2- 172.21.15.181:0/4212881593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba24062b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fba2000b9a0 tx=0x7fba20006df0).ready entity=mon.0 client_cookie=39ce8ce0e7cc9aa2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:51.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba1bfff700 1 -- 172.21.15.181:0/4212881593 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200097b0 con 0x7fba2415f390 2020-11-16T22:26:51.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba1bfff700 1 -- 172.21.15.181:0/4212881593 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20009910 con 0x7fba2415f390 2020-11-16T22:26:51.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba1bfff700 1 -- 172.21.15.181:0/4212881593 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20009c90 con 0x7fba2415f390 2020-11-16T22:26:51.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- 172.21.15.181:0/4212881593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 msgr2=0x7fba24062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 --2- 172.21.15.181:0/4212881593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba24062b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fba2000b9a0 tx=0x7fba20006df0).stop 2020-11-16T22:26:51.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- 172.21.15.181:0/4212881593 shutdown_connections 2020-11-16T22:26:51.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 --2- 172.21.15.181:0/4212881593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba24062b80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.781+0000 7fba2a217700 1 -- 172.21.15.181:0/4212881593 >> 172.21.15.181:0/4212881593 conn(0x7fba240d44d0 msgr2=0x7fba240d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- 172.21.15.181:0/4212881593 shutdown_connections 2020-11-16T22:26:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- 172.21.15.181:0/4212881593 wait complete. 2020-11-16T22:26:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 Processor -- start 2020-11-16T22:26:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- start start 2020-11-16T22:26:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba240b6150 0x7fba240b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba240cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba24170840 con 0x7fba2415f390 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba24170ae0 con 0x7fba240cee70 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba2a217700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba24170d80 con 0x7fba240b6150 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba28a14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba28a14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56396/0 (socket says 172.21.15.181:56396) 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba28a14700 1 -- 172.21.15.181:0/2537195832 learned_addr learned my addr 172.21.15.181:0/2537195832 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba29a16700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba240cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.785+0000 7fba29215700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba240b6150 0x7fba240b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 -- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba240b6150 msgr2=0x7fba240b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba240b6150 0x7fba240b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 -- 172.21.15.181:0/2537195832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 msgr2=0x7fba240cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba240cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 -- 172.21.15.181:0/2537195832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24171020 con 0x7fba240cee70 2020-11-16T22:26:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba28a14700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fba1400c710 tx=0x7fba14007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e490 con 0x7fba240cee70 2020-11-16T22:26:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400e5f0 con 0x7fba240cee70 2020-11-16T22:26:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba241712c0 con 0x7fba240cee70 2020-11-16T22:26:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e990 con 0x7fba240cee70 2020-11-16T22:26:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba240b4b00 con 0x7fba240cee70 2020-11-16T22:26:51.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fba14019030 con 0x7fba240cee70 2020-11-16T22:26:51.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba240c9bf0 con 0x7fba240cee70 2020-11-16T22:26:51.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 0x7fb9fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:51.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba29215700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 0x7fb9fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:51.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fba1406cff0 con 0x7fba240cee70 2020-11-16T22:26:51.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.789+0000 7fba29215700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 0x7fb9fc057520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fba20006e30 tx=0x7fba20015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:51.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.793+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fba14043370 con 0x7fba240cee70 2020-11-16T22:26:51.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:51.993+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fba24044f60 con 0x7fba240cee70 2020-11-16T22:26:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.005+0000 7fba197fa700 1 -- 172.21.15.181:0/2537195832 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fba14017030 con 0x7fba240cee70 2020-11-16T22:26:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 msgr2=0x7fb9fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 0x7fb9fc057520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fba20006e30 tx=0x7fba20015000).stop 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 msgr2=0x7fba240cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fba1400c710 tx=0x7fba14007c10).stop 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 shutdown_connections 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb9fc055060 0x7fb9fc057520 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba2415f390 0x7fba240cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba240b6150 0x7fba240b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 --2- 172.21.15.181:0/2537195832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba240cee70 0x7fba240cb970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:51.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 >> 172.21.15.181:0/2537195832 conn(0x7fba240d44d0 msgr2=0x7fba2415cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:51.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 shutdown_connections 2020-11-16T22:26:51.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.009+0000 7fba2a217700 1 -- 172.21.15.181:0/2537195832 wait complete. 2020-11-16T22:26:51.998 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 3 and data pool 4 2020-11-16T22:26:52.013 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:52.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 Processor -- start 2020-11-16T22:26:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- start start 2020-11-16T22:26:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a680b2190 0x7f6a6815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a680ce460 con 0x7f6a680b2570 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a680ce700 con 0x7f6a680b2190 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680ce9a0 con 0x7f6a6815ff80 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36018/0 (socket says 172.21.15.181:36018) 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 -- 172.21.15.181:0/3147948595 learned_addr learned my addr 172.21.15.181:0/3147948595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a67fff700 1 --2- 172.21.15.181:0/3147948595 >> v2:172.21.15.181:3300/0 conn(0x7f6a680b2190 0x7f6a6815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 -- 172.21.15.181:0/3147948595 >> v2:172.21.15.181:3300/0 conn(0x7f6a680b2190 msgr2=0x7f6a6815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 --2- 172.21.15.181:0/3147948595 >> v2:172.21.15.181:3300/0 conn(0x7f6a680b2190 0x7f6a6815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 -- 172.21.15.181:0/3147948595 >> v1:172.21.15.175:6789/0 conn(0x7f6a680b2570 legacy=0x7f6a680cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 -- 172.21.15.181:0/3147948595 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680cec40 con 0x7f6a6815ff80 2020-11-16T22:26:52.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6ccc5700 1 --2- 172.21.15.181:0/3147948595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6a60009d50 tx=0x7f6a60009290).ready entity=mon.2 client_cookie=7c9d4b56030690ed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a66ffd700 1 -- 172.21.15.181:0/3147948595 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000bbc0 con 0x7f6a6815ff80 2020-11-16T22:26:52.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a66ffd700 1 -- 172.21.15.181:0/3147948595 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6000bd20 con 0x7f6a6815ff80 2020-11-16T22:26:52.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a66ffd700 1 -- 172.21.15.181:0/3147948595 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011610 con 0x7f6a6815ff80 2020-11-16T22:26:52.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3147948595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 msgr2=0x7f6a681603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3147948595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6a60009d50 tx=0x7f6a60009290).stop 2020-11-16T22:26:52.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3147948595 shutdown_connections 2020-11-16T22:26:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3147948595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a681603f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3147948595 >> v2:172.21.15.181:3300/0 conn(0x7f6a680b2190 0x7f6a6815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3147948595 >> 172.21.15.181:0/3147948595 conn(0x7f6a680cfe90 msgr2=0x7f6a680d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3147948595 shutdown_connections 2020-11-16T22:26:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.141+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3147948595 wait complete. 2020-11-16T22:26:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 Processor -- start 2020-11-16T22:26:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 -- start start 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a680b2570 0x7f6a68163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a68163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68170620 con 0x7f6a680b2570 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681515c0 con 0x7f6a680b2190 2020-11-16T22:26:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6d4c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68151800 con 0x7f6a6815ff80 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a6ccc5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a68163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.145+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56410/0 (socket says 172.21.15.181:56410) 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 -- 172.21.15.181:0/3464384883 learned_addr learned my addr 172.21.15.181:0/3464384883 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a677fe700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a680b2570 0x7f6a68163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 -- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 msgr2=0x7f6a68163cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a68163cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 -- 172.21.15.181:0/3464384883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a680b2570 msgr2=0x7f6a68163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a680b2570 0x7f6a68163800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 -- 172.21.15.181:0/3464384883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68151aa0 con 0x7f6a680b2190 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a6ccc5700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a68163cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a67fff700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6a5800eba0 tx=0x7f6a58006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800b3a0 con 0x7f6a680b2190 2020-11-16T22:26:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800c8e0 con 0x7f6a680b2190 2020-11-16T22:26:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800cca0 con 0x7f6a680b2190 2020-11-16T22:26:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a68151d40 con 0x7f6a680b2190 2020-11-16T22:26:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a68152280 con 0x7f6a680b2190 2020-11-16T22:26:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68044f60 con 0x7f6a680b2190 2020-11-16T22:26:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6a58012600 con 0x7f6a680b2190 2020-11-16T22:26:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a5806e0b0 con 0x7f6a680b2190 2020-11-16T22:26:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a677fe700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.149+0000 7f6a677fe700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6a68150e90 tx=0x7f6a5c007ac0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.157+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6a580442b0 con 0x7f6a680b2190 2020-11-16T22:26:52.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.313+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a680cfb10 con 0x7f6a680b2190 2020-11-16T22:26:52.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.313+0000 7f6a47fff700 1 -- 172.21.15.181:0/3464384883 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f6a5800ce00 con 0x7f6a680b2190 2020-11-16T22:26:52.300 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:52.300 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:51.998163+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 msgr2=0x7f6a40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6a68150e90 tx=0x7f6a5c007ac0).stop 2020-11-16T22:26:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 msgr2=0x7f6a681510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6a5800eba0 tx=0x7f6a58006df0).stop 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 shutdown_connections 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6815ff80 0x7f6a68163cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a680b2190 0x7f6a681510d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 --2- 172.21.15.181:0/3464384883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a680b2570 0x7f6a68163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 >> 172.21.15.181:0/3464384883 conn(0x7f6a680cfe90 msgr2=0x7f6a68160390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 shutdown_connections 2020-11-16T22:26:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.317+0000 7f6a6d4c6700 1 -- 172.21.15.181:0/3464384883 wait complete. 2020-11-16T22:26:52.315 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 Processor -- start 2020-11-16T22:26:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- start start 2020-11-16T22:26:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bb8145b50 0x7f5bb80d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb80d2810 0x7f5bb81612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb8081f20 con 0x7f5bb8145770 2020-11-16T22:26:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb8145fb0 con 0x7f5bb8145b50 2020-11-16T22:26:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb8060470 con 0x7f5bb80d2810 2020-11-16T22:26:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb80d2810 0x7f5bb81612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb80d2810 0x7f5bb81612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56422/0 (socket says 172.21.15.181:56422) 2020-11-16T22:26:52.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 -- 172.21.15.181:0/2136522632 learned_addr learned my addr 172.21.15.181:0/2136522632 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bb7fff700 1 --2- 172.21.15.181:0/2136522632 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8145b50 0x7f5bb80d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 -- 172.21.15.181:0/2136522632 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8145b50 msgr2=0x7f5bb80d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 --2- 172.21.15.181:0/2136522632 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8145b50 0x7f5bb80d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 -- 172.21.15.181:0/2136522632 >> v1:172.21.15.175:6789/0 conn(0x7f5bb8145770 legacy=0x7f5bb8160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 -- 172.21.15.181:0/2136522632 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8060710 con 0x7f5bb80d2810 2020-11-16T22:26:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbd46d700 1 --2- 172.21.15.181:0/2136522632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb80d2810 0x7f5bb81612b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5bb8062ae0 tx=0x7f5ba0006eb0).ready entity=mon.1 client_cookie=ce6e26e609c4e54 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bb77fe700 1 -- 172.21.15.181:0/2136522632 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0009810 con 0x7f5bb80d2810 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bb77fe700 1 -- 172.21.15.181:0/2136522632 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0009970 con 0x7f5bb80d2810 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bb77fe700 1 -- 172.21.15.181:0/2136522632 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0009cf0 con 0x7f5bb80d2810 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/2136522632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb80d2810 msgr2=0x7f5bb81612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/2136522632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb80d2810 0x7f5bb81612b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5bb8062ae0 tx=0x7f5ba0006eb0).stop 2020-11-16T22:26:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/2136522632 shutdown_connections 2020-11-16T22:26:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/2136522632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb80d2810 0x7f5bb81612b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/2136522632 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8145b50 0x7f5bb80d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/2136522632 >> 172.21.15.181:0/2136522632 conn(0x7f5bb80d6760 msgr2=0x7f5bb80d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/2136522632 shutdown_connections 2020-11-16T22:26:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.445+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/2136522632 wait complete. 2020-11-16T22:26:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 Processor -- start 2020-11-16T22:26:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- start start 2020-11-16T22:26:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 0x7f5bb815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 0x7f5bb813e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8145b50 0x7f5bb815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8170080 con 0x7f5bb8145b50 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb816f100 con 0x7f5bb8145770 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb816f360 con 0x7f5bb80d2810 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbcc6c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 0x7f5bb815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbcc6c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 0x7f5bb815cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36034/0 (socket says 172.21.15.181:36034) 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbcc6c700 1 -- 172.21.15.181:0/993263680 learned_addr learned my addr 172.21.15.181:0/993263680 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:52.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 0x7f5bb813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbd46d700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8145b50 0x7f5bb815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 -- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 msgr2=0x7f5bb815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 0x7f5bb815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 -- 172.21.15.181:0/993263680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8145b50 msgr2=0x7f5bb815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8145b50 0x7f5bb815b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 -- 172.21.15.181:0/993263680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb816f600 con 0x7f5bb8145770 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb7fff700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 0x7f5bb813e790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5bb000eba0 tx=0x7f5bb0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000c8e0 con 0x7f5bb8145770 2020-11-16T22:26:52.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb000ca40 con 0x7f5bb8145770 2020-11-16T22:26:52.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000cdc0 con 0x7f5bb8145770 2020-11-16T22:26:52.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb816f8a0 con 0x7f5bb8145770 2020-11-16T22:26:52.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.449+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb816fde0 con 0x7f5bb8145770 2020-11-16T22:26:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb815d4b0 con 0x7f5bb8145770 2020-11-16T22:26:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5bb0012600 con 0x7f5bb8145770 2020-11-16T22:26:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bb4ff9700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bb006e040 con 0x7f5bb8145770 2020-11-16T22:26:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bbcc6c700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.453+0000 7f5bbcc6c700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5ba8007fd0 tx=0x7f5ba8007f40).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.461+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5bb0044240 con 0x7f5bb8145770 2020-11-16T22:26:52.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bb8044f60 con 0x7f5bb8145770 2020-11-16T22:26:52.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bb4ff9700 1 -- 172.21.15.181:0/993263680 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f5bb0047860 con 0x7f5bb8145770 2020-11-16T22:26:52.600 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:52.600 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:51.998163+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:52.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 msgr2=0x7f5b90057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5ba8007fd0 tx=0x7f5ba8007f40).stop 2020-11-16T22:26:52.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 msgr2=0x7f5bb813e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.613+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 0x7f5bb813e790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5bb000eba0 tx=0x7f5bb0006df0).stop 2020-11-16T22:26:52.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 shutdown_connections 2020-11-16T22:26:52.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8145b50 0x7f5bb815b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb80d2810 0x7f5bb815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 --2- 172.21.15.181:0/993263680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8145770 0x7f5bb813e790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 >> 172.21.15.181:0/993263680 conn(0x7f5bb80d6760 msgr2=0x7f5bb80d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 shutdown_connections 2020-11-16T22:26:52.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.617+0000 7f5bbdc6e700 1 -- 172.21.15.181:0/993263680 wait complete. 2020-11-16T22:26:52.622 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:26:52.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.761+0000 7f472d9ee700 1 Processor -- start 2020-11-16T22:26:52.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.761+0000 7f472d9ee700 1 -- start start 2020-11-16T22:26:52.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.761+0000 7f472d9ee700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4728145a30 0x7f4728145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4728061b50 con 0x7f47281492c0 2020-11-16T22:26:52.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47280618b0 con 0x7f4728145a30 2020-11-16T22:26:52.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280cb930 con 0x7f47280816e0 2020-11-16T22:26:52.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f4727fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f47281492c0 0x7f4728080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42508/0 (socket says 172.21.15.181:42508) 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36044/0 (socket says 172.21.15.181:36044) 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f4727fff700 1 -- 172.21.15.181:0/20159769 learned_addr learned my addr 172.21.15.181:0/20159769 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472c9ec700 1 --2- 172.21.15.181:0/20159769 >> v2:172.21.15.175:3300/0 conn(0x7f4728145a30 0x7f4728145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 -- 172.21.15.181:0/20159769 >> v2:172.21.15.175:3300/0 conn(0x7f4728145a30 msgr2=0x7f4728145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 --2- 172.21.15.181:0/20159769 >> v2:172.21.15.175:3300/0 conn(0x7f4728145a30 0x7f4728145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 -- 172.21.15.181:0/20159769 >> v1:172.21.15.181:6789/0 conn(0x7f47281492c0 legacy=0x7f4728080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f47277fe700 1 -- 172.21.15.181:0/20159769 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32070836 0 0) 0x7f4728061b50 con 0x7f47281492c0 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 -- 172.21.15.181:0/20159769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280cbbd0 con 0x7f47280816e0 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d1ed700 1 --2- 172.21.15.181:0/20159769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f471800b9a0 tx=0x7f4718006df0).ready entity=mon.2 client_cookie=b9de25556754570f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f47277fe700 1 -- 172.21.15.181:0/20159769 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180097b0 con 0x7f47280816e0 2020-11-16T22:26:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f47277fe700 1 -- 172.21.15.181:0/20159769 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718009910 con 0x7f47280816e0 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f47277fe700 1 -- 172.21.15.181:0/20159769 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718009c90 con 0x7f47280816e0 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- 172.21.15.181:0/20159769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 msgr2=0x7f47280cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 --2- 172.21.15.181:0/20159769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f471800b9a0 tx=0x7f4718006df0).stop 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- 172.21.15.181:0/20159769 shutdown_connections 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 --2- 172.21.15.181:0/20159769 >> v2:172.21.15.175:3300/0 conn(0x7f4728145a30 0x7f4728145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 --2- 172.21.15.181:0/20159769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cc5b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- 172.21.15.181:0/20159769 >> 172.21.15.181:0/20159769 conn(0x7f47280d68c0 msgr2=0x7f47280d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- 172.21.15.181:0/20159769 shutdown_connections 2020-11-16T22:26:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.765+0000 7f472d9ee700 1 -- 172.21.15.181:0/20159769 wait complete. 2020-11-16T22:26:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 Processor -- start 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- start start 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 0x7f472813def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 0x7f472813e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280d0c60 con 0x7f4728145a30 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280cff40 con 0x7f47281492c0 2020-11-16T22:26:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280d01e0 con 0x7f47280816e0 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472c9ec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 0x7f472813e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472c9ec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36048/0 (socket says 172.21.15.181:36048) 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472c9ec700 1 -- 172.21.15.181:0/4139818591 learned_addr learned my addr 172.21.15.181:0/4139818591 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f4727fff700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 0x7f472813def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 -- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 msgr2=0x7f47280cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 -- 172.21.15.181:0/4139818591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 msgr2=0x7f472813def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 0x7f472813def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 -- 172.21.15.181:0/4139818591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280d0480 con 0x7f47281492c0 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f4727fff700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 0x7f472813def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d1ed700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 0x7f472813e3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4718012000 tx=0x7f4718015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471801e460 con 0x7f47281492c0 2020-11-16T22:26:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471801e5c0 con 0x7f47281492c0 2020-11-16T22:26:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280d0720 con 0x7f47281492c0 2020-11-16T22:26:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471801e940 con 0x7f47281492c0 2020-11-16T22:26:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.769+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47280cfa80 con 0x7f47281492c0 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47281659f0 con 0x7f47281492c0 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4718025030 con 0x7f47281492c0 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f4724ff9700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 0x7f4700057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f471806f6f0 con 0x7f47281492c0 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f472c9ec700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 0x7f4700057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.773+0000 7f472c9ec700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 0x7f4700057520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f47280d43f0 tx=0x7f47200091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.777+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4718045950 con 0x7f47281492c0 2020-11-16T22:26:52.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:52.981+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4728044f60 con 0x7f47281492c0 2020-11-16T22:26:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.005+0000 7f4724ff9700 1 -- 172.21.15.181:0/4139818591 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4718023020 con 0x7f47281492c0 2020-11-16T22:26:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.009+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 msgr2=0x7f4700057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.009+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 0x7f4700057520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f47280d43f0 tx=0x7f47200091d0).stop 2020-11-16T22:26:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.009+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 msgr2=0x7f472813e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.009+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 0x7f472813e3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4718012000 tx=0x7f4718015000).stop 2020-11-16T22:26:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 shutdown_connections 2020-11-16T22:26:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4700055060 0x7f4700057520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47281492c0 0x7f472813e3e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47280816e0 0x7f47280cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 --2- 172.21.15.181:0/4139818591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4728145a30 0x7f472813def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:52.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 >> 172.21.15.181:0/4139818591 conn(0x7f47280d68c0 msgr2=0x7f4728080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:52.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 shutdown_connections 2020-11-16T22:26:52.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.013+0000 7f472d9ee700 1 -- 172.21.15.181:0/4139818591 wait complete. 2020-11-16T22:26:53.013 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 9 2020-11-16T22:26:53.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 Processor -- start 2020-11-16T22:26:53.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 -- start start 2020-11-16T22:26:53.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f68c015f5b0 0x7f68c00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c0160e40 con 0x7f68c015f5b0 2020-11-16T22:26:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c0160ba0 con 0x7f68c00615f0 2020-11-16T22:26:53.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68c511e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0060970 con 0x7f68c00622a0 2020-11-16T22:26:53.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68be59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.153+0000 7f68bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68be59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56452/0 (socket says 172.21.15.181:56452) 2020-11-16T22:26:53.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68be59c700 1 -- 172.21.15.181:0/1896317426 learned_addr learned my addr 172.21.15.181:0/1896317426 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:53.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bed9d700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.175:3300/0 conn(0x7f68c015f5b0 0x7f68c00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 -- 172.21.15.181:0/1896317426 >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 msgr2=0x7f68c00d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 -- 172.21.15.181:0/1896317426 >> v2:172.21.15.175:3300/0 conn(0x7f68c015f5b0 msgr2=0x7f68c00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.175:3300/0 conn(0x7f68c015f5b0 0x7f68c00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 -- 172.21.15.181:0/1896317426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0060c10 con 0x7f68c00622a0 2020-11-16T22:26:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68be59c700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bf59e700 1 --2- 172.21.15.181:0/1896317426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0062700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f68c00bcb90 tx=0x7f68b80091d0).ready entity=mon.2 client_cookie=db6e168a435ed51b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bdd9b700 1 -- 172.21.15.181:0/1896317426 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800ba70 con 0x7f68c00622a0 2020-11-16T22:26:53.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bdd9b700 1 -- 172.21.15.181:0/1896317426 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b800bbd0 con 0x7f68c00622a0 2020-11-16T22:26:53.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68bdd9b700 1 -- 172.21.15.181:0/1896317426 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8011400 con 0x7f68c00622a0 2020-11-16T22:26:53.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- 172.21.15.181:0/1896317426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 msgr2=0x7f68c0062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:53.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 --2- 172.21.15.181:0/1896317426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0062700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f68c00bcb90 tx=0x7f68b80091d0).stop 2020-11-16T22:26:53.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- 172.21.15.181:0/1896317426 shutdown_connections 2020-11-16T22:26:53.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 --2- 172.21.15.181:0/1896317426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0062700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.175:3300/0 conn(0x7f68c015f5b0 0x7f68c00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 --2- 172.21.15.181:0/1896317426 >> v2:172.21.15.181:3300/0 conn(0x7f68c00615f0 0x7f68c00d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- 172.21.15.181:0/1896317426 >> 172.21.15.181:0/1896317426 conn(0x7f68c00d69a0 msgr2=0x7f68c00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:53.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- 172.21.15.181:0/1896317426 shutdown_connections 2020-11-16T22:26:53.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- 172.21.15.181:0/1896317426 wait complete. 2020-11-16T22:26:53.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 Processor -- start 2020-11-16T22:26:53.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.157+0000 7f68c511e700 1 -- start start 2020-11-16T22:26:53.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 0x7f68c01730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0164d70 con 0x7f68c015f5b0 2020-11-16T22:26:53.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0165010 con 0x7f68c00615f0 2020-11-16T22:26:53.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01652b0 con 0x7f68c00622a0 2020-11-16T22:26:53.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56456/0 (socket says 172.21.15.181:56456) 2020-11-16T22:26:53.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 -- 172.21.15.181:0/3968193981 learned_addr learned my addr 172.21.15.181:0/3968193981 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:53.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68be59c700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bf59e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 0x7f68c01730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 -- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 msgr2=0x7f68c0165f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:53.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0165f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 -- 172.21.15.181:0/3968193981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 msgr2=0x7f68c01730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 0x7f68c01730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 -- 172.21.15.181:0/3968193981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0165550 con 0x7f68c00615f0 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bf59e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 0x7f68c01730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68bed9d700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f68a8006770 tx=0x7f68a8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8009790 con 0x7f68c00615f0 2020-11-16T22:26:53.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a80098f0 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8009c70 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c01657f0 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c0166420 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.161+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c0166640 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.165+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f68a8013690 con 0x7f68c00615f0 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.165+0000 7f68b6ffd700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 0x7f689c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:53.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.165+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f68a806e480 con 0x7f68c00615f0 2020-11-16T22:26:53.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.165+0000 7f68be59c700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 0x7f689c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:53.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.165+0000 7f68be59c700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 0x7f689c057520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68c0061a50 tx=0x7f68b0006eb0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:53.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.173+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f68a8044570 con 0x7f68c00615f0 2020-11-16T22:26:53.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:53.377+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "9"} v 0) v1 -- 0x7f68c0069eb0 con 0x7f68c00615f0 2020-11-16T22:26:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.017+0000 7f68b6ffd700 1 -- 172.21.15.181:0/3968193981 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "9"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f68a8047b90 con 0x7f68c00615f0 2020-11-16T22:26:54.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 msgr2=0x7f689c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 0x7f689c057520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68c0061a50 tx=0x7f68b0006eb0).stop 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 msgr2=0x7f68c0170860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f68a8006770 tx=0x7f68a8008040).stop 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 shutdown_connections 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f689c055060 0x7f689c057520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68c015f5b0 0x7f68c01730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00615f0 0x7f68c0170860 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0x7f68a8006770 tx=0x7f68a8008040).stop 2020-11-16T22:26:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 --2- 172.21.15.181:0/3968193981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00622a0 0x7f68c0165f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.021+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 >> 172.21.15.181:0/3968193981 conn(0x7f68c00d69a0 msgr2=0x7f68c00b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.025+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 shutdown_connections 2020-11-16T22:26:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.025+0000 7f68c511e700 1 -- 172.21.15.181:0/3968193981 wait complete. 2020-11-16T22:26:54.033 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2020-11-16T22:26:54.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.169+0000 7f1ae1897700 1 Processor -- start 2020-11-16T22:26:54.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.169+0000 7f1ae1897700 1 -- start start 2020-11-16T22:26:54.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1adc161860 0x7f1adc062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:54.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc0c8db0 con 0x7f1adc062740 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc165030 con 0x7f1adc161480 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc0c8330 con 0x7f1adc161860 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1adc161860 0x7f1adc062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1adc161860 0x7f1adc062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56478/0 (socket says 172.21.15.181:56478) 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 -- 172.21.15.181:0/1552898574 learned_addr learned my addr 172.21.15.181:0/1552898574 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae0895700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1adc161480 0x7f1adc164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42552/0 (socket says 172.21.15.181:42552) 2020-11-16T22:26:54.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 -- 172.21.15.181:0/1552898574 >> v1:172.21.15.181:6789/0 conn(0x7f1adc161480 legacy=0x7f1adc164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 -- 172.21.15.181:0/1552898574 >> v1:172.21.15.175:6789/0 conn(0x7f1adc062740 legacy=0x7f1adc0644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 -- 172.21.15.181:0/1552898574 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0c85d0 con 0x7f1adc161860 2020-11-16T22:26:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adbfff700 1 --2- 172.21.15.181:0/1552898574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc161860 0x7f1adc062250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1ac4009d50 tx=0x7f1ac4009290).ready entity=mon.1 client_cookie=95642b3771144953 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adb7fe700 1 -- 172.21.15.181:0/1552898574 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400bc80 con 0x7f1adc161860 2020-11-16T22:26:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adb7fe700 1 -- 172.21.15.181:0/1552898574 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400bde0 con 0x7f1adc161860 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1adb7fe700 1 -- 172.21.15.181:0/1552898574 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4011610 con 0x7f1adc161860 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- 172.21.15.181:0/1552898574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc161860 msgr2=0x7f1adc062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 --2- 172.21.15.181:0/1552898574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc161860 0x7f1adc062250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1ac4009d50 tx=0x7f1ac4009290).stop 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- 172.21.15.181:0/1552898574 shutdown_connections 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 --2- 172.21.15.181:0/1552898574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc161860 0x7f1adc062250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- 172.21.15.181:0/1552898574 >> 172.21.15.181:0/1552898574 conn(0x7f1adc0d2dc0 msgr2=0x7f1adc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:54.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- 172.21.15.181:0/1552898574 shutdown_connections 2020-11-16T22:26:54.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.173+0000 7f1ae1897700 1 -- 172.21.15.181:0/1552898574 wait complete. 2020-11-16T22:26:54.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 Processor -- start 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- start start 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 0x7f1adc1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1adc161480 0x7f1adc0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc165a10 con 0x7f1adc062740 2020-11-16T22:26:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc165cb0 con 0x7f1adc160ba0 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc165f50 con 0x7f1adc161480 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56486/0 (socket says 172.21.15.181:56486) 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1adbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1adc161480 0x7f1adc0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 -- 172.21.15.181:0/2215814663 learned_addr learned my addr 172.21.15.181:0/2215814663 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae0895700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 0x7f1adc1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 -- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1adc161480 msgr2=0x7f1adc0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1adc161480 0x7f1adc0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 -- 172.21.15.181:0/2215814663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 msgr2=0x7f1adc1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 0x7f1adc1606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 -- 172.21.15.181:0/2215814663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc1661f0 con 0x7f1adc160ba0 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae0895700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 0x7f1adc1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1096700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1ad400c710 tx=0x7f1ad4007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:54.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4007020 con 0x7f1adc160ba0 2020-11-16T22:26:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4007180 con 0x7f1adc160ba0 2020-11-16T22:26:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40064f0 con 0x7f1adc160ba0 2020-11-16T22:26:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc166490 con 0x7f1adc160ba0 2020-11-16T22:26:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.177+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc0cdcc0 con 0x7f1adc160ba0 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc044f60 con 0x7f1adc160ba0 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1ad4010030 con 0x7f1adc160ba0 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ad8ff9700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 0x7f1aac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ad406d3f0 con 0x7f1adc160ba0 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ae0895700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 0x7f1aac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.181+0000 7f1ae0895700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 0x7f1aac057520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1acc007c50 tx=0x7f1acc007120).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:54.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.189+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1ad4043540 con 0x7f1adc160ba0 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:54.393+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1adc0671e0 con 0x7f1adc160ba0 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.029+0000 7f1ad8ff9700 1 -- 172.21.15.181:0/2215814663 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1ad4001e60 con 0x7f1adc160ba0 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 msgr2=0x7f1aac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 0x7f1aac057520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1acc007c50 tx=0x7f1acc007120).stop 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 msgr2=0x7f1adc0cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f1ad400c710 tx=0x7f1ad4007b30).stop 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 shutdown_connections 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1aac055060 0x7f1aac057520 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1adc160ba0 0x7f1adc0cd7d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1adc062740 0x7f1adc1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 --2- 172.21.15.181:0/2215814663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1adc161480 0x7f1adc0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 >> 172.21.15.181:0/2215814663 conn(0x7f1adc0d2dc0 msgr2=0x7f1adc1648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 shutdown_connections 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.033+0000 7f1ae1897700 1 -- 172.21.15.181:0/2215814663 wait complete. 2020-11-16T22:26:55.037 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:26:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 Processor -- start 2020-11-16T22:26:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 -- start start 2020-11-16T22:26:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb89015d200 0x7fb89015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb8900667b0 0x7fb8900cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb890066480 con 0x7fb8900cdff0 2020-11-16T22:26:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb890066ee0 con 0x7fb8900667b0 2020-11-16T22:26:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb895718700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb890067180 con 0x7fb89015d200 2020-11-16T22:26:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb88effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb89015d200 0x7fb89015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb88effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb89015d200 0x7fb89015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56496/0 (socket says 172.21.15.181:56496) 2020-11-16T22:26:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb88effd700 1 -- 172.21.15.181:0/9372085 learned_addr learned my addr 172.21.15.181:0/9372085 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.165+0000 7fb88e7fc700 1 --2- 172.21.15.181:0/9372085 >> v2:172.21.15.175:3300/0 conn(0x7fb8900667b0 0x7fb8900cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88effd700 1 -- 172.21.15.181:0/9372085 >> v2:172.21.15.175:3300/0 conn(0x7fb8900667b0 msgr2=0x7fb8900cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:55.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88effd700 1 --2- 172.21.15.181:0/9372085 >> v2:172.21.15.175:3300/0 conn(0x7fb8900667b0 0x7fb8900cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88effd700 1 -- 172.21.15.181:0/9372085 >> v1:172.21.15.175:6789/0 conn(0x7fb8900cdff0 legacy=0x7fb8900cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88effd700 1 -- 172.21.15.181:0/9372085 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890067420 con 0x7fb89015d200 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88effd700 1 --2- 172.21.15.181:0/9372085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89015d200 0x7fb89015d5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb8900d46a0 tx=0x7fb880006eb0).ready entity=mon.1 client_cookie=1d78aa6ec41870f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88dffb700 1 -- 172.21.15.181:0/9372085 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009810 con 0x7fb89015d200 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88dffb700 1 -- 172.21.15.181:0/9372085 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880009970 con 0x7fb89015d200 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb88dffb700 1 -- 172.21.15.181:0/9372085 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009cf0 con 0x7fb89015d200 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- 172.21.15.181:0/9372085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89015d200 msgr2=0x7fb89015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 --2- 172.21.15.181:0/9372085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89015d200 0x7fb89015d5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb8900d46a0 tx=0x7fb880006eb0).stop 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- 172.21.15.181:0/9372085 shutdown_connections 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 --2- 172.21.15.181:0/9372085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89015d200 0x7fb89015d5e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 --2- 172.21.15.181:0/9372085 >> v2:172.21.15.175:3300/0 conn(0x7fb8900667b0 0x7fb8900cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- 172.21.15.181:0/9372085 >> 172.21.15.181:0/9372085 conn(0x7fb8900d4790 msgr2=0x7fb8900d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- 172.21.15.181:0/9372085 shutdown_connections 2020-11-16T22:26:55.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- 172.21.15.181:0/9372085 wait complete. 2020-11-16T22:26:55.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 Processor -- start 2020-11-16T22:26:55.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.169+0000 7fb895718700 1 -- start start 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8900cdff0 0x7fb89014c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89015d200 0x7fb8900621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890167560 con 0x7fb8900cdff0 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015f790 con 0x7fb8900667b0 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89016f490 con 0x7fb89015d200 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89015d200 0x7fb8900621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56498/0 (socket says 172.21.15.181:56498) 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 -- 172.21.15.181:0/1085802824 learned_addr learned my addr 172.21.15.181:0/1085802824 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88e7fc700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8900cdff0 0x7fb89014c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 -- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89015d200 msgr2=0x7fb8900621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89015d200 0x7fb8900621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 -- 172.21.15.181:0/1085802824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8900cdff0 msgr2=0x7fb89014c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8900cdff0 0x7fb89014c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 -- 172.21.15.181:0/1085802824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89016f730 con 0x7fb8900667b0 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb88effd700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb880006fd0 tx=0x7fb880015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880011d50 con 0x7fb8900667b0 2020-11-16T22:26:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88001e460 con 0x7fb8900667b0 2020-11-16T22:26:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89016f9d0 con 0x7fb8900667b0 2020-11-16T22:26:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88001e800 con 0x7fb8900667b0 2020-11-16T22:26:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.173+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89016ff10 con 0x7fb8900667b0 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89014c970 con 0x7fb8900667b0 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb880025030 con 0x7fb8900667b0 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb886ffd700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 0x7fb86c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb88006f400 con 0x7fb8900667b0 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb88e7fc700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 0x7fb86c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.177+0000 7fb88e7fc700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 0x7fb86c057550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb890062ec0 tx=0x7fb878006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.181+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb880045710 con 0x7fb8900667b0 2020-11-16T22:26:55.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8901700d0 con 0x7fb8900667b0 2020-11-16T22:26:55.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb886ffd700 1 -- 172.21.15.181:0/1085802824 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4646 (secure 0 0 0) 0x7fb880023070 con 0x7fb8900667b0 2020-11-16T22:26:55.375 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:55.375 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2},{"gid":4253,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5},{"gid":4266,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":3},{"gid":4296,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:26:55.032460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:creating","state_seq":2,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:26:55.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 msgr2=0x7fb86c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 0x7fb86c057550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb890062ec0 tx=0x7fb878006df0).stop 2020-11-16T22:26:55.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 msgr2=0x7fb890167070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb880006fd0 tx=0x7fb880015000).stop 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 shutdown_connections 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb86c055090 0x7fb86c057550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89015d200 0x7fb8900621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8900667b0 0x7fb890167070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 --2- 172.21.15.181:0/1085802824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8900cdff0 0x7fb89014c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 >> 172.21.15.181:0/1085802824 conn(0x7fb8900d4790 msgr2=0x7fb89015c560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 shutdown_connections 2020-11-16T22:26:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.389+0000 7fb895718700 1 -- 172.21.15.181:0/1085802824 wait complete. 2020-11-16T22:26:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2020-11-16T22:26:55.390 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:55.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 Processor -- start 2020-11-16T22:26:55.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 -- start start 2020-11-16T22:26:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e7815de80 0x7f9e780b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7815e2e0 con 0x7f9e780b2190 2020-11-16T22:26:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e780b2a60 con 0x7f9e7815ff80 2020-11-16T22:26:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e7cb4b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e780ce460 con 0x7f9e7815de80 2020-11-16T22:26:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e76ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e7815de80 0x7f9e780b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e77fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9e7815ff80 0x7f9e78160410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42582/0 (socket says 172.21.15.181:42582) 2020-11-16T22:26:55.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e77fff700 1 -- 172.21.15.181:0/2669985612 learned_addr learned my addr 172.21.15.181:0/2669985612 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:55.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e767fc700 1 -- 172.21.15.181:0/2669985612 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746199822 0 0) 0x7f9e780b2a60 con 0x7f9e7815ff80 2020-11-16T22:26:55.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e76ffd700 1 -- 172.21.15.181:0/2669985612 >> v1:172.21.15.181:6789/0 conn(0x7f9e7815ff80 legacy=0x7f9e78160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e76ffd700 1 -- 172.21.15.181:0/2669985612 >> v1:172.21.15.175:6789/0 conn(0x7f9e780b2190 legacy=0x7f9e780cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e76ffd700 1 -- 172.21.15.181:0/2669985612 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780ce700 con 0x7f9e7815de80 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e76ffd700 1 --2- 172.21.15.181:0/2669985612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7815de80 0x7f9e780b2570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9e7000b9a0 tx=0x7f9e70006df0).ready entity=mon.1 client_cookie=b62dca6317cff1ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e767fc700 1 -- 172.21.15.181:0/2669985612 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700096b0 con 0x7f9e7815de80 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e767fc700 1 -- 172.21.15.181:0/2669985612 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e70009810 con 0x7f9e7815de80 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.529+0000 7f9e767fc700 1 -- 172.21.15.181:0/2669985612 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70009b90 con 0x7f9e7815de80 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/2669985612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7815de80 msgr2=0x7f9e780b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/2669985612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7815de80 0x7f9e780b2570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9e7000b9a0 tx=0x7f9e70006df0).stop 2020-11-16T22:26:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/2669985612 shutdown_connections 2020-11-16T22:26:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/2669985612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7815de80 0x7f9e780b2570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/2669985612 >> 172.21.15.181:0/2669985612 conn(0x7f9e780cfe90 msgr2=0x7f9e780d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/2669985612 shutdown_connections 2020-11-16T22:26:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/2669985612 wait complete. 2020-11-16T22:26:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 Processor -- start 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- start start 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 0x7f9e7815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7815ff80 0x7f9e7815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7815fdb0 con 0x7f9e7815ff80 2020-11-16T22:26:55.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7813bf10 con 0x7f9e780b2190 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7816cde0 con 0x7f9e7815de80 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56512/0 (socket says 172.21.15.181:56512) 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 -- 172.21.15.181:0/118776791 learned_addr learned my addr 172.21.15.181:0/118776791 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e77fff700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7815ff80 0x7f9e7815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e76ffd700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 0x7f9e7815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 -- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 msgr2=0x7f9e7815bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 0x7f9e7815bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 -- 172.21.15.181:0/118776791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7815ff80 msgr2=0x7f9e7815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7815ff80 0x7f9e7815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 -- 172.21.15.181:0/118776791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7816d080 con 0x7f9e780b2190 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e76ffd700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 0x7f9e7815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:55.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e777fe700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f9e68001520 tx=0x7f9e68007df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800a7f0 con 0x7f9e780b2190 2020-11-16T22:26:55.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6800a950 con 0x7f9e780b2190 2020-11-16T22:26:55.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800ad10 con 0x7f9e780b2190 2020-11-16T22:26:55.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7816d320 con 0x7f9e780b2190 2020-11-16T22:26:55.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.533+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7816db00 con 0x7f9e780b2190 2020-11-16T22:26:55.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7816dcc0 con 0x7f9e780b2190 2020-11-16T22:26:55.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9e68010070 con 0x7f9e780b2190 2020-11-16T22:26:55.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e6f7fe700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 0x7f9e4c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9e6806dc90 con 0x7f9e780b2190 2020-11-16T22:26:55.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e76ffd700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 0x7f9e4c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.537+0000 7f9e76ffd700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 0x7f9e4c057520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9e7000b150 tx=0x7f9e70015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.545+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9e68043ef0 con 0x7f9e780b2190 2020-11-16T22:26:55.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e78044f60 con 0x7f9e780b2190 2020-11-16T22:26:55.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e6f7fe700 1 -- 172.21.15.181:0/118776791 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9e68047510 con 0x7f9e780b2190 2020-11-16T22:26:55.684 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:55.684 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 msgr2=0x7f9e4c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 0x7f9e4c057520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9e7000b150 tx=0x7f9e70015000).stop 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 msgr2=0x7f9e78142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f9e68001520 tx=0x7f9e68007df0).stop 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 shutdown_connections 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9e4c055060 0x7f9e4c057520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e7815ff80 0x7f9e7815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e780b2190 0x7f9e78142150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 --2- 172.21.15.181:0/118776791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7815de80 0x7f9e7815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 >> 172.21.15.181:0/118776791 conn(0x7f9e780cfe90 msgr2=0x7f9e780d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.697+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 shutdown_connections 2020-11-16T22:26:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.701+0000 7f9e7cb4b700 1 -- 172.21.15.181:0/118776791 wait complete. 2020-11-16T22:26:55.699 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:55.699 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-16T22:26:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.833+0000 7fe89ff83700 1 Processor -- start 2020-11-16T22:26:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.833+0000 7fe89ff83700 1 -- start start 2020-11-16T22:26:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe898147270 0x7fe8980605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89815e090 con 0x7fe89815d4e0 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89815edb0 con 0x7fe898147270 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89815e330 con 0x7fe898146e10 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36132/0 (socket says 172.21.15.181:36132) 2020-11-16T22:26:55.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 -- 172.21.15.181:0/3535635954 learned_addr learned my addr 172.21.15.181:0/3535635954 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:55.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89dd1f700 1 --1- 172.21.15.181:0/3535635954 >> v1:172.21.15.175:6789/0 conn(0x7fe89815d4e0 0x7fe89815d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:35706/0 (socket says 172.21.15.181:35706) 2020-11-16T22:26:55.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89e520700 1 --2- 172.21.15.181:0/3535635954 >> v2:172.21.15.181:3300/0 conn(0x7fe898147270 0x7fe8980605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 -- 172.21.15.181:0/3535635954 >> v2:172.21.15.181:3300/0 conn(0x7fe898147270 msgr2=0x7fe8980605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:55.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 --2- 172.21.15.181:0/3535635954 >> v2:172.21.15.181:3300/0 conn(0x7fe898147270 0x7fe8980605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 -- 172.21.15.181:0/3535635954 >> v1:172.21.15.175:6789/0 conn(0x7fe89815d4e0 legacy=0x7fe89815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 -- 172.21.15.181:0/3535635954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89815e5d0 con 0x7fe898146e10 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89d51e700 1 --2- 172.21.15.181:0/3535635954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe888006fd0 tx=0x7fe888006eb0).ready entity=mon.2 client_cookie=452bc62f6512a422 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89cd1d700 1 -- 172.21.15.181:0/3535635954 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880098d0 con 0x7fe898146e10 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89cd1d700 1 -- 172.21.15.181:0/3535635954 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe888009a30 con 0x7fe898146e10 2020-11-16T22:26:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89cd1d700 1 -- 172.21.15.181:0/3535635954 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888009db0 con 0x7fe898146e10 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- 172.21.15.181:0/3535635954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 msgr2=0x7fe8980600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 --2- 172.21.15.181:0/3535635954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe888006fd0 tx=0x7fe888006eb0).stop 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- 172.21.15.181:0/3535635954 shutdown_connections 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 --2- 172.21.15.181:0/3535635954 >> v2:172.21.15.181:3300/0 conn(0x7fe898147270 0x7fe8980605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 --2- 172.21.15.181:0/3535635954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe8980600f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- 172.21.15.181:0/3535635954 >> 172.21.15.181:0/3535635954 conn(0x7fe8980d3680 msgr2=0x7fe8980d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- 172.21.15.181:0/3535635954 shutdown_connections 2020-11-16T22:26:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.837+0000 7fe89ff83700 1 -- 172.21.15.181:0/3535635954 wait complete. 2020-11-16T22:26:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 Processor -- start 2020-11-16T22:26:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- start start 2020-11-16T22:26:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe898147270 0x7fe8980cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980cd120 con 0x7fe898147270 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980cd970 con 0x7fe89815d4e0 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980cce80 con 0x7fe898146e10 2020-11-16T22:26:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89e520700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36138/0 (socket says 172.21.15.181:36138) 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89e520700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56528/0 (socket says 172.21.15.181:56528) 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 -- 172.21.15.181:0/1975035799 learned_addr learned my addr 172.21.15.181:0/1975035799 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89d51e700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe898147270 0x7fe8980cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 -- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 msgr2=0x7fe8980cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 -- 172.21.15.181:0/1975035799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe898147270 msgr2=0x7fe8980cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe898147270 0x7fe8980cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 -- 172.21.15.181:0/1975035799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898154e20 con 0x7fe898146e10 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89e520700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89dd1f700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe89400c740 tx=0x7fe894007c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940072e0 con 0x7fe898146e10 2020-11-16T22:26:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894005530 con 0x7fe898146e10 2020-11-16T22:26:55.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940058b0 con 0x7fe898146e10 2020-11-16T22:26:55.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981550c0 con 0x7fe898146e10 2020-11-16T22:26:55.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.841+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe898155600 con 0x7fe898146e10 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe898069eb0 con 0x7fe898146e10 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe894010070 con 0x7fe898146e10 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe885ffb700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 0x7fe870057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe89406d380 con 0x7fe898146e10 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe89d51e700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 0x7fe870057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:55.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.845+0000 7fe89d51e700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 0x7fe870057520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe88800b1b0 tx=0x7fe888015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:55.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:55.853+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe8940435e0 con 0x7fe898146e10 2020-11-16T22:26:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.005+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe898062210 con 0x7fe898146e10 2020-11-16T22:26:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe885ffb700 1 -- 172.21.15.181:0/1975035799 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fe894046c00 con 0x7fe898146e10 2020-11-16T22:26:55.995 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:55.995 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:55.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 msgr2=0x7fe870057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 0x7fe870057520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe88800b1b0 tx=0x7fe888015000).stop 2020-11-16T22:26:55.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 msgr2=0x7fe898155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:55.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe89400c740 tx=0x7fe894007c40).stop 2020-11-16T22:26:55.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 shutdown_connections 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe870055060 0x7fe870057520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89815d4e0 0x7fe8980cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe898146e10 0x7fe898155aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 --2- 172.21.15.181:0/1975035799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe898147270 0x7fe8980cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 >> 172.21.15.181:0/1975035799 conn(0x7fe8980d3680 msgr2=0x7fe89815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 shutdown_connections 2020-11-16T22:26:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.009+0000 7fe89ff83700 1 -- 172.21.15.181:0/1975035799 wait complete. 2020-11-16T22:26:56.010 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:26:56.011 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:26:56.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 Processor -- start 2020-11-16T22:26:56.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 -- start start 2020-11-16T22:26:56.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e380d0a00 0x7f6e38140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e380cef10 0x7f6e38063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380cf380 con 0x7f6e38140550 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3815ebb0 con 0x7f6e380cef10 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e3ce56700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380cf5d0 con 0x7f6e380d0a00 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.149+0000 7f6e36ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e380d0a00 0x7f6e38140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e380d0a00 0x7f6e38140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56538/0 (socket says 172.21.15.181:56538) 2020-11-16T22:26:56.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 -- 172.21.15.181:0/3708530892 learned_addr learned my addr 172.21.15.181:0/3708530892 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:56.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e37fff700 1 --2- 172.21.15.181:0/3708530892 >> v2:172.21.15.175:3300/0 conn(0x7f6e380cef10 0x7f6e38063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 -- 172.21.15.181:0/3708530892 >> v2:172.21.15.175:3300/0 conn(0x7f6e380cef10 msgr2=0x7f6e38063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3708530892 >> v2:172.21.15.175:3300/0 conn(0x7f6e380cef10 0x7f6e38063ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 -- 172.21.15.181:0/3708530892 >> v1:172.21.15.175:6789/0 conn(0x7f6e38140550 legacy=0x7f6e380d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e367fc700 1 -- 172.21.15.181:0/3708530892 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011979591 0 0) 0x7f6e380cf380 con 0x7f6e38140550 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 -- 172.21.15.181:0/3708530892 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380cf870 con 0x7f6e380d0a00 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3708530892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380d0a00 0x7f6e38140930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6e380ceb20 tx=0x7f6e20006eb0).ready entity=mon.1 client_cookie=f5a573bcac10f9be server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e367fc700 1 -- 172.21.15.181:0/3708530892 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20009810 con 0x7f6e380d0a00 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e367fc700 1 -- 172.21.15.181:0/3708530892 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20009970 con 0x7f6e380d0a00 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e367fc700 1 -- 172.21.15.181:0/3708530892 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20009cf0 con 0x7f6e380d0a00 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3708530892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380d0a00 msgr2=0x7f6e38140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3708530892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380d0a00 0x7f6e38140930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6e380ceb20 tx=0x7f6e20006eb0).stop 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3708530892 shutdown_connections 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3708530892 >> v2:172.21.15.175:3300/0 conn(0x7f6e380cef10 0x7f6e38063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3708530892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380d0a00 0x7f6e38140930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3708530892 >> 172.21.15.181:0/3708530892 conn(0x7f6e380d4570 msgr2=0x7f6e380d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3708530892 shutdown_connections 2020-11-16T22:26:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.153+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3708530892 wait complete. 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 Processor -- start 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- start start 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 0x7f6e3815bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e380cbec0 0x7f6e38170400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38170d60 con 0x7f6e38140550 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38171000 con 0x7f6e380cef10 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e381712a0 con 0x7f6e380cbec0 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56540/0 (socket says 172.21.15.181:56540) 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 -- 172.21.15.181:0/3215798107 learned_addr learned my addr 172.21.15.181:0/3215798107 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e37fff700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e380cbec0 0x7f6e38170400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 0x7f6e3815bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 -- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e380cbec0 msgr2=0x7f6e38170400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e380cbec0 0x7f6e38170400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 -- 172.21.15.181:0/3215798107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 msgr2=0x7f6e3815bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 0x7f6e3815bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 -- 172.21.15.181:0/3215798107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3815c4d0 con 0x7f6e380cef10 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 0x7f6e3815bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:56.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e377fe700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f6e2800c760 tx=0x7f6e28007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800e560 con 0x7f6e380cef10 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800e6c0 con 0x7f6e380cef10 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3815c770 con 0x7f6e380cef10 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800ea40 con 0x7f6e380cef10 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.157+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3815bff0 con 0x7f6e380cef10 2020-11-16T22:26:56.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e38044f60 con 0x7f6e380cef10 2020-11-16T22:26:56.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6e28010070 con 0x7f6e380cef10 2020-11-16T22:26:56.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e2f7fe700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 0x7f6e14057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e2806d990 con 0x7f6e380cef10 2020-11-16T22:26:56.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 0x7f6e14057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.161+0000 7f6e36ffd700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 0x7f6e14057520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6e380d0fe0 tx=0x7f6e20015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.165+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6e28043b90 con 0x7f6e380cef10 2020-11-16T22:26:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6e3805d4e0 con 0x7f6e380cef10 2020-11-16T22:26:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e2f7fe700 1 -- 172.21.15.181:0/3215798107 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6e280471b0 con 0x7f6e380cef10 2020-11-16T22:26:56.307 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:26:56.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 msgr2=0x7f6e14057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 0x7f6e14057520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6e380d0fe0 tx=0x7f6e20015000).stop 2020-11-16T22:26:56.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 msgr2=0x7f6e3815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f6e2800c760 tx=0x7f6e28007a20).stop 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 shutdown_connections 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e14055060 0x7f6e14057520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e380cbec0 0x7f6e38170400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e380cef10 0x7f6e3815ceb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 --2- 172.21.15.181:0/3215798107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e38140550 0x7f6e3815bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 >> 172.21.15.181:0/3215798107 conn(0x7f6e380d4570 msgr2=0x7f6e38063c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 shutdown_connections 2020-11-16T22:26:56.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.321+0000 7f6e3ce56700 1 -- 172.21.15.181:0/3215798107 wait complete. 2020-11-16T22:26:56.328 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:26:56.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 Processor -- start 2020-11-16T22:26:56.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- start start 2020-11-16T22:26:56.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6504145e10 0x7f65041492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6504061b50 con 0x7f6504145a30 2020-11-16T22:26:56.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040618b0 con 0x7f65040816e0 2020-11-16T22:26:56.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040cb930 con 0x7f6504145e10 2020-11-16T22:26:56.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6501d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f65040816e0 0x7f65040cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42622/0 (socket says 172.21.15.181:42622) 2020-11-16T22:26:56.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6501d9b700 1 -- 172.21.15.181:0/4293276204 learned_addr learned my addr 172.21.15.181:0/4293276204 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:56.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6502d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6504145e10 0x7f65041492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216188570 0 0) 0x7f65040618b0 con 0x7f65040816e0 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e8003240 con 0x7f65040816e0 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6502d9d700 1 -- 172.21.15.181:0/4293276204 >> v1:172.21.15.181:6789/0 conn(0x7f65040816e0 legacy=0x7f65040cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6502d9d700 1 -- 172.21.15.181:0/4293276204 >> v1:172.21.15.175:6789/0 conn(0x7f6504145a30 legacy=0x7f6504080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928061488 0 0) 0x7f6504061b50 con 0x7f6504145a30 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6502d9d700 1 -- 172.21.15.181:0/4293276204 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040cbbd0 con 0x7f6504145e10 2020-11-16T22:26:56.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f6502d9d700 1 --2- 172.21.15.181:0/4293276204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6504145e10 0x7f65041492c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f64fc009c90 tx=0x7f64fc0091d0).ready entity=mon.1 client_cookie=f6d7b25ebaa3fbcc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00ba70 con 0x7f6504145e10 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc00bbd0 con 0x7f6504145e10 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650159a700 1 -- 172.21.15.181:0/4293276204 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011400 con 0x7f6504145e10 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- 172.21.15.181:0/4293276204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6504145e10 msgr2=0x7f65041492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 --2- 172.21.15.181:0/4293276204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6504145e10 0x7f65041492c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f64fc009c90 tx=0x7f64fc0091d0).stop 2020-11-16T22:26:56.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- 172.21.15.181:0/4293276204 shutdown_connections 2020-11-16T22:26:56.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 --2- 172.21.15.181:0/4293276204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6504145e10 0x7f65041492c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- 172.21.15.181:0/4293276204 >> 172.21.15.181:0/4293276204 conn(0x7f65040d68c0 msgr2=0x7f65040d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- 172.21.15.181:0/4293276204 shutdown_connections 2020-11-16T22:26:56.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.473+0000 7f650359e700 1 -- 172.21.15.181:0/4293276204 wait complete. 2020-11-16T22:26:56.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 Processor -- start 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- start start 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6504145a30 0x7f65040d5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 0x7f65040d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504170490 con 0x7f6504145e10 2020-11-16T22:26:56.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65041706c0 con 0x7f65040816e0 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6504170960 con 0x7f6504145a30 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56554/0 (socket says 172.21.15.181:56554) 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 -- 172.21.15.181:0/228603826 learned_addr learned my addr 172.21.15.181:0/228603826 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f6501d9b700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6504145a30 0x7f65040d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f6502d9d700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 0x7f65040d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 -- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6504145a30 msgr2=0x7f65040d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6504145a30 0x7f65040d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 -- 172.21.15.181:0/228603826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 msgr2=0x7f65040d5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 0x7f65040d5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 -- 172.21.15.181:0/228603826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504170c00 con 0x7f65040816e0 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f6502d9d700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 0x7f65040d5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650259c700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f64f400c730 tx=0x7f64f4007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400e560 con 0x7f65040816e0 2020-11-16T22:26:56.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f400e6e0 con 0x7f65040816e0 2020-11-16T22:26:56.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504170ea0 con 0x7f65040816e0 2020-11-16T22:26:56.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400eaa0 con 0x7f65040816e0 2020-11-16T22:26:56.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.477+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040d6170 con 0x7f65040816e0 2020-11-16T22:26:56.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6504171340 con 0x7f65040816e0 2020-11-16T22:26:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f64f4019030 con 0x7f65040816e0 2020-11-16T22:26:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f64f27fc700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 0x7f64d8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f64f406d3e0 con 0x7f65040816e0 2020-11-16T22:26:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f6501d9b700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 0x7f64d8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.481+0000 7f6501d9b700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 0x7f64d8057550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6504062720 tx=0x7f64f8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.489+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6504171340 con 0x7f65040816e0 2020-11-16T22:26:56.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6504171340 con 0x7f65040816e0 2020-11-16T22:26:56.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f64f27fc700 1 -- 172.21.15.181:0/228603826 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6504171340 con 0x7f65040816e0 2020-11-16T22:26:56.631 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:26:56.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 msgr2=0x7f64d8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 0x7f64d8057550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6504062720 tx=0x7f64f8009040).stop 2020-11-16T22:26:56.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 msgr2=0x7f650415d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f64f400c730 tx=0x7f64f4007ac0).stop 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 shutdown_connections 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64d8055090 0x7f64d8057550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6504145e10 0x7f65040d5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65040816e0 0x7f650415d210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 --2- 172.21.15.181:0/228603826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6504145a30 0x7f65040d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 >> 172.21.15.181:0/228603826 conn(0x7f65040d68c0 msgr2=0x7f65040d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 shutdown_connections 2020-11-16T22:26:56.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.645+0000 7f650359e700 1 -- 172.21.15.181:0/228603826 wait complete. 2020-11-16T22:26:56.646 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:26:56.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 Processor -- start 2020-11-16T22:26:56.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 -- start start 2020-11-16T22:26:56.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f01d00615f0 0x7f01d00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d0160e40 con 0x7f01d015f5b0 2020-11-16T22:26:56.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d0160ba0 con 0x7f01d0062790 2020-11-16T22:26:56.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01d4ebc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d01609a0 con 0x7f01d00615f0 2020-11-16T22:26:56.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01ced9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f01d0062790 0x7f01d0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42638/0 (socket says 172.21.15.181:42638) 2020-11-16T22:26:56.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cdd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f01d00615f0 0x7f01d00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01ced9d700 1 -- 172.21.15.181:0/1356719368 learned_addr learned my addr 172.21.15.181:0/1356719368 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:56.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cdd9b700 1 -- 172.21.15.181:0/1356719368 >> v1:172.21.15.181:6789/0 conn(0x7f01d0062790 legacy=0x7f01d0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cdd9b700 1 -- 172.21.15.181:0/1356719368 >> v1:172.21.15.175:6789/0 conn(0x7f01d015f5b0 legacy=0x7f01d00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cdd9b700 1 -- 172.21.15.181:0/1356719368 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0060970 con 0x7f01d00615f0 2020-11-16T22:26:56.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cd59a700 1 -- 172.21.15.181:0/1356719368 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165899951 0 0) 0x7f01d0160ba0 con 0x7f01d0062790 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cdd9b700 1 --2- 172.21.15.181:0/1356719368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d00615f0 0x7f01d00622a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f01c400b9a0 tx=0x7f01c4006df0).ready entity=mon.1 client_cookie=add37336f3173264 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cd59a700 1 -- 172.21.15.181:0/1356719368 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c40097b0 con 0x7f01d00615f0 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cd59a700 1 -- 172.21.15.181:0/1356719368 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4009910 con 0x7f01d00615f0 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.781+0000 7f01cd59a700 1 -- 172.21.15.181:0/1356719368 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4009c90 con 0x7f01d00615f0 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- 172.21.15.181:0/1356719368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d00615f0 msgr2=0x7f01d00622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/1356719368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d00615f0 0x7f01d00622a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f01c400b9a0 tx=0x7f01c4006df0).stop 2020-11-16T22:26:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- 172.21.15.181:0/1356719368 shutdown_connections 2020-11-16T22:26:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/1356719368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d00615f0 0x7f01d00622a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- 172.21.15.181:0/1356719368 >> 172.21.15.181:0/1356719368 conn(0x7f01d00d69a0 msgr2=0x7f01d00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- 172.21.15.181:0/1356719368 shutdown_connections 2020-11-16T22:26:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- 172.21.15.181:0/1356719368 wait complete. 2020-11-16T22:26:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 Processor -- start 2020-11-16T22:26:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.785+0000 7f01d4ebc700 1 -- start start 2020-11-16T22:26:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d00615f0 0x7f01d014b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d015f5b0 0x7f01d0080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00bb490 con 0x7f01d00615f0 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00bb730 con 0x7f01d015f5b0 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00bb9d0 con 0x7f01d0062790 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36180/0 (socket says 172.21.15.181:36180) 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 -- 172.21.15.181:0/4024863917 learned_addr learned my addr 172.21.15.181:0/4024863917 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01ced9d700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d015f5b0 0x7f01d0080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01ce59c700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d00615f0 0x7f01d014b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 -- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d015f5b0 msgr2=0x7f01d0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d015f5b0 0x7f01d0080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 -- 172.21.15.181:0/4024863917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d00615f0 msgr2=0x7f01d014b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d00615f0 0x7f01d014b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 -- 172.21.15.181:0/4024863917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00bbc70 con 0x7f01d0062790 2020-11-16T22:26:56.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01cdd9b700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f01c400bf10 tx=0x7f01c4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4011e50 con 0x7f01d0062790 2020-11-16T22:26:56.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d00bbf10 con 0x7f01d0062790 2020-11-16T22:26:56.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c401e460 con 0x7f01d0062790 2020-11-16T22:26:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c401e7e0 con 0x7f01d0062790 2020-11-16T22:26:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d00d3ee0 con 0x7f01d0062790 2020-11-16T22:26:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.789+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d0069eb0 con 0x7f01d0062790 2020-11-16T22:26:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.793+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f01c401a040 con 0x7f01d0062790 2020-11-16T22:26:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.793+0000 7f01be7fc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 0x7f01a4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.793+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01c406f250 con 0x7f01d0062790 2020-11-16T22:26:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.793+0000 7f01ce59c700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 0x7f01a4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.793+0000 7f01ce59c700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 0x7f01a4057550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f01c0003350 tx=0x7f01c0007af0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.797+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f01c4045560 con 0x7f01d0062790 2020-11-16T22:26:56.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.949+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01d00bc0a0 con 0x7f01d0062790 2020-11-16T22:26:56.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.949+0000 7f01be7fc700 1 -- 172.21.15.181:0/4024863917 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f01c4023020 con 0x7f01d0062790 2020-11-16T22:26:56.937 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 msgr2=0x7f01a4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 0x7f01a4057550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f01c0003350 tx=0x7f01c0007af0).stop 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 msgr2=0x7f01d014b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f01c400bf10 tx=0x7f01c4015000).stop 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 shutdown_connections 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f01a4055090 0x7f01a4057550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d015f5b0 0x7f01d0080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01d00615f0 0x7f01d014b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 --2- 172.21.15.181:0/4024863917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01d0062790 0x7f01d014b830 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 >> 172.21.15.181:0/4024863917 conn(0x7f01d00d69a0 msgr2=0x7f01d0061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 shutdown_connections 2020-11-16T22:26:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:56.953+0000 7f01d4ebc700 1 -- 172.21.15.181:0/4024863917 wait complete. 2020-11-16T22:26:56.951 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:26:57.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.081+0000 7f76c243a700 1 Processor -- start 2020-11-16T22:26:57.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.081+0000 7f76c243a700 1 -- start start 2020-11-16T22:26:57.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76bc160690 0x7f76bc066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc15e2e0 con 0x7f76bc0b2190 2020-11-16T22:26:57.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc0ce460 con 0x7f76bc15de80 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76bc066a60 con 0x7f76bc160690 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c0c37700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f76bc15de80 0x7f76bc15ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42658/0 (socket says 172.21.15.181:42658) 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c0c37700 1 -- 172.21.15.181:0/136254933 learned_addr learned my addr 172.21.15.181:0/136254933 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c1c39700 1 --2- 172.21.15.181:0/136254933 >> v2:172.21.15.175:3300/0 conn(0x7f76bc160690 0x7f76bc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607169 0 0) 0x7f76bc0ce460 con 0x7f76bc15de80 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a4003240 con 0x7f76bc15de80 2020-11-16T22:26:57.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766283774 0 0) 0x7f76bc15e2e0 con 0x7f76bc0b2190 2020-11-16T22:26:57.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76bc0ce460 con 0x7f76bc0b2190 2020-11-16T22:26:57.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c1c39700 1 -- 172.21.15.181:0/136254933 >> v1:172.21.15.181:6789/0 conn(0x7f76bc15de80 legacy=0x7f76bc15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c1c39700 1 -- 172.21.15.181:0/136254933 >> v1:172.21.15.175:6789/0 conn(0x7f76bc0b2190 legacy=0x7f76bc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c1c39700 1 -- 172.21.15.181:0/136254933 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0ce700 con 0x7f76bc160690 2020-11-16T22:26:57.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c1c39700 1 --2- 172.21.15.181:0/136254933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc066570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f76b4009c90 tx=0x7f76b40091d0).ready entity=mon.0 client_cookie=838c8c0472491948 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400b9c0 con 0x7f76bc160690 2020-11-16T22:26:57.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400bb20 con 0x7f76bc160690 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76abfff700 1 -- 172.21.15.181:0/136254933 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400bea0 con 0x7f76bc160690 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- 172.21.15.181:0/136254933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 msgr2=0x7f76bc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 --2- 172.21.15.181:0/136254933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc066570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f76b4009c90 tx=0x7f76b40091d0).stop 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- 172.21.15.181:0/136254933 shutdown_connections 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 --2- 172.21.15.181:0/136254933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc066570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- 172.21.15.181:0/136254933 >> 172.21.15.181:0/136254933 conn(0x7f76bc0cfe90 msgr2=0x7f76bc0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- 172.21.15.181:0/136254933 shutdown_connections 2020-11-16T22:26:57.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- 172.21.15.181:0/136254933 wait complete. 2020-11-16T22:26:57.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 Processor -- start 2020-11-16T22:26:57.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.085+0000 7f76c243a700 1 -- start start 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76bc15de80 0x7f76bc145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc170000 con 0x7f76bc160690 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc157f60 con 0x7f76bc0b2190 2020-11-16T22:26:57.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc158200 con 0x7f76bc15de80 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c0c37700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76bc15de80 0x7f76bc145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56592/0 (socket says 172.21.15.181:56592) 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 -- 172.21.15.181:0/1609925051 learned_addr learned my addr 172.21.15.181:0/1609925051 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1c39700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 -- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76bc15de80 msgr2=0x7f76bc145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76bc15de80 0x7f76bc145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 -- 172.21.15.181:0/1609925051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 msgr2=0x7f76bc158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 -- 172.21.15.181:0/1609925051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc1584a0 con 0x7f76bc0b2190 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c1438700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f76b800c690 tx=0x7f76b8007a10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b800e490 con 0x7f76bc0b2190 2020-11-16T22:26:57.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b800e5f0 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc158740 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b800e990 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc170660 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f76b8019030 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc1708e0 con 0x7f76bc0b2190 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 0x7f7694057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f76b806d1f0 con 0x7f76bc0b2190 2020-11-16T22:26:57.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c0c37700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 0x7f7694057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.089+0000 7f76c0c37700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 0x7f7694057520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f76bc065b80 tx=0x7f76ac009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.097+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f76b8043450 con 0x7f76bc0b2190 2020-11-16T22:26:57.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.249+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76bc044f60 con 0x7f76bc0b2190 2020-11-16T22:26:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76a97fa700 1 -- 172.21.15.181:0/1609925051 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f76b8046a70 con 0x7f76bc0b2190 2020-11-16T22:26:57.238 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:26:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 msgr2=0x7f7694057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 0x7f7694057520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f76bc065b80 tx=0x7f76ac009040).stop 2020-11-16T22:26:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 msgr2=0x7f76bc145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f76b800c690 tx=0x7f76b8007a10).stop 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 shutdown_connections 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7694055060 0x7f7694057520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc160690 0x7f76bc158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76bc0b2190 0x7f76bc145460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 --2- 172.21.15.181:0/1609925051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76bc15de80 0x7f76bc145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 >> 172.21.15.181:0/1609925051 conn(0x7f76bc0cfe90 msgr2=0x7f76bc064fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 shutdown_connections 2020-11-16T22:26:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.253+0000 7f76c243a700 1 -- 172.21.15.181:0/1609925051 wait complete. 2020-11-16T22:26:57.252 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-16T22:26:57.252 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-16T22:26:57.253 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-16T22:26:57.366 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:57.372 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:57.374 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:26:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 Processor -- start 2020-11-16T22:26:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 -- start start 2020-11-16T22:26:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c41621b0 con 0x7f64c40633d0 2020-11-16T22:26:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c4064970 con 0x7f64c40637b0 2020-11-16T22:26:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c8ba8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4065930 con 0x7f64c4061810 2020-11-16T22:26:57.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36212/0 (socket says 172.21.15.181:36212) 2020-11-16T22:26:57.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.389+0000 7f64c1d9b700 1 -- 172.21.15.181:0/2405622019 learned_addr learned my addr 172.21.15.181:0/2405622019 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:57.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c159a700 1 -- 172.21.15.181:0/2405622019 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656760750 0 0) 0x7f64c4064970 con 0x7f64c40637b0 2020-11-16T22:26:57.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c1d9b700 1 -- 172.21.15.181:0/2405622019 >> v1:172.21.15.181:6789/0 conn(0x7f64c40637b0 legacy=0x7f64c4062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c1d9b700 1 -- 172.21.15.181:0/2405622019 >> v1:172.21.15.175:6789/0 conn(0x7f64c40633d0 legacy=0x7f64c4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c1d9b700 1 -- 172.21.15.181:0/2405622019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4064c10 con 0x7f64c4061810 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c1d9b700 1 --2- 172.21.15.181:0/2405622019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f64b800ba20 tx=0x7f64b8006eb0).ready entity=mon.2 client_cookie=42855d11792e2664 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c159a700 1 -- 172.21.15.181:0/2405622019 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80098d0 con 0x7f64c4061810 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c159a700 1 -- 172.21.15.181:0/2405622019 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8009a30 con 0x7f64c4061810 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c159a700 1 -- 172.21.15.181:0/2405622019 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8009db0 con 0x7f64c4061810 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- 172.21.15.181:0/2405622019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 msgr2=0x7f64c4161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/2405622019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f64b800ba20 tx=0x7f64b8006eb0).stop 2020-11-16T22:26:57.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- 172.21.15.181:0/2405622019 shutdown_connections 2020-11-16T22:26:57.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/2405622019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4061810 0x7f64c4161cc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- 172.21.15.181:0/2405622019 >> 172.21.15.181:0/2405622019 conn(0x7f64c40cf890 msgr2=0x7f64c40cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- 172.21.15.181:0/2405622019 shutdown_connections 2020-11-16T22:26:57.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- 172.21.15.181:0/2405622019 wait complete. 2020-11-16T22:26:57.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 Processor -- start 2020-11-16T22:26:57.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.393+0000 7f64c8ba8700 1 -- start start 2020-11-16T22:26:57.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 0x7f64c416ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c40633d0 0x7f64c416e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 0x7f64c4170890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c416d300 con 0x7f64c40633d0 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4171340 con 0x7f64c4061810 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41715e0 con 0x7f64c40637b0 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 0x7f64c416ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 0x7f64c416ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56606/0 (socket says 172.21.15.181:56606) 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c259c700 1 -- 172.21.15.181:0/3128291169 learned_addr learned my addr 172.21.15.181:0/3128291169 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:57.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 0x7f64c4170890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c1d9b700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c40633d0 0x7f64c416e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 -- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 msgr2=0x7f64c416ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 0x7f64c416ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 -- 172.21.15.181:0/3128291169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c40633d0 msgr2=0x7f64c416e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c40633d0 0x7f64c416e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 -- 172.21.15.181:0/3128291169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c416d4f0 con 0x7f64c40637b0 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c2d9d700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 0x7f64c4170890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f64b400c710 tx=0x7f64b4007890).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4013410 con 0x7f64c40637b0 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4013570 con 0x7f64c40637b0 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c416d790 con 0x7f64c40637b0 2020-11-16T22:26:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4013930 con 0x7f64c40637b0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.397+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c416ec80 con 0x7f64c40637b0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c41717a0 con 0x7f64c40637b0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f64b4010030 con 0x7f64c40637b0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64b27fc700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 0x7f6498057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f64b406daf0 con 0x7f64c40637b0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64c259c700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 0x7f6498057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.401+0000 7f64c259c700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 0x7f6498057520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f64bc007fd0 tx=0x7f64bc001450).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.405+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f64b4043d50 con 0x7f64c40637b0 2020-11-16T22:26:57.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.585+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f64c4069eb0 con 0x7f64c40637b0 2020-11-16T22:26:57.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64b27fc700 1 -- 172.21.15.181:0/3128291169 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+132415 (secure 0 0 0) 0x7f64b4047370 con 0x7f64c40637b0 2020-11-16T22:26:57.579 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:57.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 msgr2=0x7f6498057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 0x7f6498057520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f64bc007fd0 tx=0x7f64bc001450).stop 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 msgr2=0x7f64c4170890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 0x7f64c4170890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f64b400c710 tx=0x7f64b4007890).stop 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 shutdown_connections 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6498055060 0x7f6498057520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c40637b0 0x7f64c4170890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c4061810 0x7f64c416ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 --2- 172.21.15.181:0/3128291169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64c40633d0 0x7f64c416e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 >> 172.21.15.181:0/3128291169 conn(0x7f64c40cf890 msgr2=0x7f64c40610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 shutdown_connections 2020-11-16T22:26:57.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.593+0000 7f64c8ba8700 1 -- 172.21.15.181:0/3128291169 wait complete. 2020-11-16T22:26:57.593 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":5,"active_gid":4111,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":15040},{"type":"v1","addr":"172.21.15.181:6833","nonce":15040}]},"active_addr":"172.21.15.181:6833/15040","active_change":"2020-11-16T22:26:38.658323+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1312024782}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3489205090}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3557332167}]}]}} 2020-11-16T22:26:57.600 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-16T22:26:57.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-16T22:26:57.600 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 Processor -- start 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 -- start start 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 0x7f26e40d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e40d1c70 con 0x7f26e40816e0 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e40d1f10 con 0x7f26e4145c60 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26ea05a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40d21b0 con 0x7f26e4145880 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26e8857700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f26e4145c60 0x7f26e4080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42686/0 (socket says 172.21.15.181:42686) 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26e8857700 1 -- 172.21.15.181:0/3026143330 learned_addr learned my addr 172.21.15.181:0/3026143330 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26e9058700 1 --2- 172.21.15.181:0/3026143330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 0x7f26e40d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26d7fff700 1 -- 172.21.15.181:0/3026143330 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249041052 0 0) 0x7f26e40d1f10 con 0x7f26e4145c60 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.769+0000 7f26d7fff700 1 -- 172.21.15.181:0/3026143330 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c8003240 con 0x7f26e4145c60 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26e9058700 1 -- 172.21.15.181:0/3026143330 >> v1:172.21.15.181:6789/0 conn(0x7f26e4145c60 legacy=0x7f26e4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26e9058700 1 -- 172.21.15.181:0/3026143330 >> v1:172.21.15.175:6789/0 conn(0x7f26e40816e0 legacy=0x7f26e40606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26e9058700 1 -- 172.21.15.181:0/3026143330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40d2450 con 0x7f26e4145880 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26e9058700 1 --2- 172.21.15.181:0/3026143330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 0x7f26e40d1110 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f26dc00b9d0 tx=0x7f26dc006df0).ready entity=mon.2 client_cookie=6222822c069896a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26d7fff700 1 -- 172.21.15.181:0/3026143330 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0097b0 con 0x7f26e4145880 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26d7fff700 1 -- 172.21.15.181:0/3026143330 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc009910 con 0x7f26e4145880 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26d7fff700 1 -- 172.21.15.181:0/3026143330 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc009c90 con 0x7f26e4145880 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- 172.21.15.181:0/3026143330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 msgr2=0x7f26e40d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3026143330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 0x7f26e40d1110 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f26dc00b9d0 tx=0x7f26dc006df0).stop 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- 172.21.15.181:0/3026143330 shutdown_connections 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3026143330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e4145880 0x7f26e40d1110 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- 172.21.15.181:0/3026143330 >> 172.21.15.181:0/3026143330 conn(0x7f26e40d6770 msgr2=0x7f26e40d6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- 172.21.15.181:0/3026143330 shutdown_connections 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- 172.21.15.181:0/3026143330 wait complete. 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 Processor -- start 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.773+0000 7f26ea05a700 1 -- start start 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e4145880 0x7f26e41613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 0x7f26e4141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4160000 con 0x7f26e4145c60 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4161bb0 con 0x7f26e4145880 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4161e50 con 0x7f26e40816e0 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e8857700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e4145880 0x7f26e41613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36232/0 (socket says 172.21.15.181:36232) 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 -- 172.21.15.181:0/3449255584 learned_addr learned my addr 172.21.15.181:0/3449255584 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9859700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 0x7f26e4141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 -- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e4145880 msgr2=0x7f26e41613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e4145880 0x7f26e41613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 -- 172.21.15.181:0/3449255584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 msgr2=0x7f26e4141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 0x7f26e4141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 -- 172.21.15.181:0/3449255584 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e41620f0 con 0x7f26e40816e0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9859700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 0x7f26e4141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e9058700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f26dc002fc0 tx=0x7f26dc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc011d50 con 0x7f26e40816e0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc01e460 con 0x7f26e40816e0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc01e7e0 con 0x7f26e40816e0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e4162390 con 0x7f26e40816e0 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e4162950 con 0x7f26e40816e0 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4044f60 con 0x7f26e40816e0 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f26dc025030 con 0x7f26e40816e0 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 0x7f26bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26dc06f4d0 con 0x7f26e40816e0 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.777+0000 7f26e8857700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 0x7f26bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.781+0000 7f26e8857700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 0x7f26bc057520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f26e4144d20 tx=0x7f26d8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.789+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f26dc0457e0 con 0x7f26e40816e0 2020-11-16T22:26:57.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.941+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26e405c4a0 con 0x7f26e40816e0 2020-11-16T22:26:57.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.941+0000 7f26d57fa700 1 -- 172.21.15.181:0/3449255584 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f26dc031200 con 0x7f26e40816e0 2020-11-16T22:26:57.929 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:57.930 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:57.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 msgr2=0x7f26bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 0x7f26bc057520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f26e4144d20 tx=0x7f26d8009040).stop 2020-11-16T22:26:57.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 msgr2=0x7f26e415fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f26dc002fc0 tx=0x7f26dc015000).stop 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 shutdown_connections 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26bc055060 0x7f26bc057520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26e4145c60 0x7f26e4141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26e40816e0 0x7f26e415fb10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 --2- 172.21.15.181:0/3449255584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e4145880 0x7f26e41613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:57.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 >> 172.21.15.181:0/3449255584 conn(0x7f26e40d6770 msgr2=0x7f26e4080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:57.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 shutdown_connections 2020-11-16T22:26:57.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:57.945+0000 7f26ea05a700 1 -- 172.21.15.181:0/3449255584 wait complete. 2020-11-16T22:26:57.945 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-16T22:26:57.945 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:26:58.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 Processor -- start 2020-11-16T22:26:58.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- start start 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 0x7fbff415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 0x7fbff40c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 0x7fbff415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff415d8d0 con 0x7fbff40c8380 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40ce070 con 0x7fbff415fec0 2020-11-16T22:26:58.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40ce310 con 0x7fbff40c7ab0 2020-11-16T22:26:58.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 0x7fbff40c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 0x7fbff415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 0x7fbff40c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56630/0 (socket says 172.21.15.181:56630) 2020-11-16T22:26:58.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff2ffd700 1 -- 172.21.15.181:0/4007541903 learned_addr learned my addr 172.21.15.181:0/4007541903 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff3fff700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 0x7fbff415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 -- 172.21.15.181:0/4007541903 >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 msgr2=0x7fbff40c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 0x7fbff40c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 -- 172.21.15.181:0/4007541903 >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 msgr2=0x7fbff415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 0x7fbff415dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 -- 172.21.15.181:0/4007541903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40ce5b0 con 0x7fbff40c7ab0 2020-11-16T22:26:58.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff3fff700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 0x7fbff415dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff37fe700 1 --2- 172.21.15.181:0/4007541903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 0x7fbff415f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbff405ce10 tx=0x7fbfe4006df0).ready entity=mon.2 client_cookie=6e9aef21412b0b2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff27fc700 1 -- 172.21.15.181:0/4007541903 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4017070 con 0x7fbff40c7ab0 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff27fc700 1 -- 172.21.15.181:0/4007541903 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe400f300 con 0x7fbff40c7ab0 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff27fc700 1 -- 172.21.15.181:0/4007541903 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4013a90 con 0x7fbff40c7ab0 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- 172.21.15.181:0/4007541903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 msgr2=0x7fbff415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- 172.21.15.181:0/4007541903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 0x7fbff415f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbff405ce10 tx=0x7fbfe4006df0).stop 2020-11-16T22:26:58.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- 172.21.15.181:0/4007541903 shutdown_connections 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.175:3300/0 conn(0x7fbff40c8380 0x7fbff415dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- 172.21.15.181:0/4007541903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c7ab0 0x7fbff415f9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 --2- 172.21.15.181:0/4007541903 >> v2:172.21.15.181:3300/0 conn(0x7fbff415fec0 0x7fbff40c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- 172.21.15.181:0/4007541903 >> 172.21.15.181:0/4007541903 conn(0x7fbff4060bc0 msgr2=0x7fbff40d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- 172.21.15.181:0/4007541903 shutdown_connections 2020-11-16T22:26:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.085+0000 7fbff9a75700 1 -- 172.21.15.181:0/4007541903 wait complete. 2020-11-16T22:26:58.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 Processor -- start 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- start start 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff40c7ab0 0x7fbff4161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbff415fec0 0x7fbff416eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff416bfb0 con 0x7fbff40c7ab0 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40c95d0 con 0x7fbff415fec0 2020-11-16T22:26:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40c9870 con 0x7fbff40c8380 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36246/0 (socket says 172.21.15.181:36246) 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 -- 172.21.15.181:0/3825890346 learned_addr learned my addr 172.21.15.181:0/3825890346 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff3fff700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbff415fec0 0x7fbff416eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff37fe700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff40c7ab0 0x7fbff4161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 -- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbff415fec0 msgr2=0x7fbff416eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbff415fec0 0x7fbff416eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 -- 172.21.15.181:0/3825890346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff40c7ab0 msgr2=0x7fbff4161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff40c7ab0 0x7fbff4161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 -- 172.21.15.181:0/3825890346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40c9b10 con 0x7fbff40c8380 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff2ffd700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbfe800f890 tx=0x7fbfe8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8014670 con 0x7fbff40c8380 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe80147d0 con 0x7fbff40c8380 2020-11-16T22:26:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff40c9db0 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8014b50 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff416f1d0 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.089+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff416c1b0 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.093+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbfe8024070 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.093+0000 7fbfdb7fe700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 0x7fbfc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.093+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbfe806e280 con 0x7fbff40c8380 2020-11-16T22:26:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.093+0000 7fbff37fe700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 0x7fbfc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.093+0000 7fbff37fe700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 0x7fbfc8057520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbfe400be30 tx=0x7fbfe4019000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.097+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbfe80444e0 con 0x7fbff40c8380 2020-11-16T22:26:58.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.245+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbff40c9f40 con 0x7fbff40c8380 2020-11-16T22:26:58.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.245+0000 7fbfdb7fe700 1 -- 172.21.15.181:0/3825890346 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fbfe8018020 con 0x7fbff40c8380 2020-11-16T22:26:58.232 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:26:58.232 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:26:58.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 msgr2=0x7fbfc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 0x7fbfc8057520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fbfe400be30 tx=0x7fbfe4019000).stop 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 msgr2=0x7fbff4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbfe800f890 tx=0x7fbfe8008040).stop 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 shutdown_connections 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfc8055060 0x7fbfc8057520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbff415fec0 0x7fbff416eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbff40c7ab0 0x7fbff4161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 --2- 172.21.15.181:0/3825890346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbff40c8380 0x7fbff4161d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 >> 172.21.15.181:0/3825890346 conn(0x7fbff4060bc0 msgr2=0x7fbff40b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 shutdown_connections 2020-11-16T22:26:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.249+0000 7fbff9a75700 1 -- 172.21.15.181:0/3825890346 wait complete. 2020-11-16T22:26:58.246 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-16T22:26:58.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.381+0000 7f99f63b3700 1 Processor -- start 2020-11-16T22:26:58.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.381+0000 7f99f63b3700 1 -- start start 2020-11-16T22:26:58.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 0x7f99f00cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99f0163970 0x7f99f0163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0060580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00cfc60 con 0x7f99f00cf390 2020-11-16T22:26:58.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f0065190 con 0x7f99f0163970 2020-11-16T22:26:58.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0065430 con 0x7f99f00600f0 2020-11-16T22:26:58.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99f0163970 0x7f99f0163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99f0163970 0x7f99f0163df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56642/0 (socket says 172.21.15.181:56642) 2020-11-16T22:26:58.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 -- 172.21.15.181:0/3399056848 learned_addr learned my addr 172.21.15.181:0/3399056848 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f5bb2700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f53b1700 1 --2- 172.21.15.181:0/3399056848 >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 0x7f99f00cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 -- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 msgr2=0x7f99f0060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0060580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 -- 172.21.15.181:0/3399056848 >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 msgr2=0x7f99f00cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3399056848 >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 0x7f99f00cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 -- 172.21.15.181:0/3399056848 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00656d0 con 0x7f99f0163970 2020-11-16T22:26:58.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f53b1700 1 --2- 172.21.15.181:0/3399056848 >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 0x7f99f00cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f0163df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f99f00ca440 tx=0x7f99e0006eb0).ready entity=mon.1 client_cookie=1017ab4e4381025a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99dffff700 1 -- 172.21.15.181:0/3399056848 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0009810 con 0x7f99f0163970 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99dffff700 1 -- 172.21.15.181:0/3399056848 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0009970 con 0x7f99f0163970 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99dffff700 1 -- 172.21.15.181:0/3399056848 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0009cf0 con 0x7f99f0163970 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 msgr2=0x7f99f0163df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f0163df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f99f00ca440 tx=0x7f99e0006eb0).stop 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- 172.21.15.181:0/3399056848 shutdown_connections 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0060580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3399056848 >> v2:172.21.15.175:3300/0 conn(0x7f99f00cf390 0x7f99f00cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3399056848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f0163df0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- 172.21.15.181:0/3399056848 >> 172.21.15.181:0/3399056848 conn(0x7f99f00cfe90 msgr2=0x7f99f00d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- 172.21.15.181:0/3399056848 shutdown_connections 2020-11-16T22:26:58.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.385+0000 7f99f63b3700 1 -- 172.21.15.181:0/3399056848 wait complete. 2020-11-16T22:26:58.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 Processor -- start 2020-11-16T22:26:58.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- start start 2020-11-16T22:26:58.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00cf390 0x7f99f013da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f016f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f016f940 con 0x7f99f00cf390 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f016fbe0 con 0x7f99f0163970 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f016fe80 con 0x7f99f00600f0 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f016f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f53b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f016f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56652/0 (socket says 172.21.15.181:56652) 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 -- 172.21.15.181:0/3704431232 learned_addr learned my addr 172.21.15.181:0/3704431232 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00cf390 0x7f99f013da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 -- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 msgr2=0x7f99f0160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f00600f0 0x7f99f0160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 -- 172.21.15.181:0/3704431232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00cf390 msgr2=0x7f99f013da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f00cf390 0x7f99f013da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f01625e0 con 0x7f99f0163970 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f5bb2700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f016f050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f99e80091a0 tx=0x7f99e80091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80094c0 con 0x7f99f0163970 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e800c8e0 con 0x7f99f0163970 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0162880 con 0x7f99f0163970 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800cca0 con 0x7f99f0163970 2020-11-16T22:26:58.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.389+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f016f540 con 0x7f99f0163970 2020-11-16T22:26:58.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99f0170040 con 0x7f99f0163970 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f99e8022070 con 0x7f99f0163970 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8055090 0x7f99c8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99e806eaa0 con 0x7f99f0163970 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99f53b1700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8055090 0x7f99c8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f99c805c540 0x7f99c805e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99c805efb0 con 0x7f99c805c540 2020-11-16T22:26:58.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e806ed90 con 0x7f99f0163970 2020-11-16T22:26:58.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f99c805c540 0x7f99c805e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.393+0000 7f99f53b1700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8055090 0x7f99c8057550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f99ec00a610 tx=0x7f99ec008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.397+0000 7f99f4bb0700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f99c805c540 0x7f99c805e960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.397+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== osd.0 v2:172.21.15.181:6808/15203 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f99c805efb0 con 0x7f99c805c540 2020-11-16T22:26:58.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.417+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99f0162a10 con 0x7f99c805c540 2020-11-16T22:26:58.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.417+0000 7f99dd7fa700 1 -- 172.21.15.181:0/3704431232 <== osd.0 v2:172.21.15.181:6808/15203 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99f0162a10 con 0x7f99c805c540 2020-11-16T22:26:58.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f99c805c540 msgr2=0x7f99c805e960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f99c805c540 0x7f99c805e960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8055090 msgr2=0x7f99c8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8055090 0x7f99c8057550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f99ec00a610 tx=0x7f99ec008040).stop 2020-11-16T22:26:58.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 msgr2=0x7f99f016f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 --2- 172.21.15.181:0/3704431232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f0163970 0x7f99f016f050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f99e80091a0 tx=0x7f99e80091d0).stop 2020-11-16T22:26:58.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f5bb2700 1 -- 172.21.15.181:0/3704431232 reap_dead start 2020-11-16T22:26:58.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 shutdown_connections 2020-11-16T22:26:58.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 >> 172.21.15.181:0/3704431232 conn(0x7f99f00cfe90 msgr2=0x7f99f00d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 shutdown_connections 2020-11-16T22:26:58.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.421+0000 7f99f63b3700 1 -- 172.21.15.181:0/3704431232 wait complete. 2020-11-16T22:26:58.423 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:26:58.424 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-16T22:26:58.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.557+0000 7f40afbb2700 1 Processor -- start 2020-11-16T22:26:58.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.557+0000 7f40afbb2700 1 -- start start 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a815d4e0 0x7f40a815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8146e10 0x7f40a80600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40a8147290 0x7f40a80605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a815edb0 con 0x7f40a8147290 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a815e090 con 0x7f40a815d4e0 2020-11-16T22:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a815e330 con 0x7f40a8146e10 2020-11-16T22:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a815d4e0 0x7f40a815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a815d4e0 0x7f40a815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56658/0 (socket says 172.21.15.181:56658) 2020-11-16T22:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 -- 172.21.15.181:0/1143738620 learned_addr learned my addr 172.21.15.181:0/1143738620 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad14d700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8146e10 0x7f40a80600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ae14f700 1 --2- 172.21.15.181:0/1143738620 >> v2:172.21.15.175:3300/0 conn(0x7f40a8147290 0x7f40a80605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 -- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8146e10 msgr2=0x7f40a80600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8146e10 0x7f40a80600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 -- 172.21.15.181:0/1143738620 >> v2:172.21.15.175:3300/0 conn(0x7f40a8147290 msgr2=0x7f40a80605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1143738620 >> v2:172.21.15.175:3300/0 conn(0x7f40a8147290 0x7f40a80605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 -- 172.21.15.181:0/1143738620 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a815e5d0 con 0x7f40a815d4e0 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a815d4e0 0x7f40a815d8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f40a80681b0 tx=0x7f40a4006eb0).ready entity=mon.1 client_cookie=887d40db8c67697f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ac94c700 1 -- 172.21.15.181:0/1143738620 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4009720 con 0x7f40a815d4e0 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ac94c700 1 -- 172.21.15.181:0/1143738620 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4009880 con 0x7f40a815d4e0 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40ac94c700 1 -- 172.21.15.181:0/1143738620 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4009c00 con 0x7f40a815d4e0 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a815d4e0 msgr2=0x7f40a815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a815d4e0 0x7f40a815d8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f40a80681b0 tx=0x7f40a4006eb0).stop 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- 172.21.15.181:0/1143738620 shutdown_connections 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1143738620 >> v2:172.21.15.175:3300/0 conn(0x7f40a8147290 0x7f40a80605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a815d4e0 0x7f40a815d8c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1143738620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8146e10 0x7f40a80600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- 172.21.15.181:0/1143738620 >> 172.21.15.181:0/1143738620 conn(0x7f40a80d3680 msgr2=0x7f40a80d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- 172.21.15.181:0/1143738620 shutdown_connections 2020-11-16T22:26:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.561+0000 7f40afbb2700 1 -- 172.21.15.181:0/1143738620 wait complete. 2020-11-16T22:26:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 Processor -- start 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 -- start start 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 0x7f40a8155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8147290 0x7f40a80d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815d4e0 0x7f40a80d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a80caf30 con 0x7f40a815d4e0 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8145420 con 0x7f40a8146e10 2020-11-16T22:26:58.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40afbb2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a81456c0 con 0x7f40a8147290 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 0x7f40a8155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad14d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8147290 0x7f40a80d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 0x7f40a8155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56664/0 (socket says 172.21.15.181:56664) 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 -- 172.21.15.181:0/1966986481 learned_addr learned my addr 172.21.15.181:0/1966986481 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ae14f700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815d4e0 0x7f40a80d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 -- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8147290 msgr2=0x7f40a80d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a8147290 0x7f40a80d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 -- 172.21.15.181:0/1966986481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815d4e0 msgr2=0x7f40a80d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a815d4e0 0x7f40a80d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8145960 con 0x7f40a8146e10 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.565+0000 7f40ad94e700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 0x7f40a8155b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f40a4006ef0 tx=0x7f40a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4011b40 con 0x7f40a8146e10 2020-11-16T22:26:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4011ca0 con 0x7f40a8146e10 2020-11-16T22:26:58.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a401e670 con 0x7f40a8146e10 2020-11-16T22:26:58.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a8145c00 con 0x7f40a8146e10 2020-11-16T22:26:58.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a8155ff0 con 0x7f40a8146e10 2020-11-16T22:26:58.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40a8156180 con 0x7f40a8146e10 2020-11-16T22:26:58.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f40a402f070 con 0x7f40a8146e10 2020-11-16T22:26:58.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4080055060 0x7f4080057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f40a406fee0 con 0x7f40a8146e10 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40ad14d700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4080055060 0x7f4080057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f408005c5a0 0x7f408005e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f408005f010 con 0x7f408005c5a0 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a4070270 con 0x7f40a8146e10 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40ad14d700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4080055060 0x7f4080057520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f40a8156e70 tx=0x7f4098006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40ae14f700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f408005c5a0 0x7f408005e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.569+0000 7f40ae14f700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f408005c5a0 0x7f408005e9c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.573+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f408005f010 con 0x7f408005c5a0 2020-11-16T22:26:58.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40a80ce740 con 0x7f408005c5a0 2020-11-16T22:26:58.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f4095ffb700 1 -- 172.21.15.181:0/1966986481 <== osd.1 v2:172.21.15.181:6809/15204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40a80ce740 con 0x7f408005c5a0 2020-11-16T22:26:58.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f408005c5a0 msgr2=0x7f408005e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f408005c5a0 0x7f408005e9c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4080055060 msgr2=0x7f4080057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4080055060 0x7f4080057520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f40a8156e70 tx=0x7f4098006df0).stop 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 msgr2=0x7f40a8155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 --2- 172.21.15.181:0/1966986481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a8146e10 0x7f40a8155b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f40a4006ef0 tx=0x7f40a4015000).stop 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40ae14f700 1 -- 172.21.15.181:0/1966986481 reap_dead start 2020-11-16T22:26:58.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 shutdown_connections 2020-11-16T22:26:58.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 >> 172.21.15.181:0/1966986481 conn(0x7f40a80d3680 msgr2=0x7f40a80b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 shutdown_connections 2020-11-16T22:26:58.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.589+0000 7f40afbb2700 1 -- 172.21.15.181:0/1966986481 wait complete. 2020-11-16T22:26:58.589 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:26:58.590 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-16T22:26:58.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 Processor -- start 2020-11-16T22:26:58.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 -- start start 2020-11-16T22:26:58.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65440816e0 0x7f6544145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6544061b50 con 0x7f6544145a30 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65440618b0 con 0x7f65440816e0 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65440cb930 con 0x7f65441492c0 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36286/0 (socket says 172.21.15.181:36286) 2020-11-16T22:26:58.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 -- 172.21.15.181:0/1172096942 learned_addr learned my addr 172.21.15.181:0/1172096942 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6542ffd700 1 --2- 172.21.15.181:0/1172096942 >> v2:172.21.15.175:3300/0 conn(0x7f65440816e0 0x7f6544145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 -- 172.21.15.181:0/1172096942 >> v2:172.21.15.175:3300/0 conn(0x7f65440816e0 msgr2=0x7f6544145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 --2- 172.21.15.181:0/1172096942 >> v2:172.21.15.175:3300/0 conn(0x7f65440816e0 0x7f6544145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 -- 172.21.15.181:0/1172096942 >> v1:172.21.15.175:6789/0 conn(0x7f6544145a30 legacy=0x7f6544080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 -- 172.21.15.181:0/1172096942 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440cbbd0 con 0x7f65441492c0 2020-11-16T22:26:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6543fff700 1 --2- 172.21.15.181:0/1172096942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6534009d50 tx=0x7f6534009290).ready entity=mon.2 client_cookie=698f3ccc4f85ab7d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f65427fc700 1 -- 172.21.15.181:0/1172096942 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400bbc0 con 0x7f65441492c0 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f65427fc700 1 -- 172.21.15.181:0/1172096942 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400bd20 con 0x7f65441492c0 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f65427fc700 1 -- 172.21.15.181:0/1172096942 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534011610 con 0x7f65441492c0 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 -- 172.21.15.181:0/1172096942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 msgr2=0x7f65440cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.733+0000 7f6548af0700 1 --2- 172.21.15.181:0/1172096942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6534009d50 tx=0x7f6534009290).stop 2020-11-16T22:26:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- 172.21.15.181:0/1172096942 shutdown_connections 2020-11-16T22:26:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 --2- 172.21.15.181:0/1172096942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65441492c0 0x7f65440cc5b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 --2- 172.21.15.181:0/1172096942 >> v2:172.21.15.175:3300/0 conn(0x7f65440816e0 0x7f6544145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- 172.21.15.181:0/1172096942 >> 172.21.15.181:0/1172096942 conn(0x7f65440d68c0 msgr2=0x7f65440d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- 172.21.15.181:0/1172096942 shutdown_connections 2020-11-16T22:26:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- 172.21.15.181:0/1172096942 wait complete. 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 Processor -- start 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- start start 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 0x7f65440b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544145a30 0x7f654413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65441492c0 0x7f654413fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654415ef80 con 0x7f65441492c0 2020-11-16T22:26:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65441431c0 con 0x7f65440816e0 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6548af0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6544143460 con 0x7f6544145a30 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6542ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544145a30 0x7f654413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f65437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 0x7f65440b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f65437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 0x7f65440b9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56680/0 (socket says 172.21.15.181:56680) 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6542ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544145a30 0x7f654413f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36292/0 (socket says 172.21.15.181:36292) 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f65437fe700 1 -- 172.21.15.181:0/3754054510 learned_addr learned my addr 172.21.15.181:0/3754054510 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.737+0000 7f6543fff700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65441492c0 0x7f654413fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 -- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544145a30 msgr2=0x7f654413f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544145a30 0x7f654413f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 -- 172.21.15.181:0/3754054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65441492c0 msgr2=0x7f654413fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65441492c0 0x7f654413fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654415f1b0 con 0x7f65440816e0 2020-11-16T22:26:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f65437fe700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 0x7f65440b9e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f653c00c740 tx=0x7f653c007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c00e490 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c00e5f0 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c00e9b0 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654415f450 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654415cbb0 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6544044f60 con 0x7f65440816e0 2020-11-16T22:26:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f653c019030 con 0x7f65440816e0 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6514055060 0x7f6514057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f653c06d130 con 0x7f65440816e0 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f6542ffd700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6514055060 0x7f6514057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7f651405c5a0 0x7f651405e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f651405f010 con 0x7f651405c5a0 2020-11-16T22:26:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.741+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f653c06f7a0 con 0x7f65440816e0 2020-11-16T22:26:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.745+0000 7f6542ffd700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6514055060 0x7f6514057520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6544160be0 tx=0x7f652c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.745+0000 7f6543fff700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7f651405c5a0 0x7f651405e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.745+0000 7f6543fff700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7f651405c5a0 0x7f651405e9c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.745+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f651405f010 con 0x7f651405c5a0 2020-11-16T22:26:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.765+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6544145280 con 0x7f651405c5a0 2020-11-16T22:26:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.765+0000 7f653b7fe700 1 -- 172.21.15.181:0/3754054510 <== osd.2 v2:172.21.15.181:6800/15202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6544145280 con 0x7f651405c5a0 2020-11-16T22:26:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7f651405c5a0 msgr2=0x7f651405e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7f651405c5a0 0x7f651405e9c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6514055060 msgr2=0x7f6514057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6514055060 0x7f6514057520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6544160be0 tx=0x7f652c006df0).stop 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 msgr2=0x7f65440b9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 --2- 172.21.15.181:0/3754054510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440816e0 0x7f65440b9e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f653c00c740 tx=0x7f653c007c10).stop 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6543fff700 1 -- 172.21.15.181:0/3754054510 reap_dead start 2020-11-16T22:26:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 shutdown_connections 2020-11-16T22:26:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 >> 172.21.15.181:0/3754054510 conn(0x7f65440d68c0 msgr2=0x7f654415e000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 shutdown_connections 2020-11-16T22:26:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.769+0000 7f6548af0700 1 -- 172.21.15.181:0/3754054510 wait complete. 2020-11-16T22:26:58.767 INFO:teuthology.orchestra.run.smithi181.stdout:60129542151 2020-11-16T22:26:58.768 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2020-11-16T22:26:58.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 Processor -- start 2020-11-16T22:26:58.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- start start 2020-11-16T22:26:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae54161480 0x7fae54164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae540c8db0 con 0x7fae54161860 2020-11-16T22:26:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae540c8330 con 0x7fae54164e10 2020-11-16T22:26:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae540c85d0 con 0x7fae54161480 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae54161480 0x7fae54164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae52ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae54164e10 0x7fae54062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42762/0 (socket says 172.21.15.181:42762) 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae52ffd700 1 -- 172.21.15.181:0/3297776905 learned_addr learned my addr 172.21.15.181:0/3297776905 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae527fc700 1 -- 172.21.15.181:0/3297776905 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089123742 0 0) 0x7fae540c8330 con 0x7fae54164e10 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae537fe700 1 -- 172.21.15.181:0/3297776905 >> v1:172.21.15.181:6789/0 conn(0x7fae54164e10 legacy=0x7fae54062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae537fe700 1 -- 172.21.15.181:0/3297776905 >> v1:172.21.15.175:6789/0 conn(0x7fae54161860 legacy=0x7fae540644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae537fe700 1 -- 172.21.15.181:0/3297776905 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540c8870 con 0x7fae54161480 2020-11-16T22:26:58.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae537fe700 1 --2- 172.21.15.181:0/3297776905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae54161480 0x7fae54164920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fae3c00b9a0 tx=0x7fae3c006df0).ready entity=mon.1 client_cookie=8f0e0ea70b920f8e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae527fc700 1 -- 172.21.15.181:0/3297776905 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0097b0 con 0x7fae54161480 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae527fc700 1 -- 172.21.15.181:0/3297776905 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c009910 con 0x7fae54161480 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae527fc700 1 -- 172.21.15.181:0/3297776905 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c009c90 con 0x7fae54161480 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- 172.21.15.181:0/3297776905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae54161480 msgr2=0x7fae54164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 --2- 172.21.15.181:0/3297776905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae54161480 0x7fae54164920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fae3c00b9a0 tx=0x7fae3c006df0).stop 2020-11-16T22:26:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- 172.21.15.181:0/3297776905 shutdown_connections 2020-11-16T22:26:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 --2- 172.21.15.181:0/3297776905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae54161480 0x7fae54164920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- 172.21.15.181:0/3297776905 >> 172.21.15.181:0/3297776905 conn(0x7fae540d2dc0 msgr2=0x7fae540ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- 172.21.15.181:0/3297776905 shutdown_connections 2020-11-16T22:26:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.901+0000 7fae58867700 1 -- 172.21.15.181:0/3297776905 wait complete. 2020-11-16T22:26:58.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 Processor -- start 2020-11-16T22:26:58.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 -- start start 2020-11-16T22:26:58.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 0x7fae54143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae54161860 0x7fae54177980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae540cbf50 0x7fae54174ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae541442d0 con 0x7fae54161860 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae54144570 con 0x7fae540cbf50 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae58867700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae54144810 con 0x7fae54161480 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 0x7fae54143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 0x7fae54143d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36306/0 (socket says 172.21.15.181:36306) 2020-11-16T22:26:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 -- 172.21.15.181:0/1064689433 learned_addr learned my addr 172.21.15.181:0/1064689433 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae53fff700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae540cbf50 0x7fae54174ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae52ffd700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae54161860 0x7fae54177980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 -- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae540cbf50 msgr2=0x7fae54174ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae540cbf50 0x7fae54174ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 -- 172.21.15.181:0/1064689433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae54161860 msgr2=0x7fae54177980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae54161860 0x7fae54177980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae537fe700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54176fa0 con 0x7fae54161480 2020-11-16T22:26:58.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.905+0000 7fae53fff700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae540cbf50 0x7fae54174ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:58.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae537fe700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 0x7fae54143d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fae3c006e30 tx=0x7fae3c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01e460 con 0x7fae54161480 2020-11-16T22:26:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c01e5e0 con 0x7fae54161480 2020-11-16T22:26:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae58867700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae54177240 con 0x7fae54161480 2020-11-16T22:26:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01e9a0 con 0x7fae54161480 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae58867700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae540cc3d0 con 0x7fae54161480 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae24000f80 con 0x7fae54161480 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fae3c025030 con 0x7fae54161480 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fae30055090 0x7fae30057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fae3c06f630 con 0x7fae54161480 2020-11-16T22:26:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae52ffd700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fae30055090 0x7fae30057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7fae3005c5d0 0x7fae3005e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae3005f040 con 0x7fae3005c5d0 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3c071ca0 con 0x7fae54161480 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae52ffd700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fae30055090 0x7fae30057550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fae54143cd0 tx=0x7fae44007940).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae53fff700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7fae3005c5d0 0x7fae3005e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.909+0000 7fae53fff700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7fae3005c5d0 0x7fae3005e9f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:58.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.913+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fae3005f040 con 0x7fae3005c5d0 2020-11-16T22:26:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.929+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae240029d0 con 0x7fae3005c5d0 2020-11-16T22:26:58.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.929+0000 7fae4b7fe700 1 -- 172.21.15.181:0/1064689433 <== osd.3 v2:172.21.15.181:6824/15205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fae240029d0 con 0x7fae3005c5d0 2020-11-16T22:26:58.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7fae3005c5d0 msgr2=0x7fae3005e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7fae3005c5d0 0x7fae3005e9f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fae30055090 msgr2=0x7fae30057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fae30055090 0x7fae30057550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fae54143cd0 tx=0x7fae44007940).stop 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 msgr2=0x7fae54143d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 --2- 172.21.15.181:0/1064689433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae54161480 0x7fae54143d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fae3c006e30 tx=0x7fae3c015000).stop 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae53fff700 1 -- 172.21.15.181:0/1064689433 reap_dead start 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 shutdown_connections 2020-11-16T22:26:58.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 >> 172.21.15.181:0/1064689433 conn(0x7fae540d2dc0 msgr2=0x7fae54062f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:58.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 shutdown_connections 2020-11-16T22:26:58.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:58.933+0000 7fae497fa700 1 -- 172.21.15.181:0/1064689433 wait complete. 2020-11-16T22:26:58.931 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:26:58.932 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2020-11-16T22:26:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 Processor -- start 2020-11-16T22:26:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 -- start start 2020-11-16T22:26:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 0x7fd284163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd2840600f0 0x7fd2840605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840cfc60 con 0x7fd284163970 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd284065190 con 0x7fd2840600f0 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28a48d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd284065430 con 0x7fd2840cf390 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36316/0 (socket says 172.21.15.181:36316) 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 -- 172.21.15.181:0/3974648131 learned_addr learned my addr 172.21.15.181:0/3974648131 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd289c8c700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.181:3300/0 conn(0x7fd2840600f0 0x7fd2840605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd288c8a700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 0x7fd284163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 -- 172.21.15.181:0/3974648131 >> v2:172.21.15.181:3300/0 conn(0x7fd2840600f0 msgr2=0x7fd2840605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.181:3300/0 conn(0x7fd2840600f0 0x7fd2840605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 -- 172.21.15.181:0/3974648131 >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 msgr2=0x7fd284163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 0x7fd284163df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 -- 172.21.15.181:0/3974648131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840656d0 con 0x7fd2840cf390 2020-11-16T22:26:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd288c8a700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 0x7fd284163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd28948b700 1 --2- 172.21.15.181:0/3974648131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd284142f80 tx=0x7fd274009290).ready entity=mon.2 client_cookie=ae05b502b94959ea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd283fff700 1 -- 172.21.15.181:0/3974648131 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400bbc0 con 0x7fd2840cf390 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd283fff700 1 -- 172.21.15.181:0/3974648131 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27400bd20 con 0x7fd2840cf390 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.065+0000 7fd283fff700 1 -- 172.21.15.181:0/3974648131 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274011610 con 0x7fd2840cf390 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- 172.21.15.181:0/3974648131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 msgr2=0x7fd2840cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 --2- 172.21.15.181:0/3974648131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd284142f80 tx=0x7fd274009290).stop 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- 172.21.15.181:0/3974648131 shutdown_connections 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.181:3300/0 conn(0x7fd2840600f0 0x7fd2840605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 --2- 172.21.15.181:0/3974648131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840cf390 0x7fd2840cf770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 --2- 172.21.15.181:0/3974648131 >> v2:172.21.15.175:3300/0 conn(0x7fd284163970 0x7fd284163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- 172.21.15.181:0/3974648131 >> 172.21.15.181:0/3974648131 conn(0x7fd2840cfe90 msgr2=0x7fd2840d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- 172.21.15.181:0/3974648131 shutdown_connections 2020-11-16T22:26:59.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- 172.21.15.181:0/3974648131 wait complete. 2020-11-16T22:26:59.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 Processor -- start 2020-11-16T22:26:59.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.069+0000 7fd28a48d700 1 -- start start 2020-11-16T22:26:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840600f0 0x7fd28416f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2840cf390 0x7fd2840ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd284163970 0x7fd284065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28416f9c0 con 0x7fd284163970 2020-11-16T22:26:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28416fc60 con 0x7fd2840cf390 2020-11-16T22:26:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28a48d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28416ff00 con 0x7fd2840600f0 2020-11-16T22:26:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28948b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840600f0 0x7fd28416f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2840cf390 0x7fd2840ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28948b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840600f0 0x7fd28416f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36322/0 (socket says 172.21.15.181:36322) 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd28948b700 1 -- 172.21.15.181:0/1221655713 learned_addr learned my addr 172.21.15.181:0/1221655713 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 -- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840600f0 msgr2=0x7fd28416f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2840600f0 0x7fd28416f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 -- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd284163970 msgr2=0x7fd284065fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd284163970 0x7fd284065fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2841701a0 con 0x7fd2840cf390 2020-11-16T22:26:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.073+0000 7fd288c8a700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2840cf390 0x7fd2840ccec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd27800b900 tx=0x7fd278008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800f5e0 con 0x7fd2840cf390 2020-11-16T22:26:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800f740 con 0x7fd2840cf390 2020-11-16T22:26:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800fac0 con 0x7fd2840cf390 2020-11-16T22:26:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284170790 con 0x7fd2840cf390 2020-11-16T22:26:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28416f710 con 0x7fd2840cf390 2020-11-16T22:26:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd284170620 con 0x7fd2840cf390 2020-11-16T22:26:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd27801c460 con 0x7fd2840cf390 2020-11-16T22:26:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd25c055060 0x7fd25c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd28948b700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd25c055060 0x7fd25c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd27806e120 con 0x7fd2840cf390 2020-11-16T22:26:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7fd25c05c520 0x7fd25c05e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.077+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd25c05ef90 con 0x7fd25c05c520 2020-11-16T22:26:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.081+0000 7fd28948b700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd25c055060 0x7fd25c057520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd2740090f0 tx=0x7fd274016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.081+0000 7fd289c8c700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7fd25c05c520 0x7fd25c05e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.081+0000 7fd289c8c700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7fd25c05c520 0x7fd25c05e940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.081+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== osd.4 v2:172.21.15.175:6800/15329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fd25c05ef90 con 0x7fd25c05c520 2020-11-16T22:26:59.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.085+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd278044210 con 0x7fd2840cf390 2020-11-16T22:26:59.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2840cd670 con 0x7fd25c05c520 2020-11-16T22:26:59.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd2817fa700 1 -- 172.21.15.181:0/1221655713 <== osd.4 v2:172.21.15.175:6800/15329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd2840cd670 con 0x7fd25c05c520 2020-11-16T22:26:59.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7fd25c05c520 msgr2=0x7fd25c05e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7fd25c05c520 0x7fd25c05e940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd25c055060 msgr2=0x7fd25c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd25c055060 0x7fd25c057520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd2740090f0 tx=0x7fd274016000).stop 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2840cf390 msgr2=0x7fd2840ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 --2- 172.21.15.181:0/1221655713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2840cf390 0x7fd2840ccec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd27800b900 tx=0x7fd278008040).stop 2020-11-16T22:26:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd289c8c700 1 -- 172.21.15.181:0/1221655713 reap_dead start 2020-11-16T22:26:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 shutdown_connections 2020-11-16T22:26:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 >> 172.21.15.181:0/1221655713 conn(0x7fd2840cfe90 msgr2=0x7fd2840b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 shutdown_connections 2020-11-16T22:26:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.105+0000 7fd28a48d700 1 -- 172.21.15.181:0/1221655713 wait complete. 2020-11-16T22:26:59.103 INFO:teuthology.orchestra.run.smithi181.stdout:55834574854 2020-11-16T22:26:59.104 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.237+0000 7fc64cb74700 1 Processor -- start 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.237+0000 7fc64cb74700 1 -- start start 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 0x7fc648167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc648168390 con 0x7fc648167ac0 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480b5f00 con 0x7fc648166bb0 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480b2060 con 0x7fc6480b3930 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc646d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 0x7fc648167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36342/0 (socket says 172.21.15.181:36342) 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 -- 172.21.15.181:0/3565798465 learned_addr learned my addr 172.21.15.181:0/3565798465 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 -- 172.21.15.181:0/3565798465 >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 msgr2=0x7fc648167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 --2- 172.21.15.181:0/3565798465 >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 0x7fc648167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 -- 172.21.15.181:0/3565798465 >> v1:172.21.15.175:6789/0 conn(0x7fc648167ac0 legacy=0x7fc6480b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 -- 172.21.15.181:0/3565798465 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480b2300 con 0x7fc6480b3930 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc646d9d700 1 --2- 172.21.15.181:0/3565798465 >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 0x7fc648167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc645d9b700 1 --2- 172.21.15.181:0/3565798465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc630009d50 tx=0x7fc630009290).ready entity=mon.2 client_cookie=52a2066bcc207a94 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64559a700 1 -- 172.21.15.181:0/3565798465 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63000bbc0 con 0x7fc6480b3930 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64559a700 1 -- 172.21.15.181:0/3565798465 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63000bd20 con 0x7fc6480b3930 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64559a700 1 -- 172.21.15.181:0/3565798465 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630011610 con 0x7fc6480b3930 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- 172.21.15.181:0/3565798465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 msgr2=0x7fc648167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3565798465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc630009d50 tx=0x7fc630009290).stop 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- 172.21.15.181:0/3565798465 shutdown_connections 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3565798465 >> v2:172.21.15.181:3300/0 conn(0x7fc648166bb0 0x7fc648167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3565798465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648167ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- 172.21.15.181:0/3565798465 >> 172.21.15.181:0/3565798465 conn(0x7fc648060bc0 msgr2=0x7fc6480d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- 172.21.15.181:0/3565798465 shutdown_connections 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.241+0000 7fc64cb74700 1 -- 172.21.15.181:0/3565798465 wait complete. 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 Processor -- start 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- start start 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648062950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 0x7fc648064130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc648167ac0 0x7fc648177600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648062e40 con 0x7fc648167ac0 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648061a30 con 0x7fc648166bb0 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc648061cd0 con 0x7fc6480b3930 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 0x7fc648064130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 0x7fc648064130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56738/0 (socket says 172.21.15.181:56738) 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648062950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 -- 172.21.15.181:0/3114721896 learned_addr learned my addr 172.21.15.181:0/3114721896 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc646d9d700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc648167ac0 0x7fc648177600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 -- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 msgr2=0x7fc648062950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6480b3930 0x7fc648062950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 -- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc648167ac0 msgr2=0x7fc648177600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc648167ac0 0x7fc648177600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648061f70 con 0x7fc648166bb0 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc645d9b700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 0x7fc648064130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fc648168990 tx=0x7fc630016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63001b5e0 con 0x7fc648166bb0 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63001b740 con 0x7fc648166bb0 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc648062210 con 0x7fc648166bb0 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63001bac0 con 0x7fc648166bb0 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc648064620 con 0x7fc648166bb0 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.245+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc648064870 con 0x7fc648166bb0 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc630023030 con 0x7fc648166bb0 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc618055060 0x7fc618057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc63006ef70 con 0x7fc648166bb0 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc64659c700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc618055060 0x7fc618057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7fc61805c5a0 0x7fc61805e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc61805f010 con 0x7fc61805c5a0 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc630071580 con 0x7fc648166bb0 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc64659c700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc618055060 0x7fc618057520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc638003300 tx=0x7fc638007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc646d9d700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7fc61805c5a0 0x7fc61805e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.249+0000 7fc646d9d700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7fc61805c5a0 0x7fc61805e9c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.253+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fc61805f010 con 0x7fc61805c5a0 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.273+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc648177db0 con 0x7fc61805c5a0 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.273+0000 7fc63e7fc700 1 -- 172.21.15.181:0/3114721896 <== osd.5 v2:172.21.15.175:6823/15351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc648177db0 con 0x7fc61805c5a0 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7fc61805c5a0 msgr2=0x7fc61805e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7fc61805c5a0 0x7fc61805e9c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc618055060 msgr2=0x7fc618057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc618055060 0x7fc618057520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc638003300 tx=0x7fc638007b90).stop 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 msgr2=0x7fc648064130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 --2- 172.21.15.181:0/3114721896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc648166bb0 0x7fc648064130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fc648168990 tx=0x7fc630016000).stop 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc646d9d700 1 -- 172.21.15.181:0/3114721896 reap_dead start 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 shutdown_connections 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 >> 172.21.15.181:0/3114721896 conn(0x7fc648060bc0 msgr2=0x7fc648162580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 shutdown_connections 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.277+0000 7fc64cb74700 1 -- 172.21.15.181:0/3114721896 wait complete. 2020-11-16T22:26:59.266 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:26:59.280+0000 7fa5ccdc1700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2020-11-16T22:26:59.266 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:26:59.280+0000 7fa5ccdc1700 -1 mds.pinger is_rank_lagging: rank=5 was never sent ping request. 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 Processor -- start 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- start start 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 0x7fd0ec080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ec061b50 con 0x7fd0ec0814c0 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ec0618b0 con 0x7fd0ec1492c0 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0cb930 con 0x7fd0ec145a30 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36358/0 (socket says 172.21.15.181:36358) 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 -- 172.21.15.181:0/1327084119 learned_addr learned my addr 172.21.15.181:0/1327084119 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f0d8c700 1 --2- 172.21.15.181:0/1327084119 >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 0x7fd0ec080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 -- 172.21.15.181:0/1327084119 >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 msgr2=0x7fd0ec080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 --2- 172.21.15.181:0/1327084119 >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 0x7fd0ec080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 -- 172.21.15.181:0/1327084119 >> v1:172.21.15.175:6789/0 conn(0x7fd0ec0814c0 legacy=0x7fd0ec0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 -- 172.21.15.181:0/1327084119 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0cbbd0 con 0x7fd0ec145a30 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f0d8c700 1 --2- 172.21.15.181:0/1327084119 >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 0x7fd0ec080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f158d700 1 --2- 172.21.15.181:0/1327084119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd0e0006fd0 tx=0x7fd0e0006eb0).ready entity=mon.2 client_cookie=c61a53da3c8826b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0dffff700 1 -- 172.21.15.181:0/1327084119 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0009890 con 0x7fd0ec145a30 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0dffff700 1 -- 172.21.15.181:0/1327084119 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e00099f0 con 0x7fd0ec145a30 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0dffff700 1 -- 172.21.15.181:0/1327084119 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0009d70 con 0x7fd0ec145a30 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- 172.21.15.181:0/1327084119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 msgr2=0x7fd0ec145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 --2- 172.21.15.181:0/1327084119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd0e0006fd0 tx=0x7fd0e0006eb0).stop 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- 172.21.15.181:0/1327084119 shutdown_connections 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 --2- 172.21.15.181:0/1327084119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec145a30 0x7fd0ec145e10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 --2- 172.21.15.181:0/1327084119 >> v2:172.21.15.175:3300/0 conn(0x7fd0ec1492c0 0x7fd0ec080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- 172.21.15.181:0/1327084119 >> 172.21.15.181:0/1327084119 conn(0x7fd0ec0d68c0 msgr2=0x7fd0ec0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- 172.21.15.181:0/1327084119 shutdown_connections 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.413+0000 7fd0f258f700 1 -- 172.21.15.181:0/1327084119 wait complete. 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 Processor -- start 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 -- start start 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 0x7fd0ec0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0ec145a30 0x7fd0ec0d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0ec1492c0 0x7fd0ec0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1470b0 con 0x7fd0ec1492c0 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec147350 con 0x7fd0ec145a30 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0d2700 con 0x7fd0ec0814c0 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 0x7fd0ec0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 0x7fd0ec0d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36362/0 (socket says 172.21.15.181:36362) 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 -- 172.21.15.181:0/11270588 learned_addr learned my addr 172.21.15.181:0/11270588 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f0d8c700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0ec145a30 0x7fd0ec0d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f1d8e700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0ec1492c0 0x7fd0ec0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 -- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0ec145a30 msgr2=0x7fd0ec0d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0ec145a30 0x7fd0ec0d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 -- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0ec1492c0 msgr2=0x7fd0ec0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0ec1492c0 0x7fd0ec0d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0d29a0 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f158d700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 0x7fd0ec0d30e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd0ec0d78a0 tx=0x7fd0e0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0006800 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.417+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec0d2c40 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e0011dd0 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec0d21d0 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e001e670 con 0x7fd0ec0814c0 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0ec0d2560 con 0x7fd0ec0814c0 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd0e0025030 con 0x7fd0ec0814c0 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd0c4055060 0x7fd0c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd0e0070510 con 0x7fd0ec0814c0 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7fd0c405c520 0x7fd0c405e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0c405ef90 con 0x7fd0c405c520 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e0070910 con 0x7fd0ec0814c0 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f0d8c700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd0c4055060 0x7fd0c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f0d8c700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd0c4055060 0x7fd0c4057520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd0ec060e00 tx=0x7fd0e8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f1d8e700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7fd0c405c520 0x7fd0c405e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.421+0000 7fd0f1d8e700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7fd0c405c520 0x7fd0c405e940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.425+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== osd.6 v2:172.21.15.175:6808/15330 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fd0c405ef90 con 0x7fd0c405c520 2020-11-16T22:26:59.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0ec147510 con 0x7fd0c405c520 2020-11-16T22:26:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0dd7fa700 1 -- 172.21.15.181:0/11270588 <== osd.6 v2:172.21.15.175:6808/15330 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0ec147510 con 0x7fd0c405c520 2020-11-16T22:26:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7fd0c405c520 msgr2=0x7fd0c405e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7fd0c405c520 0x7fd0c405e940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd0c4055060 msgr2=0x7fd0c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd0c4055060 0x7fd0c4057520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd0ec060e00 tx=0x7fd0e8008040).stop 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 msgr2=0x7fd0ec0d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 --2- 172.21.15.181:0/11270588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0ec0814c0 0x7fd0ec0d30e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd0ec0d78a0 tx=0x7fd0e0015000).stop 2020-11-16T22:26:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f1d8e700 1 -- 172.21.15.181:0/11270588 reap_dead start 2020-11-16T22:26:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 shutdown_connections 2020-11-16T22:26:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 >> 172.21.15.181:0/11270588 conn(0x7fd0ec0d68c0 msgr2=0x7fd0ec13ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 shutdown_connections 2020-11-16T22:26:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.445+0000 7fd0f258f700 1 -- 172.21.15.181:0/11270588 wait complete. 2020-11-16T22:26:59.449 INFO:teuthology.orchestra.run.smithi181.stdout:55834574854 2020-11-16T22:26:59.449 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2020-11-16T22:26:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.589+0000 7f571a2e5700 1 Processor -- start 2020-11-16T22:26:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.589+0000 7f571a2e5700 1 -- start start 2020-11-16T22:26:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5714160690 0x7f5714066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571415e2e0 con 0x7f57140b2190 2020-11-16T22:26:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57140ce460 con 0x7f571415de80 2020-11-16T22:26:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5714066a60 con 0x7f5714160690 2020-11-16T22:26:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f571415de80 0x7f571415ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42834/0 (socket says 172.21.15.181:42834) 2020-11-16T22:26:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 -- 172.21.15.181:0/1922544492 learned_addr learned my addr 172.21.15.181:0/1922544492 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:26:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5719ae4700 1 --2- 172.21.15.181:0/1922544492 >> v2:172.21.15.181:3300/0 conn(0x7f5714160690 0x7f5714066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639100814 0 0) 0x7f57140ce460 con 0x7f571415de80 2020-11-16T22:26:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc003240 con 0x7f571415de80 2020-11-16T22:26:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2229602137 0 0) 0x7f56fc003240 con 0x7f571415de80 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 >> v2:172.21.15.181:3300/0 conn(0x7f5714160690 msgr2=0x7f5714066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 --2- 172.21.15.181:0/1922544492 >> v2:172.21.15.181:3300/0 conn(0x7f5714160690 0x7f5714066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 >> v1:172.21.15.175:6789/0 conn(0x7f57140b2190 legacy=0x7f57140cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140ce700 con 0x7f571415de80 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f57040014e0 con 0x7f571415de80 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 >> v1:172.21.15.181:6789/0 conn(0x7f571415de80 legacy=0x7f571415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc003730 0x7f56fc003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 0x7f56fc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56fc004e50 0x7f56fc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714066a60 con 0x7f56fc004e50 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc0041d0 con 0x7f56fc0044e0 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc00de30 con 0x7f56fc003730 2020-11-16T22:26:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5719ae4700 1 --2- 172.21.15.181:0/1922544492 >> v2:172.21.15.181:3300/0 conn(0x7f5714160690 0x7f5714066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 0x7f56fc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5719ae4700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56fc004e50 0x7f56fc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f57192e3700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc003730 0x7f56fc003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 -- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc003730 msgr2=0x7f56fc003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc003730 0x7f56fc003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 -- 172.21.15.181:0/1922544492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56fc004e50 msgr2=0x7f56fc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56fc004e50 0x7f56fc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 -- 172.21.15.181:0/1922544492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc003240 con 0x7f56fc0044e0 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5719ae4700 1 -- 172.21.15.181:0/1922544492 reap_dead start 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5718ae2700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 0x7f56fc004960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f570400c440 tx=0x7f571415a510).ready entity=mon.1 client_cookie=ebf71fa92aa66607 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400ce20 con 0x7f56fc0044e0 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57040070d0 con 0x7f56fc0044e0 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f5703fff700 1 -- 172.21.15.181:0/1922544492 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704007490 con 0x7f56fc0044e0 2020-11-16T22:26:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 msgr2=0x7f56fc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 0x7f56fc004960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f570400c440 tx=0x7f571415a510).stop 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- 172.21.15.181:0/1922544492 shutdown_connections 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 --2- 172.21.15.181:0/1922544492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc0044e0 0x7f56fc004960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- 172.21.15.181:0/1922544492 >> 172.21.15.181:0/1922544492 conn(0x7f57140cfe90 msgr2=0x7f57140d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- 172.21.15.181:0/1922544492 shutdown_connections 2020-11-16T22:26:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.593+0000 7f571a2e5700 1 -- 172.21.15.181:0/1922544492 wait complete. 2020-11-16T22:26:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.597+0000 7f571a2e5700 1 Processor -- start 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.597+0000 7f571a2e5700 1 -- start start 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc0044e0 0x7f57140b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 0x7f5714134670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f571416c270 0x7f57140610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b7370 con 0x7f571416c270 2020-11-16T22:26:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b7610 con 0x7f5714159980 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b78b0 con 0x7f56fc0044e0 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 0x7f5714134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f57192e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc0044e0 0x7f57140b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 0x7f5714134670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56776/0 (socket says 172.21.15.181:56776) 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 -- 172.21.15.181:0/4247436285 learned_addr learned my addr 172.21.15.181:0/4247436285 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5719ae4700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f571416c270 0x7f57140610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 -- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc0044e0 msgr2=0x7f57140b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc0044e0 0x7f57140b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 -- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f571416c270 msgr2=0x7f57140610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f571416c270 0x7f57140610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140b7b50 con 0x7f5714159980 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5719ae4700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f571416c270 0x7f57140610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:26:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f57192e3700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56fc0044e0 0x7f57140b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f5718ae2700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 0x7f5714134670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f56fc004360 tx=0x7f5704015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704006c70 con 0x7f5714159980 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704006dd0 con 0x7f5714159980 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704005030 con 0x7f5714159980 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57140b7df0 con 0x7f5714159980 2020-11-16T22:26:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714134b60 con 0x7f5714159980 2020-11-16T22:26:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.601+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5714044f60 con 0x7f5714159980 2020-11-16T22:26:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5704010070 con 0x7f5714159980 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56f0055060 0x7f56f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f570406f080 con 0x7f5714159980 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57192e3700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56f0055060 0x7f56f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7f56f005c520 0x7f56f005e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56f005ef90 con 0x7f56f005c520 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f570406f480 con 0x7f5714159980 2020-11-16T22:26:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f57192e3700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56f0055060 0x7f56f0057520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5710000bc0 tx=0x7f5710007ae0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f5719ae4700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7f56f005c520 0x7f56f005e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.605+0000 7f5719ae4700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7f56f005c520 0x7f56f005e940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.609+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f56f005ef90 con 0x7f56f005c520 2020-11-16T22:26:59.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57140b8290 con 0x7f56f005c520 2020-11-16T22:26:59.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f57017fa700 1 -- 172.21.15.181:0/4247436285 <== osd.7 v2:172.21.15.175:6816/15354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f570c009b40 con 0x7f56f005c520 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7f56f005c520 msgr2=0x7f56f005e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7f56f005c520 0x7f56f005e940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56f0055060 msgr2=0x7f56f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56f0055060 0x7f56f0057520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5710000bc0 tx=0x7f5710007ae0).stop 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 msgr2=0x7f5714134670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 --2- 172.21.15.181:0/4247436285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714159980 0x7f5714134670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f56fc004360 tx=0x7f5704015000).stop 2020-11-16T22:26:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f5719ae4700 1 -- 172.21.15.181:0/4247436285 reap_dead start 2020-11-16T22:26:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 shutdown_connections 2020-11-16T22:26:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 >> 172.21.15.181:0/4247436285 conn(0x7f57140cfe90 msgr2=0x7f57140600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 shutdown_connections 2020-11-16T22:26:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.625+0000 7f571a2e5700 1 -- 172.21.15.181:0/4247436285 wait complete. 2020-11-16T22:26:59.625 INFO:teuthology.orchestra.run.smithi181.stdout:60129542151 2020-11-16T22:26:59.625 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:26:59.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 Processor -- start 2020-11-16T22:26:59.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- start start 2020-11-16T22:26:59.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff764145a30 0x7ff764145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 0x7ff764080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff764061b50 con 0x7ff7640814c0 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640618b0 con 0x7ff764145a30 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640cb930 con 0x7ff7641492c0 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff7694e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 0x7ff764080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff7694e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 0x7ff764080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36396/0 (socket says 172.21.15.181:36396) 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff764145a30 0x7ff764145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76a4e2700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7ff7640814c0 0x7ff7640cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:35972/0 (socket says 172.21.15.181:35972) 2020-11-16T22:26:59.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff7694e0700 1 -- 172.21.15.181:0/2792070281 learned_addr learned my addr 172.21.15.181:0/2792070281 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 -- 172.21.15.181:0/2792070281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 msgr2=0x7ff764080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:26:59.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 --2- 172.21.15.181:0/2792070281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 0x7ff764080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 -- 172.21.15.181:0/2792070281 >> v1:172.21.15.175:6789/0 conn(0x7ff7640814c0 legacy=0x7ff7640cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 -- 172.21.15.181:0/2792070281 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640cbbd0 con 0x7ff764145a30 2020-11-16T22:26:59.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff769ce1700 1 --2- 172.21.15.181:0/2792070281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764145e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ff76000b9a0 tx=0x7ff760006df0).ready entity=mon.0 client_cookie=7780655f3efeeeee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff768cdf700 1 -- 172.21.15.181:0/2792070281 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600097b0 con 0x7ff764145a30 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff768cdf700 1 -- 172.21.15.181:0/2792070281 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760009910 con 0x7ff764145a30 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff768cdf700 1 -- 172.21.15.181:0/2792070281 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760009c90 con 0x7ff764145a30 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- 172.21.15.181:0/2792070281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 msgr2=0x7ff764145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 --2- 172.21.15.181:0/2792070281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764145e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ff76000b9a0 tx=0x7ff760006df0).stop 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- 172.21.15.181:0/2792070281 shutdown_connections 2020-11-16T22:26:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 --2- 172.21.15.181:0/2792070281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764145e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 --2- 172.21.15.181:0/2792070281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7641492c0 0x7ff764080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- 172.21.15.181:0/2792070281 >> 172.21.15.181:0/2792070281 conn(0x7ff7640d68c0 msgr2=0x7ff7640d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.769+0000 7ff76ace3700 1 -- 172.21.15.181:0/2792070281 shutdown_connections 2020-11-16T22:26:59.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 -- 172.21.15.181:0/2792070281 wait complete. 2020-11-16T22:26:59.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 Processor -- start 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 -- start start 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 0x7ff764146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764163ad0 con 0x7ff764145a30 2020-11-16T22:26:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764147400 con 0x7ff7641492c0 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76ace3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7641476a0 con 0x7ff7640814c0 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff769ce1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff769ce1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36402/0 (socket says 172.21.15.181:36402) 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff7694e0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff76a4e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 0x7ff764146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.773+0000 7ff769ce1700 1 -- 172.21.15.181:0/3672473502 learned_addr learned my addr 172.21.15.181:0/3672473502 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 -- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 msgr2=0x7ff764160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 -- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 msgr2=0x7ff764146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 0x7ff764146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 -- 172.21.15.181:0/3672473502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764147940 con 0x7ff764145a30 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff769ce1700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff76a4e2700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 0x7ff764146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:26:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff7694e0700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764146910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff754006fd0 tx=0x7ff754008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754011660 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7540117c0 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754011b40 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff764147be0 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640b9ad0 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff754023070 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff764163c90 con 0x7ff764145a30 2020-11-16T22:26:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 0x7ff73c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:26:59.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff75406de70 con 0x7ff764145a30 2020-11-16T22:26:59.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.777+0000 7ff769ce1700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 0x7ff73c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:26:59.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.781+0000 7ff769ce1700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 0x7ff73c057550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff76000be60 tx=0x7ff760015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:26:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.789+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff764163c90 con 0x7ff764145a30 2020-11-16T22:26:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.941+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff764163c90 con 0x7ff764145a30 2020-11-16T22:26:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff751ffb700 1 -- 172.21.15.181:0/3672473502 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff764163c90 con 0x7ff764145a30 2020-11-16T22:26:59.930 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:26:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 msgr2=0x7ff73c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 0x7ff73c057550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff76000be60 tx=0x7ff760015000).stop 2020-11-16T22:26:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 msgr2=0x7ff764146910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764146910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff754006fd0 tx=0x7ff754008040).stop 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 shutdown_connections 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff73c055090 0x7ff73c057550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641492c0 0x7ff764146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7640814c0 0x7ff764160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 --2- 172.21.15.181:0/3672473502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff764145a30 0x7ff764146910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:26:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 >> 172.21.15.181:0/3672473502 conn(0x7ff7640d68c0 msgr2=0x7ff764162d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:26:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 shutdown_connections 2020-11-16T22:26:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:26:59.945+0000 7ff76ace3700 1 -- 172.21.15.181:0/3672473502 wait complete. 2020-11-16T22:26:59.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2020-11-16T22:26:59.944 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-16T22:27:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 Processor -- start 2020-11-16T22:27:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 -- start start 2020-11-16T22:27:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 0x7f94200b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f942015ff80 0x7f9420160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942015e2e0 con 0x7f94200b2190 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200b2a60 con 0x7f942015ff80 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9426653700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200ce460 con 0x7f942015de80 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9425651700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94200b2190 0x7f94200cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42870/0 (socket says 172.21.15.181:42870) 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9425651700 1 -- 172.21.15.181:0/2350393680 learned_addr learned my addr 172.21.15.181:0/2350393680 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 --2- 172.21.15.181:0/2350393680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 0x7f94200b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9425e52700 1 --2- 172.21.15.181:0/2350393680 >> v2:172.21.15.175:3300/0 conn(0x7f942015ff80 0x7f9420160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f940ffff700 1 -- 172.21.15.181:0/2350393680 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637708975 0 0) 0x7f942015e2e0 con 0x7f94200b2190 2020-11-16T22:27:00.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f940ffff700 1 -- 172.21.15.181:0/2350393680 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9408003240 con 0x7f94200b2190 2020-11-16T22:27:00.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 -- 172.21.15.181:0/2350393680 >> v2:172.21.15.175:3300/0 conn(0x7f942015ff80 msgr2=0x7f9420160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:00.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 --2- 172.21.15.181:0/2350393680 >> v2:172.21.15.175:3300/0 conn(0x7f942015ff80 0x7f9420160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 -- 172.21.15.181:0/2350393680 >> v1:172.21.15.181:6789/0 conn(0x7f94200b2190 legacy=0x7f94200cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 -- 172.21.15.181:0/2350393680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200ce700 con 0x7f942015de80 2020-11-16T22:27:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f9424e50700 1 --2- 172.21.15.181:0/2350393680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 0x7f94200b2570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f941c00b9a0 tx=0x7f941c006df0).ready entity=mon.2 client_cookie=92af1a40810efa26 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f940ffff700 1 -- 172.21.15.181:0/2350393680 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c009680 con 0x7f942015de80 2020-11-16T22:27:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f940ffff700 1 -- 172.21.15.181:0/2350393680 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0097e0 con 0x7f942015de80 2020-11-16T22:27:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.097+0000 7f940ffff700 1 -- 172.21.15.181:0/2350393680 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c009b60 con 0x7f942015de80 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- 172.21.15.181:0/2350393680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 msgr2=0x7f94200b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 --2- 172.21.15.181:0/2350393680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 0x7f94200b2570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f941c00b9a0 tx=0x7f941c006df0).stop 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- 172.21.15.181:0/2350393680 shutdown_connections 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 --2- 172.21.15.181:0/2350393680 >> v2:172.21.15.175:3300/0 conn(0x7f942015ff80 0x7f9420160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 --2- 172.21.15.181:0/2350393680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942015de80 0x7f94200b2570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- 172.21.15.181:0/2350393680 >> 172.21.15.181:0/2350393680 conn(0x7f94200cfe90 msgr2=0x7f94200d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- 172.21.15.181:0/2350393680 shutdown_connections 2020-11-16T22:27:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- 172.21.15.181:0/2350393680 wait complete. 2020-11-16T22:27:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 Processor -- start 2020-11-16T22:27:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.101+0000 7f9426653700 1 -- start start 2020-11-16T22:27:00.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94200b2190 0x7f94200600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 0x7f942012e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420060a10 con 0x7f942015de80 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420060cb0 con 0x7f942015ff80 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9420060f50 con 0x7f94200b2190 2020-11-16T22:27:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56806/0 (socket says 172.21.15.181:56806) 2020-11-16T22:27:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 -- 172.21.15.181:0/3424707842 learned_addr learned my addr 172.21.15.181:0/3424707842 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425651700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94200b2190 0x7f94200600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9424e50700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 0x7f942012e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 -- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94200b2190 msgr2=0x7f94200600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94200b2190 0x7f94200600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 -- 172.21.15.181:0/3424707842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 msgr2=0x7f942012e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 0x7f942012e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 -- 172.21.15.181:0/3424707842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200611f0 con 0x7f942015ff80 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9424e50700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 0x7f942012e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9425e52700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f941000eba0 tx=0x7f9410006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000c820 con 0x7f942015ff80 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9420061490 con 0x7f942015ff80 2020-11-16T22:27:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9420061bd0 con 0x7f942015ff80 2020-11-16T22:27:00.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941000c980 con 0x7f942015ff80 2020-11-16T22:27:00.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000cd40 con 0x7f942015ff80 2020-11-16T22:27:00.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.105+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420068fb0 con 0x7f942015ff80 2020-11-16T22:27:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.109+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9410012600 con 0x7f942015ff80 2020-11-16T22:27:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.109+0000 7f940d7fa700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 0x7f93f8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.109+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f941006e070 con 0x7f942015ff80 2020-11-16T22:27:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.109+0000 7f9425651700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 0x7f93f8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.109+0000 7f9425651700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 0x7f93f8057520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9418005920 tx=0x7f9418009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.113+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9420068fb0 con 0x7f942015ff80 2020-11-16T22:27:00.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.261+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9420068fb0 con 0x7f942015ff80 2020-11-16T22:27:00.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.261+0000 7f940d7fa700 1 -- 172.21.15.181:0/3424707842 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9420068fb0 con 0x7f942015ff80 2020-11-16T22:27:00.249 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:27:00.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 msgr2=0x7f93f8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 0x7f93f8057520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9418005920 tx=0x7f9418009040).stop 2020-11-16T22:27:00.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 msgr2=0x7f9420170880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f941000eba0 tx=0x7f9410006df0).stop 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 shutdown_connections 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f93f8055060 0x7f93f8057520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942015ff80 0x7f9420170880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94200b2190 0x7f94200600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 --2- 172.21.15.181:0/3424707842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942015de80 0x7f942012e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 >> 172.21.15.181:0/3424707842 conn(0x7f94200cfe90 msgr2=0x7f94200d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:00.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 shutdown_connections 2020-11-16T22:27:00.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.265+0000 7f9426653700 1 -- 172.21.15.181:0/3424707842 wait complete. 2020-11-16T22:27:00.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2020-11-16T22:27:00.267 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:27:00.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.401+0000 7f7cb37d8700 1 Processor -- start 2020-11-16T22:27:00.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.401+0000 7f7cb37d8700 1 -- start start 2020-11-16T22:27:00.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac064050 con 0x7f7cac15e270 2020-11-16T22:27:00.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac0642f0 con 0x7f7cac063b80 2020-11-16T22:27:00.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac064590 con 0x7f7cac0b6c20 2020-11-16T22:27:00.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36426/0 (socket says 172.21.15.181:36426) 2020-11-16T22:27:00.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 -- 172.21.15.181:0/1684549433 learned_addr learned my addr 172.21.15.181:0/1684549433 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:00.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 -- 172.21.15.181:0/1684549433 >> v1:172.21.15.181:6789/0 conn(0x7f7cac063b80 legacy=0x7f7cac15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 -- 172.21.15.181:0/1684549433 >> v1:172.21.15.175:6789/0 conn(0x7f7cac15e270 legacy=0x7f7cac0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 -- 172.21.15.181:0/1684549433 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac064830 con 0x7f7cac0b6c20 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb1d75700 1 --2- 172.21.15.181:0/1684549433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ca4009d80 tx=0x7f7ca4009290).ready entity=mon.2 client_cookie=ef23c3f2ed9fc4ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7ca3fff700 1 -- 172.21.15.181:0/1684549433 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400bc80 con 0x7f7cac0b6c20 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7ca3fff700 1 -- 172.21.15.181:0/1684549433 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400bde0 con 0x7f7cac0b6c20 2020-11-16T22:27:00.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7ca3fff700 1 -- 172.21.15.181:0/1684549433 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4011610 con 0x7f7cac0b6c20 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- 172.21.15.181:0/1684549433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 msgr2=0x7f7cac066ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/1684549433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ca4009d80 tx=0x7f7ca4009290).stop 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- 172.21.15.181:0/1684549433 shutdown_connections 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/1684549433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac0b6c20 0x7f7cac066ec0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- 172.21.15.181:0/1684549433 >> 172.21.15.181:0/1684549433 conn(0x7f7cac0d4790 msgr2=0x7f7cac0d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- 172.21.15.181:0/1684549433 shutdown_connections 2020-11-16T22:27:00.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.405+0000 7f7cb37d8700 1 -- 172.21.15.181:0/1684549433 wait complete. 2020-11-16T22:27:00.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 Processor -- start 2020-11-16T22:27:00.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- start start 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac063b80 0x7f7cac0b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cac0b6c20 0x7f7cac0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac161640 con 0x7f7cac0b6c20 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac16f370 con 0x7f7cac15e270 2020-11-16T22:27:00.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac16f610 con 0x7f7cac063b80 2020-11-16T22:27:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1574700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac063b80 0x7f7cac0b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56822/0 (socket says 172.21.15.181:56822) 2020-11-16T22:27:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 -- 172.21.15.181:0/590509996 learned_addr learned my addr 172.21.15.181:0/590509996 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb0d73700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cac0b6c20 0x7f7cac0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 -- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac063b80 msgr2=0x7f7cac0b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac063b80 0x7f7cac0b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 -- 172.21.15.181:0/590509996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cac0b6c20 msgr2=0x7f7cac0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cac0b6c20 0x7f7cac0b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 -- 172.21.15.181:0/590509996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac16f8b0 con 0x7f7cac15e270 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb1d75700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7ca400c010 tx=0x7f7ca4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400bc80 con 0x7f7cac15e270 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4010b90 con 0x7f7cac15e270 2020-11-16T22:27:00.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac16fb50 con 0x7f7cac15e270 2020-11-16T22:27:00.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca401c470 con 0x7f7cac15e270 2020-11-16T22:27:00.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.409+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac170090 con 0x7f7cac15e270 2020-11-16T22:27:00.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac069eb0 con 0x7f7cac15e270 2020-11-16T22:27:00.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7ca4017070 con 0x7f7cac15e270 2020-11-16T22:27:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7ca17fa700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 0x7f7c84057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7cb1574700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 0x7f7c84057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7ca406ff90 con 0x7f7cac15e270 2020-11-16T22:27:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.413+0000 7f7cb1574700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 0x7f7c84057550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f7ca8003300 tx=0x7f7ca8007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:00.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.421+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7ca4046120 con 0x7f7cac15e270 2020-11-16T22:27:00.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7cac16f110 con 0x7f7cac15e270 2020-11-16T22:27:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7ca17fa700 1 -- 172.21.15.181:0/590509996 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ca4015070 con 0x7f7cac15e270 2020-11-16T22:27:00.560 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:27:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 msgr2=0x7f7c84057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 0x7f7c84057550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f7ca8003300 tx=0x7f7ca8007b90).stop 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 msgr2=0x7f7cac0b9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.573+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7ca400c010 tx=0x7f7ca4016000).stop 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 shutdown_connections 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7c84055090 0x7f7c84057550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e270 0x7f7cac0b9830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cac063b80 0x7f7cac0b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 --2- 172.21.15.181:0/590509996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cac0b6c20 0x7f7cac0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 >> 172.21.15.181:0/590509996 conn(0x7f7cac0d4790 msgr2=0x7f7cac157500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 shutdown_connections 2020-11-16T22:27:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:00.577+0000 7f7cb37d8700 1 -- 172.21.15.181:0/590509996 wait complete. 2020-11-16T22:27:00.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.2 2020-11-16T22:27:01.576 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:27:01.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 Processor -- start 2020-11-16T22:27:01.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 -- start start 2020-11-16T22:27:01.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff9e0145a30 0x7ff9e0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e0061b50 con 0x7ff9e01492c0 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00618b0 con 0x7ff9e0145a30 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e65e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00cb930 con 0x7ff9e00816e0 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.721+0000 7ff9e5ddf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e4ddd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9e01492c0 0x7ff9e0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42908/0 (socket says 172.21.15.181:42908) 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e4ddd700 1 -- 172.21.15.181:0/3125313594 learned_addr learned my addr 172.21.15.181:0/3125313594 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36450/0 (socket says 172.21.15.181:36450) 2020-11-16T22:27:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e55de700 1 --2- 172.21.15.181:0/3125313594 >> v2:172.21.15.175:3300/0 conn(0x7ff9e0145a30 0x7ff9e0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9d7fff700 1 -- 172.21.15.181:0/3125313594 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493419108 0 0) 0x7ff9e0061b50 con 0x7ff9e01492c0 2020-11-16T22:27:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9d7fff700 1 -- 172.21.15.181:0/3125313594 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c4003240 con 0x7ff9e01492c0 2020-11-16T22:27:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/3125313594 >> v2:172.21.15.175:3300/0 conn(0x7ff9e0145a30 msgr2=0x7ff9e0145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 --2- 172.21.15.181:0/3125313594 >> v2:172.21.15.175:3300/0 conn(0x7ff9e0145a30 0x7ff9e0145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/3125313594 >> v1:172.21.15.181:6789/0 conn(0x7ff9e01492c0 legacy=0x7ff9e0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/3125313594 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00cbbd0 con 0x7ff9e00816e0 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e5ddf700 1 --2- 172.21.15.181:0/3125313594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff9d8009c90 tx=0x7ff9d80091d0).ready entity=mon.2 client_cookie=75a0391f7d8d6281 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9d7fff700 1 -- 172.21.15.181:0/3125313594 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800b9c0 con 0x7ff9e00816e0 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9d7fff700 1 -- 172.21.15.181:0/3125313594 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800bb20 con 0x7ff9e00816e0 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9d7fff700 1 -- 172.21.15.181:0/3125313594 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800bea0 con 0x7ff9e00816e0 2020-11-16T22:27:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 -- 172.21.15.181:0/3125313594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 msgr2=0x7ff9e00cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/3125313594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff9d8009c90 tx=0x7ff9d80091d0).stop 2020-11-16T22:27:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 -- 172.21.15.181:0/3125313594 shutdown_connections 2020-11-16T22:27:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/3125313594 >> v2:172.21.15.175:3300/0 conn(0x7ff9e0145a30 0x7ff9e0145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/3125313594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e00816e0 0x7ff9e00cc5b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 -- 172.21.15.181:0/3125313594 >> 172.21.15.181:0/3125313594 conn(0x7ff9e00d68c0 msgr2=0x7ff9e00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 -- 172.21.15.181:0/3125313594 shutdown_connections 2020-11-16T22:27:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.725+0000 7ff9e65e0700 1 -- 172.21.15.181:0/3125313594 wait complete. 2020-11-16T22:27:01.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 Processor -- start 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- start start 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e00816e0 0x7ff9e0165ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e0145a30 0x7ff9e013d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01703e0 con 0x7ff9e00816e0 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0166310 con 0x7ff9e01492c0 2020-11-16T22:27:01.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01665b0 con 0x7ff9e0145a30 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56844/0 (socket says 172.21.15.181:56844) 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/1702325729 learned_addr learned my addr 172.21.15.181:0/1702325729 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e4ddd700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e0145a30 0x7ff9e013d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e55de700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e00816e0 0x7ff9e0165ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e0145a30 msgr2=0x7ff9e013d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e0145a30 0x7ff9e013d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/1702325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e00816e0 msgr2=0x7ff9e0165ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e00816e0 0x7ff9e0165ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 -- 172.21.15.181:0/1702325729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0166850 con 0x7ff9e01492c0 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e5ddf700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff9d8011000 tx=0x7ff9d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d801c6c0 con 0x7ff9e01492c0 2020-11-16T22:27:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d801c820 con 0x7ff9e01492c0 2020-11-16T22:27:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d801cbc0 con 0x7ff9e01492c0 2020-11-16T22:27:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e0166af0 con 0x7ff9e01492c0 2020-11-16T22:27:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.729+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e0166030 con 0x7ff9e01492c0 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff9d8017070 con 0x7ff9e01492c0 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9d57fa700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 0x7ff9b8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e0170520 con 0x7ff9e01492c0 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9e55de700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 0x7ff9b8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff9d8070150 con 0x7ff9e01492c0 2020-11-16T22:27:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.733+0000 7ff9e55de700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 0x7ff9b8057550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ff9d0006fd0 tx=0x7ff9d0006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:01.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.741+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff9d8049de0 con 0x7ff9e01492c0 2020-11-16T22:27:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.893+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff9e0044f60 con 0x7ff9e01492c0 2020-11-16T22:27:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.893+0000 7ff9d57fa700 1 -- 172.21.15.181:0/1702325729 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff9d8015020 con 0x7ff9e01492c0 2020-11-16T22:27:01.881 INFO:teuthology.orchestra.run.smithi181.stdout:60129542151 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 msgr2=0x7ff9b8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 0x7ff9b8057550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ff9d0006fd0 tx=0x7ff9d0006df0).stop 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 msgr2=0x7ff9e013d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff9d8011000 tx=0x7ff9d8016000).stop 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 shutdown_connections 2020-11-16T22:27:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff9b8055090 0x7ff9b8057550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e01492c0 0x7ff9e013d6f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e00816e0 0x7ff9e0165ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 --2- 172.21.15.181:0/1702325729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9e0145a30 0x7ff9e013d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 >> 172.21.15.181:0/1702325729 conn(0x7ff9e00d68c0 msgr2=0x7ff9e00d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 shutdown_connections 2020-11-16T22:27:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:01.897+0000 7ff9e65e0700 1 -- 172.21.15.181:0/1702325729 wait complete. 2020-11-16T22:27:01.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2020-11-16T22:27:01.903 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2020-11-16T22:27:02.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.037+0000 7f2e4359e700 1 Processor -- start 2020-11-16T22:27:02.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- start start 2020-11-16T22:27:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e44160aa0 0x7f2e440b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e440c9800 con 0x7f2e441606c0 2020-11-16T22:27:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440c9aa0 con 0x7f2e44160aa0 2020-11-16T22:27:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440c9d40 con 0x7f2e440b7680 2020-11-16T22:27:02.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e3bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e44160aa0 0x7f2e440b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36462/0 (socket says 172.21.15.181:36462) 2020-11-16T22:27:02.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 -- 172.21.15.181:0/3494761517 learned_addr learned my addr 172.21.15.181:0/3494761517 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e41d9b700 1 -- 172.21.15.181:0/3494761517 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919573867 0 0) 0x7f2e440c9800 con 0x7f2e441606c0 2020-11-16T22:27:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e41d9b700 1 -- 172.21.15.181:0/3494761517 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e24003240 con 0x7f2e441606c0 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 -- 172.21.15.181:0/3494761517 >> v2:172.21.15.181:3300/0 conn(0x7f2e44160aa0 msgr2=0x7f2e440b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 --2- 172.21.15.181:0/3494761517 >> v2:172.21.15.181:3300/0 conn(0x7f2e44160aa0 0x7f2e440b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 -- 172.21.15.181:0/3494761517 >> v1:172.21.15.181:6789/0 conn(0x7f2e441606c0 legacy=0x7f2e440c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 -- 172.21.15.181:0/3494761517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e440c9fe0 con 0x7f2e440b7680 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e42d9d700 1 --2- 172.21.15.181:0/3494761517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2e3c009d50 tx=0x7f2e3c009290).ready entity=mon.2 client_cookie=c320d7d9b5f60a03 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e41d9b700 1 -- 172.21.15.181:0/3494761517 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00bb10 con 0x7f2e440b7680 2020-11-16T22:27:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e41d9b700 1 -- 172.21.15.181:0/3494761517 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00bc70 con 0x7f2e440b7680 2020-11-16T22:27:02.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e41d9b700 1 -- 172.21.15.181:0/3494761517 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c011400 con 0x7f2e440b7680 2020-11-16T22:27:02.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- 172.21.15.181:0/3494761517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 msgr2=0x7f2e440cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3494761517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2e3c009d50 tx=0x7f2e3c009290).stop 2020-11-16T22:27:02.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- 172.21.15.181:0/3494761517 shutdown_connections 2020-11-16T22:27:02.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3494761517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e440b7680 0x7f2e440cdff0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3494761517 >> v2:172.21.15.181:3300/0 conn(0x7f2e44160aa0 0x7f2e440b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- 172.21.15.181:0/3494761517 >> 172.21.15.181:0/3494761517 conn(0x7f2e440d33f0 msgr2=0x7f2e440d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- 172.21.15.181:0/3494761517 shutdown_connections 2020-11-16T22:27:02.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.041+0000 7f2e4359e700 1 -- 172.21.15.181:0/3494761517 wait complete. 2020-11-16T22:27:02.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 Processor -- start 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- start start 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 0x7f2e4414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e44160aa0 0x7f2e440603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440cf810 con 0x7f2e440b7680 2020-11-16T22:27:02.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441481c0 con 0x7f2e441606c0 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44148460 con 0x7f2e44160aa0 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56858/0 (socket says 172.21.15.181:56858) 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 -- 172.21.15.181:0/3727880328 learned_addr learned my addr 172.21.15.181:0/3727880328 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e42d9d700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e44160aa0 0x7f2e440603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4259c700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 0x7f2e4414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 -- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e44160aa0 msgr2=0x7f2e440603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e44160aa0 0x7f2e440603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 -- 172.21.15.181:0/3727880328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 msgr2=0x7f2e4414cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 0x7f2e4414cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 -- 172.21.15.181:0/3727880328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44148700 con 0x7f2e441606c0 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4259c700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 0x7f2e4414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:27:02.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3bfff700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2e3400eba0 tx=0x7f2e34006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400c9b0 con 0x7f2e441606c0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400cb10 con 0x7f2e441606c0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e441489a0 con 0x7f2e441606c0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400ce90 con 0x7f2e441606c0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.045+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e44149180 con 0x7f2e441606c0 2020-11-16T22:27:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2e34012600 con 0x7f2e441606c0 2020-11-16T22:27:02.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e3a7fc700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 0x7f2e20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e3406e130 con 0x7f2e441606c0 2020-11-16T22:27:02.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44044f60 con 0x7f2e441606c0 2020-11-16T22:27:02.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e4259c700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 0x7f2e20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.049+0000 7f2e4259c700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 0x7f2e20057520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2e2c000bc0 tx=0x7f2e2c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.057+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2e34044330 con 0x7f2e441606c0 2020-11-16T22:27:02.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.209+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e44147730 con 0x7f2e441606c0 2020-11-16T22:27:02.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.209+0000 7f2e3a7fc700 1 -- 172.21.15.181:0/3727880328 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e34047950 con 0x7f2e441606c0 2020-11-16T22:27:02.197 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2020-11-16T22:27:02.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 msgr2=0x7f2e20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 0x7f2e20057520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2e2c000bc0 tx=0x7f2e2c009040).stop 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 msgr2=0x7f2e440cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2e3400eba0 tx=0x7f2e34006df0).stop 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 shutdown_connections 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e20055060 0x7f2e20057520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e44160aa0 0x7f2e440603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e440b7680 0x7f2e4414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 --2- 172.21.15.181:0/3727880328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e441606c0 0x7f2e440cf320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 >> 172.21.15.181:0/3727880328 conn(0x7f2e440d33f0 msgr2=0x7f2e44152970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 shutdown_connections 2020-11-16T22:27:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.213+0000 7f2e4359e700 1 -- 172.21.15.181:0/3727880328 wait complete. 2020-11-16T22:27:02.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2020-11-16T22:27:02.212 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-16T22:27:02.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 Processor -- start 2020-11-16T22:27:02.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 -- start start 2020-11-16T22:27:02.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc1c4142660 0x7fc1c4142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1c4063620 0x7fc1c4063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c4061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c4062700 con 0x7fc1c4142660 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40629a0 con 0x7fc1c4063620 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c8b50700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4062c40 con 0x7fc1c4160530 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1c4063620 0x7fc1c4063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1c4063620 0x7fc1c4063ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56866/0 (socket says 172.21.15.181:56866) 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 -- 172.21.15.181:0/2837425056 learned_addr learned my addr 172.21.15.181:0/2837425056 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c37fe700 1 --2- 172.21.15.181:0/2837425056 >> v2:172.21.15.175:3300/0 conn(0x7fc1c4142660 0x7fc1c4142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 -- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 msgr2=0x7fc1c4061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c4061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 -- 172.21.15.181:0/2837425056 >> v2:172.21.15.175:3300/0 conn(0x7fc1c4142660 msgr2=0x7fc1c4142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/2837425056 >> v2:172.21.15.175:3300/0 conn(0x7fc1c4142660 0x7fc1c4142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 -- 172.21.15.181:0/2837425056 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4062ee0 con 0x7fc1c4063620 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c4063ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc1c405ff50 tx=0x7fc1ac006eb0).ready entity=mon.1 client_cookie=9e4fe9cb27bf2542 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c27fc700 1 -- 172.21.15.181:0/2837425056 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0098a0 con 0x7fc1c4063620 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c27fc700 1 -- 172.21.15.181:0/2837425056 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac009a00 con 0x7fc1c4063620 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.349+0000 7fc1c27fc700 1 -- 172.21.15.181:0/2837425056 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac009d80 con 0x7fc1c4063620 2020-11-16T22:27:02.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 msgr2=0x7fc1c4063ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c4063ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc1c405ff50 tx=0x7fc1ac006eb0).stop 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/2837425056 shutdown_connections 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c4061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/2837425056 >> v2:172.21.15.175:3300/0 conn(0x7fc1c4142660 0x7fc1c4142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/2837425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c4063ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/2837425056 >> 172.21.15.181:0/2837425056 conn(0x7fc1c40d2dc0 msgr2=0x7fc1c40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/2837425056 shutdown_connections 2020-11-16T22:27:02.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/2837425056 wait complete. 2020-11-16T22:27:02.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 Processor -- start 2020-11-16T22:27:02.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- start start 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1c4142660 0x7fc1c413d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c413ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4145fc0 con 0x7fc1c4142660 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c413e2c0 con 0x7fc1c4063620 2020-11-16T22:27:02.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4173460 con 0x7fc1c4160530 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56874/0 (socket says 172.21.15.181:56874) 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 -- 172.21.15.181:0/260956152 learned_addr learned my addr 172.21.15.181:0/260956152 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c3fff700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c413ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1c4142660 0x7fc1c413d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 -- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 msgr2=0x7fc1c413ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c413ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 -- 172.21.15.181:0/260956152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1c4142660 msgr2=0x7fc1c413d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1c4142660 0x7fc1c413d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 -- 172.21.15.181:0/260956152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c413e4d0 con 0x7fc1c4063620 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c3fff700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c413ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c37fe700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc1b400db90 tx=0x7fc1b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4009730 con 0x7fc1c4063620 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4009890 con 0x7fc1c4063620 2020-11-16T22:27:02.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4009c10 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4172540 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.353+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c4172d20 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc1b4013480 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c4044f60 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1bb7fe700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 0x7fc194057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc1b406e3b0 con 0x7fc1c4063620 2020-11-16T22:27:02.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 0x7fc194057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.357+0000 7fc1c2ffd700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 0x7fc194057520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc1ac00b1b0 tx=0x7fc1ac015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.365+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc1b40445b0 con 0x7fc1c4063620 2020-11-16T22:27:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc1c4172b00 con 0x7fc1c4063620 2020-11-16T22:27:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/260956152 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1b4047bd0 con 0x7fc1c4063620 2020-11-16T22:27:02.502 INFO:teuthology.orchestra.run.smithi181.stdout:55834574854 2020-11-16T22:27:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 msgr2=0x7fc194057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 0x7fc194057520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc1ac00b1b0 tx=0x7fc1ac015000).stop 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 msgr2=0x7fc1c413cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc1b400db90 tx=0x7fc1b4008040).stop 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 shutdown_connections 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc194055060 0x7fc194057520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c4160530 0x7fc1c413ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c4063620 0x7fc1c413cbc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 --2- 172.21.15.181:0/260956152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1c4142660 0x7fc1c413d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 >> 172.21.15.181:0/260956152 conn(0x7fc1c40d2dc0 msgr2=0x7fc1c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 shutdown_connections 2020-11-16T22:27:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.517+0000 7fc1c8b50700 1 -- 172.21.15.181:0/260956152 wait complete. 2020-11-16T22:27:02.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2020-11-16T22:27:02.522 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2020-11-16T22:27:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.657+0000 7f44bfeb1700 1 Processor -- start 2020-11-16T22:27:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- start start 2020-11-16T22:27:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44b815f5b0 0x7f44b80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44b80622a0 0x7f44b8062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b8160e40 con 0x7f44b815f5b0 2020-11-16T22:27:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b8160ba0 con 0x7f44b80622a0 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8060970 con 0x7f44b80615f0 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44be44e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44b80622a0 0x7f44b8062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36490/0 (socket says 172.21.15.181:36490) 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 -- 172.21.15.181:0/783433134 learned_addr learned my addr 172.21.15.181:0/783433134 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bdc4d700 1 --2- 172.21.15.181:0/783433134 >> v2:172.21.15.175:3300/0 conn(0x7f44b815f5b0 0x7f44b80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 -- 172.21.15.181:0/783433134 >> v2:172.21.15.181:3300/0 conn(0x7f44b80622a0 msgr2=0x7f44b8062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 --2- 172.21.15.181:0/783433134 >> v2:172.21.15.181:3300/0 conn(0x7f44b80622a0 0x7f44b8062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 -- 172.21.15.181:0/783433134 >> v2:172.21.15.175:3300/0 conn(0x7f44b815f5b0 msgr2=0x7f44b80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 --2- 172.21.15.181:0/783433134 >> v2:172.21.15.175:3300/0 conn(0x7f44b815f5b0 0x7f44b80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 -- 172.21.15.181:0/783433134 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8060c10 con 0x7f44b80615f0 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bd44c700 1 --2- 172.21.15.181:0/783433134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f44b80b92a0 tx=0x7f44b4006eb0).ready entity=mon.2 client_cookie=313ecbc35e6ca6d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bcc4b700 1 -- 172.21.15.181:0/783433134 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009720 con 0x7f44b80615f0 2020-11-16T22:27:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bcc4b700 1 -- 172.21.15.181:0/783433134 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4009880 con 0x7f44b80615f0 2020-11-16T22:27:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bcc4b700 1 -- 172.21.15.181:0/783433134 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009c00 con 0x7f44b80615f0 2020-11-16T22:27:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- 172.21.15.181:0/783433134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 msgr2=0x7f44b80d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/783433134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f44b80b92a0 tx=0x7f44b4006eb0).stop 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- 172.21.15.181:0/783433134 shutdown_connections 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/783433134 >> v2:172.21.15.181:3300/0 conn(0x7f44b80622a0 0x7f44b8062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/783433134 >> v2:172.21.15.175:3300/0 conn(0x7f44b815f5b0 0x7f44b80d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/783433134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b80615f0 0x7f44b80d1830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- 172.21.15.181:0/783433134 >> 172.21.15.181:0/783433134 conn(0x7f44b80d69a0 msgr2=0x7f44b80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- 172.21.15.181:0/783433134 shutdown_connections 2020-11-16T22:27:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.661+0000 7f44bfeb1700 1 -- 172.21.15.181:0/783433134 wait complete. 2020-11-16T22:27:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 Processor -- start 2020-11-16T22:27:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 -- start start 2020-11-16T22:27:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44b80622a0 0x7f44b814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b815f5b0 0x7f44b815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b816f590 con 0x7f44b80622a0 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b816f830 con 0x7f44b80615f0 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bfeb1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b816fad0 con 0x7f44b815f5b0 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.665+0000 7f44bdc4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56886/0 (socket says 172.21.15.181:56886) 2020-11-16T22:27:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 -- 172.21.15.181:0/4034540499 learned_addr learned my addr 172.21.15.181:0/4034540499 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bd44c700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44b80622a0 0x7f44b814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44be44e700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b815f5b0 0x7f44b815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 -- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b815f5b0 msgr2=0x7f44b815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b815f5b0 0x7f44b815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 -- 172.21.15.181:0/4034540499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44b80622a0 msgr2=0x7f44b814b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44b80622a0 0x7f44b814b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 -- 172.21.15.181:0/4034540499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b816fd70 con 0x7f44b80615f0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bdc4d700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f44a800eba0 tx=0x7f44a8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800b310 con 0x7f44b80615f0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a800c8e0 con 0x7f44b80615f0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800cc80 con 0x7f44b80615f0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b8170010 con 0x7f44b80615f0 2020-11-16T22:27:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b80d5c30 con 0x7f44b80615f0 2020-11-16T22:27:02.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8069eb0 con 0x7f44b80615f0 2020-11-16T22:27:02.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.669+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f44a80123f0 con 0x7f44b80615f0 2020-11-16T22:27:02.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.673+0000 7f44a5ffb700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 0x7f4490057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.673+0000 7f44bd44c700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 0x7f4490057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.673+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f44a806df60 con 0x7f44b80615f0 2020-11-16T22:27:02.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.673+0000 7f44bd44c700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 0x7f4490057520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f44b8061a50 tx=0x7f44b4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.677+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f44a8097ef0 con 0x7f44b80615f0 2020-11-16T22:27:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.829+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f44b80d5450 con 0x7f44b80615f0 2020-11-16T22:27:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44a5ffb700 1 -- 172.21.15.181:0/4034540499 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44a8016070 con 0x7f44b80615f0 2020-11-16T22:27:02.818 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2020-11-16T22:27:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 msgr2=0x7f4490057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 0x7f4490057520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f44b8061a50 tx=0x7f44b4015000).stop 2020-11-16T22:27:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 msgr2=0x7f44b80d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f44a800eba0 tx=0x7f44a8008040).stop 2020-11-16T22:27:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 shutdown_connections 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4490055060 0x7f4490057520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44b815f5b0 0x7f44b815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b80615f0 0x7f44b80d5740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 --2- 172.21.15.181:0/4034540499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44b80622a0 0x7f44b814b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 >> 172.21.15.181:0/4034540499 conn(0x7f44b80d69a0 msgr2=0x7f44b815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 shutdown_connections 2020-11-16T22:27:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.833+0000 7f44bfeb1700 1 -- 172.21.15.181:0/4034540499 wait complete. 2020-11-16T22:27:02.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2020-11-16T22:27:02.838 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2020-11-16T22:27:02.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.977+0000 7f312be44700 1 Processor -- start 2020-11-16T22:27:02.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.977+0000 7f312be44700 1 -- start start 2020-11-16T22:27:02.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 0x7f31240cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 0x7f312415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312415cda0 con 0x7f31240612a0 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312415d040 con 0x7f312415fa10 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312415d2e0 con 0x7f3124061680 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3129be0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f31240612a0 0x7f31240ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42966/0 (socket says 172.21.15.181:42966) 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3129be0700 1 -- 172.21.15.181:0/3643836321 learned_addr learned my addr 172.21.15.181:0/3643836321 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 --2- 172.21.15.181:0/3643836321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 0x7f31240cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312a3e1700 1 --2- 172.21.15.181:0/3643836321 >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 0x7f312415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3128bde700 1 -- 172.21.15.181:0/3643836321 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600169731 0 0) 0x7f312415cda0 con 0x7f31240612a0 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 -- 172.21.15.181:0/3643836321 >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 msgr2=0x7f312415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 --2- 172.21.15.181:0/3643836321 >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 0x7f312415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 -- 172.21.15.181:0/3643836321 >> v1:172.21.15.181:6789/0 conn(0x7f31240612a0 legacy=0x7f31240ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 -- 172.21.15.181:0/3643836321 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240cdd70 con 0x7f3124061680 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312a3e1700 1 --2- 172.21.15.181:0/3643836321 >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 0x7f312415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:02.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f31293df700 1 --2- 172.21.15.181:0/3643836321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 0x7f31240cee60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f311400cb60 tx=0x7f3114006e80).ready entity=mon.2 client_cookie=7474c7d76c06ca7e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3128bde700 1 -- 172.21.15.181:0/3643836321 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400aa10 con 0x7f3124061680 2020-11-16T22:27:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3128bde700 1 -- 172.21.15.181:0/3643836321 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311400ab70 con 0x7f3124061680 2020-11-16T22:27:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f3128bde700 1 -- 172.21.15.181:0/3643836321 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400ace0 con 0x7f3124061680 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- 172.21.15.181:0/3643836321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 msgr2=0x7f31240cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 --2- 172.21.15.181:0/3643836321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 0x7f31240cee60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f311400cb60 tx=0x7f3114006e80).stop 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- 172.21.15.181:0/3643836321 shutdown_connections 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 --2- 172.21.15.181:0/3643836321 >> v2:172.21.15.181:3300/0 conn(0x7f312415fa10 0x7f312415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 --2- 172.21.15.181:0/3643836321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3124061680 0x7f31240cee60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- 172.21.15.181:0/3643836321 >> 172.21.15.181:0/3643836321 conn(0x7f31240cf890 msgr2=0x7f31240cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- 172.21.15.181:0/3643836321 shutdown_connections 2020-11-16T22:27:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.981+0000 7f312be44700 1 -- 172.21.15.181:0/3643836321 wait complete. 2020-11-16T22:27:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 Processor -- start 2020-11-16T22:27:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- start start 2020-11-16T22:27:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 0x7f3124170a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312416d590 con 0x7f3124061680 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241710a0 con 0x7f31240612a0 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3124171340 con 0x7f312415fa10 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312a3e1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56902/0 (socket says 172.21.15.181:56902) 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312a3e1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36510/0 (socket says 172.21.15.181:36510) 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 -- 172.21.15.181:0/3863572839 learned_addr learned my addr 172.21.15.181:0/3863572839 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f31293df700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 0x7f3124170a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 -- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 msgr2=0x7f312415a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 -- 172.21.15.181:0/3863572839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 msgr2=0x7f3124170a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 0x7f3124170a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 -- 172.21.15.181:0/3863572839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31241715e0 con 0x7f31240612a0 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312a3e1700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f31293df700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 0x7f3124170a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3129be0700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f312000ca90 tx=0x7f3120007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120005b80 con 0x7f31240612a0 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3120005ce0 con 0x7f31240612a0 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120007320 con 0x7f31240612a0 2020-11-16T22:27:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312416d7c0 con 0x7f31240612a0 2020-11-16T22:27:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.985+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312416dc60 con 0x7f31240612a0 2020-11-16T22:27:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31240cdf50 con 0x7f31240612a0 2020-11-16T22:27:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3120010030 con 0x7f31240612a0 2020-11-16T22:27:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f3111ffb700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 0x7f30fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f312006dfb0 con 0x7f31240612a0 2020-11-16T22:27:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f31293df700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 0x7f30fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.989+0000 7f31293df700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 0x7f30fc057520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f311400cb30 tx=0x7f3124063560).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:02.997+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3120044100 con 0x7f31240612a0 2020-11-16T22:27:03.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3124069eb0 con 0x7f31240612a0 2020-11-16T22:27:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f3111ffb700 1 -- 172.21.15.181:0/3863572839 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3120047720 con 0x7f31240612a0 2020-11-16T22:27:03.135 INFO:teuthology.orchestra.run.smithi181.stdout:55834574855 2020-11-16T22:27:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 msgr2=0x7f30fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 0x7f30fc057520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f311400cb30 tx=0x7f3124063560).stop 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 msgr2=0x7f312416e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f312000ca90 tx=0x7f3120007e40).stop 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.149+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 shutdown_connections 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f30fc055060 0x7f30fc057520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f312415fa10 0x7f312415a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31240612a0 0x7f312416e950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 --2- 172.21.15.181:0/3863572839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3124061680 0x7f3124170a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 >> 172.21.15.181:0/3863572839 conn(0x7f31240cf890 msgr2=0x7f312415fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 shutdown_connections 2020-11-16T22:27:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.153+0000 7f312be44700 1 -- 172.21.15.181:0/3863572839 wait complete. 2020-11-16T22:27:03.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2020-11-16T22:27:03.150 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2020-11-16T22:27:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.281+0000 7f5313fff700 1 Processor -- start 2020-11-16T22:27:03.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.281+0000 7f5313fff700 1 -- start start 2020-11-16T22:27:03.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.281+0000 7f5313fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5314061810 0x7f531415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.281+0000 7f5313fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f531415f1a0 0x7f531415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53140cccd0 con 0x7f5314156a90 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53140ccf70 con 0x7f531415f1a0 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531415e030 con 0x7f5314061810 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5314061810 0x7f531415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5314061810 0x7f531415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56910/0 (socket says 172.21.15.181:56910) 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53137fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f531415f1a0 0x7f531415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 -- 172.21.15.181:0/4065440165 learned_addr learned my addr 172.21.15.181:0/4065440165 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 -- 172.21.15.181:0/4065440165 >> v2:172.21.15.175:3300/0 conn(0x7f531415f1a0 msgr2=0x7f531415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 --2- 172.21.15.181:0/4065440165 >> v2:172.21.15.175:3300/0 conn(0x7f531415f1a0 0x7f531415f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 -- 172.21.15.181:0/4065440165 >> v1:172.21.15.175:6789/0 conn(0x7f5314156a90 legacy=0x7f5314061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 -- 172.21.15.181:0/4065440165 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531415e2d0 con 0x7f5314061810 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f53127fc700 1 --2- 172.21.15.181:0/4065440165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314061810 0x7f531415ecb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f53140c9de0 tx=0x7f5308006eb0).ready entity=mon.1 client_cookie=8d274b6ce8f6f0d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5311ffb700 1 -- 172.21.15.181:0/4065440165 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308009810 con 0x7f5314061810 2020-11-16T22:27:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5311ffb700 1 -- 172.21.15.181:0/4065440165 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5308009970 con 0x7f5314061810 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5311ffb700 1 -- 172.21.15.181:0/4065440165 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308009cf0 con 0x7f5314061810 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- 172.21.15.181:0/4065440165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314061810 msgr2=0x7f531415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 --2- 172.21.15.181:0/4065440165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314061810 0x7f531415ecb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f53140c9de0 tx=0x7f5308006eb0).stop 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- 172.21.15.181:0/4065440165 shutdown_connections 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 --2- 172.21.15.181:0/4065440165 >> v2:172.21.15.175:3300/0 conn(0x7f531415f1a0 0x7f531415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 --2- 172.21.15.181:0/4065440165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314061810 0x7f531415ecb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- 172.21.15.181:0/4065440165 >> 172.21.15.181:0/4065440165 conn(0x7f53140cf630 msgr2=0x7f53140cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:03.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- 172.21.15.181:0/4065440165 shutdown_connections 2020-11-16T22:27:03.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.285+0000 7f5313fff700 1 -- 172.21.15.181:0/4065440165 wait complete. 2020-11-16T22:27:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.289+0000 7f5313fff700 1 Processor -- start 2020-11-16T22:27:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.289+0000 7f5313fff700 1 -- start start 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314061810 0x7f53140c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 0x7f531415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140cc850 con 0x7f5314061810 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314170da0 con 0x7f5314156a90 2020-11-16T22:27:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5314171040 con 0x7f531415f1a0 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 0x7f531415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36528/0 (socket says 172.21.15.181:36528) 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 0x7f531415fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56916/0 (socket says 172.21.15.181:56916) 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 -- 172.21.15.181:0/3018018146 learned_addr learned my addr 172.21.15.181:0/3018018146 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:03.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5312ffd700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314061810 0x7f53140c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 -- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 msgr2=0x7f531415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 0x7f531415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 -- 172.21.15.181:0/3018018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314061810 msgr2=0x7f53140c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314061810 0x7f53140c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 -- 172.21.15.181:0/3018018146 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53141712e0 con 0x7f531415f1a0 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f53137fe700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f530c00eba0 tx=0x7f530c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00b2e0 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00c820 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5314171580 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00cbc0 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53140c7cb0 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5314171710 con 0x7f531415f1a0 2020-11-16T22:27:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f530c01c460 con 0x7f531415f1a0 2020-11-16T22:27:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 0x7f52f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f530c06dff0 con 0x7f531415f1a0 2020-11-16T22:27:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.293+0000 7f5312ffd700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 0x7f52f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.297+0000 7f5312ffd700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 0x7f52f0057520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5304004250 tx=0x7f5304007f10).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.297+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f530c0441f0 con 0x7f531415f1a0 2020-11-16T22:27:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.449+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5314044f60 con 0x7f531415f1a0 2020-11-16T22:27:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f52faffd700 1 -- 172.21.15.181:0/3018018146 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f530c047810 con 0x7f531415f1a0 2020-11-16T22:27:03.437 INFO:teuthology.orchestra.run.smithi181.stdout:60129542151 2020-11-16T22:27:03.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 msgr2=0x7f52f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 0x7f52f0057520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5304004250 tx=0x7f5304007f10).stop 2020-11-16T22:27:03.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 msgr2=0x7f53141601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f530c00eba0 tx=0x7f530c006df0).stop 2020-11-16T22:27:03.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 shutdown_connections 2020-11-16T22:27:03.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52f0055060 0x7f52f0057520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531415f1a0 0x7f53141601b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5314061810 0x7f53140c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 --2- 172.21.15.181:0/3018018146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5314156a90 0x7f531415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 >> 172.21.15.181:0/3018018146 conn(0x7f53140cf630 msgr2=0x7f5314151000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:03.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 shutdown_connections 2020-11-16T22:27:03.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.453+0000 7f5313fff700 1 -- 172.21.15.181:0/3018018146 wait complete. 2020-11-16T22:27:03.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2020-11-16T22:27:03.452 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T22:27:03.452 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:27:03.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.585+0000 7f89c759e700 1 Processor -- start 2020-11-16T22:27:03.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- start start 2020-11-16T22:27:03.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f89c815d080 0x7f89c80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c8065150 con 0x7f89c815d460 2020-11-16T22:27:03.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c80653f0 con 0x7f89c80600f0 2020-11-16T22:27:03.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c8066a20 con 0x7f89c815d080 2020-11-16T22:27:03.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f89c80600f0 0x7f89c8060560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42998/0 (socket says 172.21.15.181:42998) 2020-11-16T22:27:03.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c6d9d700 1 -- 172.21.15.181:0/2936415832 learned_addr learned my addr 172.21.15.181:0/2936415832 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:03.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c659c700 1 --2- 172.21.15.181:0/2936415832 >> v2:172.21.15.175:3300/0 conn(0x7f89c815d080 0x7f89c80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c559a700 1 -- 172.21.15.181:0/2936415832 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506316030 0 0) 0x7f89c80653f0 con 0x7f89c80600f0 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c559a700 1 -- 172.21.15.181:0/2936415832 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a8003240 con 0x7f89c80600f0 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c659c700 1 -- 172.21.15.181:0/2936415832 >> v1:172.21.15.181:6789/0 conn(0x7f89c80600f0 legacy=0x7f89c8060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c659c700 1 -- 172.21.15.181:0/2936415832 >> v1:172.21.15.175:6789/0 conn(0x7f89c815d460 legacy=0x7f89c8066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c659c700 1 -- 172.21.15.181:0/2936415832 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c8065690 con 0x7f89c815d080 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c659c700 1 --2- 172.21.15.181:0/2936415832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d080 0x7f89c80cdd90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f89c000b9d0 tx=0x7f89c0006df0).ready entity=mon.0 client_cookie=3579ba2cbf759229 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c559a700 1 -- 172.21.15.181:0/2936415832 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0009770 con 0x7f89c815d080 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c559a700 1 -- 172.21.15.181:0/2936415832 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c00098d0 con 0x7f89c815d080 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c559a700 1 -- 172.21.15.181:0/2936415832 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0009c50 con 0x7f89c815d080 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- 172.21.15.181:0/2936415832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d080 msgr2=0x7f89c80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 --2- 172.21.15.181:0/2936415832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d080 0x7f89c80cdd90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f89c000b9d0 tx=0x7f89c0006df0).stop 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- 172.21.15.181:0/2936415832 shutdown_connections 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 --2- 172.21.15.181:0/2936415832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d080 0x7f89c80cdd90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- 172.21.15.181:0/2936415832 >> 172.21.15.181:0/2936415832 conn(0x7f89c80cfe90 msgr2=0x7f89c80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:03.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- 172.21.15.181:0/2936415832 shutdown_connections 2020-11-16T22:27:03.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.589+0000 7f89c759e700 1 -- 172.21.15.181:0/2936415832 wait complete. 2020-11-16T22:27:03.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 Processor -- start 2020-11-16T22:27:03.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- start start 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 0x7f89c80b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d460 0x7f89c81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80b7370 con 0x7f89c815d460 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80b70d0 con 0x7f89c80600f0 2020-11-16T22:27:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80b3930 con 0x7f89c815d080 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 0x7f89c80b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56928/0 (socket says 172.21.15.181:56928) 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c6d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d460 0x7f89c81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c659c700 1 -- 172.21.15.181:0/3527160156 learned_addr learned my addr 172.21.15.181:0/3527160156 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 -- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 msgr2=0x7f89c80b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 -- 172.21.15.181:0/3527160156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d460 msgr2=0x7f89c81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d460 0x7f89c81552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 -- 172.21.15.181:0/3527160156 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80b3bd0 con 0x7f89c815d080 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c659c700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c5d9b700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 0x7f89c80b8d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f89b000ca90 tx=0x7f89b0007f10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000ce60 con 0x7f89c815d080 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0006870 con 0x7f89c815d080 2020-11-16T22:27:03.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c80b3e70 con 0x7f89c815d080 2020-11-16T22:27:03.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0006bf0 con 0x7f89c815d080 2020-11-16T22:27:03.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.593+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c81557a0 con 0x7f89c815d080 2020-11-16T22:27:03.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c8066050 con 0x7f89c815d080 2020-11-16T22:27:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f89b0018020 con 0x7f89c815d080 2020-11-16T22:27:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89be7fc700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 0x7f89a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f89b006d210 con 0x7f89c815d080 2020-11-16T22:27:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89c659c700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 0x7f89a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.597+0000 7f89c659c700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 0x7f89a4057520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f89c000be90 tx=0x7f89c0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:03.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.605+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f89c8066050 con 0x7f89c815d080 2020-11-16T22:27:03.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.753+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89c8066050 con 0x7f89a4055060 2020-11-16T22:27:03.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.761+0000 7f89be7fc700 1 -- 172.21.15.181:0/3527160156 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82310 (secure 0 0 0) 0x7f89c8066050 con 0x7f89a4055060 2020-11-16T22:27:03.748 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:03.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 msgr2=0x7f89a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 0x7f89a4057520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f89c000be90 tx=0x7f89c0015000).stop 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 msgr2=0x7f89c80b8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 0x7f89c80b8d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f89b000ca90 tx=0x7f89b0007f10).stop 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 shutdown_connections 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f89a4055060 0x7f89a4057520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c815d460 0x7f89c81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c80600f0 0x7f89c80b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 --2- 172.21.15.181:0/3527160156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89c815d080 0x7f89c80b8d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:03.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 >> 172.21.15.181:0/3527160156 conn(0x7f89c80cfe90 msgr2=0x7f89c8157790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:03.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 shutdown_connections 2020-11-16T22:27:03.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:03.765+0000 7f89c759e700 1 -- 172.21.15.181:0/3527160156 wait complete. 2020-11-16T22:27:03.752 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:27:03.764 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2020-11-16T22:27:02.671981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10809,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":93,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41620,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":39360,"kb_avail":754933100,"statfs":{"total":773094113280,"available":773051494400,"internally_reserved":0,"allocated":2248704,"data_stored":738236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40304640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10809,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":91,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002715"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034281+0000","last_change":"2020-11-16T22:26:52.029699+0000","last_active":"2020-11-16T22:26:52.034281+0000","last_peered":"2020-11-16T22:26:52.034281+0000","last_clean":"2020-11-16T22:26:52.034281+0000","last_became_active":"2020-11-16T22:26:52.029212+0000","last_became_peered":"2020-11-16T22:26:52.029212+0000","last_unstale":"2020-11-16T22:26:52.034281+0000","last_undegraded":"2020-11-16T22:26:52.034281+0000","last_fullsized":"2020-11-16T22:26:52.034281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.014137+0000","last_change":"2020-11-16T22:26:48.228531+0000","last_active":"2020-11-16T22:26:52.014137+0000","last_peered":"2020-11-16T22:26:52.014137+0000","last_clean":"2020-11-16T22:26:52.014137+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:26:52.014137+0000","last_undegraded":"2020-11-16T22:26:52.014137+0000","last_fullsized":"2020-11-16T22:26:52.014137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'27","reported_seq":"37","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:58.086159+0000","last_change":"2020-11-16T22:26:51.646588+0000","last_active":"2020-11-16T22:26:58.086159+0000","last_peered":"2020-11-16T22:26:58.086159+0000","last_clean":"2020-11-16T22:26:58.086159+0000","last_became_active":"2020-11-16T22:26:51.646240+0000","last_became_peered":"2020-11-16T22:26:51.646240+0000","last_unstale":"2020-11-16T22:26:58.086159+0000","last_undegraded":"2020-11-16T22:26:58.086159+0000","last_fullsized":"2020-11-16T22:26:58.086159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":825,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034277+0000","last_change":"2020-11-16T22:26:52.029996+0000","last_active":"2020-11-16T22:26:52.034277+0000","last_peered":"2020-11-16T22:26:52.034277+0000","last_clean":"2020-11-16T22:26:52.034277+0000","last_became_active":"2020-11-16T22:26:52.029574+0000","last_became_peered":"2020-11-16T22:26:52.029574+0000","last_unstale":"2020-11-16T22:26:52.034277+0000","last_undegraded":"2020-11-16T22:26:52.034277+0000","last_fullsized":"2020-11-16T22:26:52.034277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007123+0000","last_change":"2020-11-16T22:26:48.226107+0000","last_active":"2020-11-16T22:26:52.007123+0000","last_peered":"2020-11-16T22:26:52.007123+0000","last_clean":"2020-11-16T22:26:52.007123+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:26:52.007123+0000","last_undegraded":"2020-11-16T22:26:52.007123+0000","last_fullsized":"2020-11-16T22:26:52.007123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'32","reported_seq":"42","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:58.090375+0000","last_change":"2020-11-16T22:26:51.643344+0000","last_active":"2020-11-16T22:26:58.090375+0000","last_peered":"2020-11-16T22:26:58.090375+0000","last_clean":"2020-11-16T22:26:58.090375+0000","last_became_active":"2020-11-16T22:26:51.642906+0000","last_became_peered":"2020-11-16T22:26:51.642906+0000","last_unstale":"2020-11-16T22:26:58.090375+0000","last_undegraded":"2020-11-16T22:26:58.090375+0000","last_fullsized":"2020-11-16T22:26:58.090375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":890,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034598+0000","last_change":"2020-11-16T22:26:52.030213+0000","last_active":"2020-11-16T22:26:52.034598+0000","last_peered":"2020-11-16T22:26:52.034598+0000","last_clean":"2020-11-16T22:26:52.034598+0000","last_became_active":"2020-11-16T22:26:52.029810+0000","last_became_peered":"2020-11-16T22:26:52.029810+0000","last_unstale":"2020-11-16T22:26:52.034598+0000","last_undegraded":"2020-11-16T22:26:52.034598+0000","last_fullsized":"2020-11-16T22:26:52.034598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.017880+0000","last_change":"2020-11-16T22:26:48.145112+0000","last_active":"2020-11-16T22:26:52.017880+0000","last_peered":"2020-11-16T22:26:52.017880+0000","last_clean":"2020-11-16T22:26:52.017880+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:26:52.017880+0000","last_undegraded":"2020-11-16T22:26:52.017880+0000","last_fullsized":"2020-11-16T22:26:52.017880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'35","reported_seq":"45","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.099288+0000","last_change":"2020-11-16T22:26:51.648521+0000","last_active":"2020-11-16T22:26:59.099288+0000","last_peered":"2020-11-16T22:26:59.099288+0000","last_clean":"2020-11-16T22:26:59.099288+0000","last_became_active":"2020-11-16T22:26:51.648069+0000","last_became_peered":"2020-11-16T22:26:51.648069+0000","last_unstale":"2020-11-16T22:26:59.099288+0000","last_undegraded":"2020-11-16T22:26:59.099288+0000","last_fullsized":"2020-11-16T22:26:59.099288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":777,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.230134+0000","last_change":"2020-11-16T22:26:52.227373+0000","last_active":"2020-11-16T22:26:52.230134+0000","last_peered":"2020-11-16T22:26:52.230134+0000","last_clean":"2020-11-16T22:26:52.230134+0000","last_became_active":"2020-11-16T22:26:52.226925+0000","last_became_peered":"2020-11-16T22:26:52.226925+0000","last_unstale":"2020-11-16T22:26:52.230134+0000","last_undegraded":"2020-11-16T22:26:52.230134+0000","last_fullsized":"2020-11-16T22:26:52.230134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007029+0000","last_change":"2020-11-16T22:26:48.223877+0000","last_active":"2020-11-16T22:26:52.007029+0000","last_peered":"2020-11-16T22:26:52.007029+0000","last_clean":"2020-11-16T22:26:52.007029+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:26:52.007029+0000","last_undegraded":"2020-11-16T22:26:52.007029+0000","last_fullsized":"2020-11-16T22:26:52.007029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.223528+0000","last_change":"2020-11-16T22:26:44.683337+0000","last_active":"2020-11-16T22:26:52.223528+0000","last_peered":"2020-11-16T22:26:52.223528+0000","last_clean":"2020-11-16T22:26:52.223528+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:26:52.223528+0000","last_undegraded":"2020-11-16T22:26:52.223528+0000","last_fullsized":"2020-11-16T22:26:52.223528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_clean_scrub_stamp":"2020-11-16T22:26:39.662675+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'29","reported_seq":"39","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.099757+0000","last_change":"2020-11-16T22:26:51.650278+0000","last_active":"2020-11-16T22:26:59.099757+0000","last_peered":"2020-11-16T22:26:59.099757+0000","last_clean":"2020-11-16T22:26:59.099757+0000","last_became_active":"2020-11-16T22:26:51.649870+0000","last_became_peered":"2020-11-16T22:26:51.649870+0000","last_unstale":"2020-11-16T22:26:59.099757+0000","last_undegraded":"2020-11-16T22:26:59.099757+0000","last_fullsized":"2020-11-16T22:26:59.099757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":664,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034073+0000","last_change":"2020-11-16T22:26:52.022648+0000","last_active":"2020-11-16T22:26:52.034073+0000","last_peered":"2020-11-16T22:26:52.034073+0000","last_clean":"2020-11-16T22:26:52.034073+0000","last_became_active":"2020-11-16T22:26:52.020827+0000","last_became_peered":"2020-11-16T22:26:52.020827+0000","last_unstale":"2020-11-16T22:26:52.034073+0000","last_undegraded":"2020-11-16T22:26:52.034073+0000","last_fullsized":"2020-11-16T22:26:52.034073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.006885+0000","last_change":"2020-11-16T22:26:48.143290+0000","last_active":"2020-11-16T22:26:52.006885+0000","last_peered":"2020-11-16T22:26:52.006885+0000","last_clean":"2020-11-16T22:26:52.006885+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:26:52.006885+0000","last_undegraded":"2020-11-16T22:26:52.006885+0000","last_fullsized":"2020-11-16T22:26:52.006885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'33","reported_seq":"43","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:27:01.126921+0000","last_change":"2020-11-16T22:26:51.648794+0000","last_active":"2020-11-16T22:27:01.126921+0000","last_peered":"2020-11-16T22:27:01.126921+0000","last_clean":"2020-11-16T22:27:01.126921+0000","last_became_active":"2020-11-16T22:26:51.648229+0000","last_became_peered":"2020-11-16T22:26:51.648229+0000","last_unstale":"2020-11-16T22:27:01.126921+0000","last_undegraded":"2020-11-16T22:27:01.126921+0000","last_fullsized":"2020-11-16T22:27:01.126921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":866,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.031582+0000","last_change":"2020-11-16T22:26:52.026962+0000","last_active":"2020-11-16T22:26:52.031582+0000","last_peered":"2020-11-16T22:26:52.031582+0000","last_clean":"2020-11-16T22:26:52.031582+0000","last_became_active":"2020-11-16T22:26:52.026541+0000","last_became_peered":"2020-11-16T22:26:52.026541+0000","last_unstale":"2020-11-16T22:26:52.031582+0000","last_undegraded":"2020-11-16T22:26:52.031582+0000","last_fullsized":"2020-11-16T22:26:52.031582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007997+0000","last_change":"2020-11-16T22:26:48.141598+0000","last_active":"2020-11-16T22:26:52.007997+0000","last_peered":"2020-11-16T22:26:52.007997+0000","last_clean":"2020-11-16T22:26:52.007997+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:26:52.007997+0000","last_undegraded":"2020-11-16T22:26:52.007997+0000","last_fullsized":"2020-11-16T22:26:52.007997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'33","reported_seq":"43","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:27:01.128471+0000","last_change":"2020-11-16T22:26:51.648997+0000","last_active":"2020-11-16T22:27:01.128471+0000","last_peered":"2020-11-16T22:27:01.128471+0000","last_clean":"2020-11-16T22:27:01.128471+0000","last_became_active":"2020-11-16T22:26:51.648228+0000","last_became_peered":"2020-11-16T22:26:51.648228+0000","last_unstale":"2020-11-16T22:27:01.128471+0000","last_undegraded":"2020-11-16T22:27:01.128471+0000","last_fullsized":"2020-11-16T22:27:01.128471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4827,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.229631+0000","last_change":"2020-11-16T22:26:52.228532+0000","last_active":"2020-11-16T22:26:52.229631+0000","last_peered":"2020-11-16T22:26:52.229631+0000","last_clean":"2020-11-16T22:26:52.229631+0000","last_became_active":"2020-11-16T22:26:52.227737+0000","last_became_peered":"2020-11-16T22:26:52.227737+0000","last_unstale":"2020-11-16T22:26:52.229631+0000","last_undegraded":"2020-11-16T22:26:52.229631+0000","last_fullsized":"2020-11-16T22:26:52.229631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'32","reported_seq":"42","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.109279+0000","last_change":"2020-11-16T22:26:51.650732+0000","last_active":"2020-11-16T22:26:59.109279+0000","last_peered":"2020-11-16T22:26:59.109279+0000","last_clean":"2020-11-16T22:26:59.109279+0000","last_became_active":"2020-11-16T22:26:51.650314+0000","last_became_peered":"2020-11-16T22:26:51.650314+0000","last_unstale":"2020-11-16T22:26:59.109279+0000","last_undegraded":"2020-11-16T22:26:59.109279+0000","last_fullsized":"2020-11-16T22:26:59.109279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":867,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.005506+0000","last_change":"2020-11-16T22:26:48.228444+0000","last_active":"2020-11-16T22:26:52.005506+0000","last_peered":"2020-11-16T22:26:52.005506+0000","last_clean":"2020-11-16T22:26:52.005506+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:26:52.005506+0000","last_undegraded":"2020-11-16T22:26:52.005506+0000","last_fullsized":"2020-11-16T22:26:52.005506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.018274+0000","last_change":"2020-11-16T22:26:52.014000+0000","last_active":"2020-11-16T22:26:52.018274+0000","last_peered":"2020-11-16T22:26:52.018274+0000","last_clean":"2020-11-16T22:26:52.018274+0000","last_became_active":"2020-11-16T22:26:52.013586+0000","last_became_peered":"2020-11-16T22:26:52.013586+0000","last_unstale":"2020-11-16T22:26:52.018274+0000","last_undegraded":"2020-11-16T22:26:52.018274+0000","last_fullsized":"2020-11-16T22:26:52.018274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007438+0000","last_change":"2020-11-16T22:26:48.225986+0000","last_active":"2020-11-16T22:26:52.007438+0000","last_peered":"2020-11-16T22:26:52.007438+0000","last_clean":"2020-11-16T22:26:52.007438+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:26:52.007438+0000","last_undegraded":"2020-11-16T22:26:52.007438+0000","last_fullsized":"2020-11-16T22:26:52.007438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'29","reported_seq":"39","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.104534+0000","last_change":"2020-11-16T22:26:51.648714+0000","last_active":"2020-11-16T22:26:59.104534+0000","last_peered":"2020-11-16T22:26:59.104534+0000","last_clean":"2020-11-16T22:26:59.104534+0000","last_became_active":"2020-11-16T22:26:51.648304+0000","last_became_peered":"2020-11-16T22:26:51.648304+0000","last_unstale":"2020-11-16T22:26:59.104534+0000","last_undegraded":"2020-11-16T22:26:59.104534+0000","last_fullsized":"2020-11-16T22:26:59.104534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1093,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":10809,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":91,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":21627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":91549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.48}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5212,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366628,"statfs":{"total":96636764160,"available":96631427072,"internally_reserved":0,"allocated":282624,"data_stored":94100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.782}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.331}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5420,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366420,"statfs":{"total":96636764160,"available":96631214080,"internally_reserved":0,"allocated":364544,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5208,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366632,"statfs":{"total":96636764160,"available":96631431168,"internally_reserved":0,"allocated":278528,"data_stored":91750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5830000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.073}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0139999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4489999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":90389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8919999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0179999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.982}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.835}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":89679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3929999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.048}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.452}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0739999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5312,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366528,"statfs":{"total":96636764160,"available":96631324672,"internally_reserved":0,"allocated":319488,"data_stored":92800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":89679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6629999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1960000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3610000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5470000000000002}]}]}]}} 2020-11-16T22:27:04.171 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:27:04.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.321+0000 7f6d8359e700 1 Processor -- start 2020-11-16T22:27:04.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.321+0000 7f6d8359e700 1 -- start start 2020-11-16T22:27:04.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d84061810 0x7f6d8415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:04.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d840cccd0 con 0x7f6d84156a90 2020-11-16T22:27:04.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d840ccf70 con 0x7f6d8415f1a0 2020-11-16T22:27:04.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8415e030 con 0x7f6d84061810 2020-11-16T22:27:04.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d82d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6d8415f1a0 0x7f6d840cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43012/0 (socket says 172.21.15.181:43012) 2020-11-16T22:27:04.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d82d9d700 1 -- 172.21.15.181:0/1532285244 learned_addr learned my addr 172.21.15.181:0/1532285244 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:04.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d81d9b700 1 --2- 172.21.15.181:0/1532285244 >> v2:172.21.15.175:3300/0 conn(0x7f6d84061810 0x7f6d8415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:04.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8159a700 1 -- 172.21.15.181:0/1532285244 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626592898 0 0) 0x7f6d840ccf70 con 0x7f6d8415f1a0 2020-11-16T22:27:04.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8159a700 1 -- 172.21.15.181:0/1532285244 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d68003240 con 0x7f6d8415f1a0 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d81d9b700 1 -- 172.21.15.181:0/1532285244 >> v1:172.21.15.181:6789/0 conn(0x7f6d8415f1a0 legacy=0x7f6d840cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d81d9b700 1 -- 172.21.15.181:0/1532285244 >> v1:172.21.15.175:6789/0 conn(0x7f6d84156a90 legacy=0x7f6d84061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d81d9b700 1 -- 172.21.15.181:0/1532285244 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8415e2d0 con 0x7f6d84061810 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d81d9b700 1 --2- 172.21.15.181:0/1532285244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d84061810 0x7f6d8415ecb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6d7800b9a0 tx=0x7f6d78006df0).ready entity=mon.0 client_cookie=38d1700198dbe283 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8159a700 1 -- 172.21.15.181:0/1532285244 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780097b0 con 0x7f6d84061810 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8159a700 1 -- 172.21.15.181:0/1532285244 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78009910 con 0x7f6d84061810 2020-11-16T22:27:04.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8159a700 1 -- 172.21.15.181:0/1532285244 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78009c90 con 0x7f6d84061810 2020-11-16T22:27:04.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- 172.21.15.181:0/1532285244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d84061810 msgr2=0x7f6d8415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 --2- 172.21.15.181:0/1532285244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d84061810 0x7f6d8415ecb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6d7800b9a0 tx=0x7f6d78006df0).stop 2020-11-16T22:27:04.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- 172.21.15.181:0/1532285244 shutdown_connections 2020-11-16T22:27:04.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 --2- 172.21.15.181:0/1532285244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d84061810 0x7f6d8415ecb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- 172.21.15.181:0/1532285244 >> 172.21.15.181:0/1532285244 conn(0x7f6d840cf630 msgr2=0x7f6d840cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:04.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- 172.21.15.181:0/1532285244 shutdown_connections 2020-11-16T22:27:04.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.325+0000 7f6d8359e700 1 -- 172.21.15.181:0/1532285244 wait complete. 2020-11-16T22:27:04.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 Processor -- start 2020-11-16T22:27:04.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 -- start start 2020-11-16T22:27:04.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:04.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 0x7f6d84158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:04.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 0x7f6d84158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:04.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84159c50 con 0x7f6d8415f1a0 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84159ef0 con 0x7f6d84061810 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8415a190 con 0x7f6d84156a90 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 0x7f6d84158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 0x7f6d84158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36554/0 (socket says 172.21.15.181:36554) 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56944/0 (socket says 172.21.15.181:56944) 2020-11-16T22:27:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.329+0000 7f6d81d9b700 1 -- 172.21.15.181:0/2572852590 learned_addr learned my addr 172.21.15.181:0/2572852590 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d82d9d700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 0x7f6d84158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 -- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 msgr2=0x7f6d84158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 0x7f6d84158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 -- 172.21.15.181:0/2572852590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 msgr2=0x7f6d84158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 0x7f6d84158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 -- 172.21.15.181:0/2572852590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8415a6f0 con 0x7f6d84061810 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d82d9d700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 0x7f6d84158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8259c700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f6d7400ca90 tx=0x7f6d74007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74006870 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d740069d0 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74006d50 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8415a990 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d84159440 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6d74018020 con 0x7f6d84061810 2020-11-16T22:27:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d84044f60 con 0x7f6d84061810 2020-11-16T22:27:04.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 0x7f6d58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:04.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d81d9b700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 0x7f6d58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:04.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d7406d420 con 0x7f6d84061810 2020-11-16T22:27:04.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.333+0000 7f6d81d9b700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 0x7f6d58057520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f6d8414f3e0 tx=0x7f6d78017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:04.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.337+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6d740435b0 con 0x7f6d84061810 2020-11-16T22:27:04.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.485+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d8405c0f0 con 0x7f6d58055060 2020-11-16T22:27:04.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d727fc700 1 -- 172.21.15.181:0/2572852590 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82310 (secure 0 0 0) 0x7f6d8405c0f0 con 0x7f6d58055060 2020-11-16T22:27:04.480 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 msgr2=0x7f6d58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 0x7f6d58057520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f6d8414f3e0 tx=0x7f6d78017000).stop 2020-11-16T22:27:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 msgr2=0x7f6d84158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f6d7400ca90 tx=0x7f6d74007f10).stop 2020-11-16T22:27:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 shutdown_connections 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d58055060 0x7f6d58057520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d8415f1a0 0x7f6d84158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d84061810 0x7f6d84158410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 --2- 172.21.15.181:0/2572852590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d84156a90 0x7f6d84158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 >> 172.21.15.181:0/2572852590 conn(0x7f6d840cf630 msgr2=0x7f6d84061030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:04.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 shutdown_connections 2020-11-16T22:27:04.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:04.493+0000 7f6d8359e700 1 -- 172.21.15.181:0/2572852590 wait complete. 2020-11-16T22:27:04.483 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:27:04.494 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2020-11-16T22:27:02.671981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10809,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":93,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41620,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":39360,"kb_avail":754933100,"statfs":{"total":773094113280,"available":773051494400,"internally_reserved":0,"allocated":2248704,"data_stored":738236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40304640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10809,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":91,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002715"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034281+0000","last_change":"2020-11-16T22:26:52.029699+0000","last_active":"2020-11-16T22:26:52.034281+0000","last_peered":"2020-11-16T22:26:52.034281+0000","last_clean":"2020-11-16T22:26:52.034281+0000","last_became_active":"2020-11-16T22:26:52.029212+0000","last_became_peered":"2020-11-16T22:26:52.029212+0000","last_unstale":"2020-11-16T22:26:52.034281+0000","last_undegraded":"2020-11-16T22:26:52.034281+0000","last_fullsized":"2020-11-16T22:26:52.034281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.014137+0000","last_change":"2020-11-16T22:26:48.228531+0000","last_active":"2020-11-16T22:26:52.014137+0000","last_peered":"2020-11-16T22:26:52.014137+0000","last_clean":"2020-11-16T22:26:52.014137+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:26:52.014137+0000","last_undegraded":"2020-11-16T22:26:52.014137+0000","last_fullsized":"2020-11-16T22:26:52.014137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"23'27","reported_seq":"37","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:58.086159+0000","last_change":"2020-11-16T22:26:51.646588+0000","last_active":"2020-11-16T22:26:58.086159+0000","last_peered":"2020-11-16T22:26:58.086159+0000","last_clean":"2020-11-16T22:26:58.086159+0000","last_became_active":"2020-11-16T22:26:51.646240+0000","last_became_peered":"2020-11-16T22:26:51.646240+0000","last_unstale":"2020-11-16T22:26:58.086159+0000","last_undegraded":"2020-11-16T22:26:58.086159+0000","last_fullsized":"2020-11-16T22:26:58.086159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":825,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034277+0000","last_change":"2020-11-16T22:26:52.029996+0000","last_active":"2020-11-16T22:26:52.034277+0000","last_peered":"2020-11-16T22:26:52.034277+0000","last_clean":"2020-11-16T22:26:52.034277+0000","last_became_active":"2020-11-16T22:26:52.029574+0000","last_became_peered":"2020-11-16T22:26:52.029574+0000","last_unstale":"2020-11-16T22:26:52.034277+0000","last_undegraded":"2020-11-16T22:26:52.034277+0000","last_fullsized":"2020-11-16T22:26:52.034277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007123+0000","last_change":"2020-11-16T22:26:48.226107+0000","last_active":"2020-11-16T22:26:52.007123+0000","last_peered":"2020-11-16T22:26:52.007123+0000","last_clean":"2020-11-16T22:26:52.007123+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:26:52.007123+0000","last_undegraded":"2020-11-16T22:26:52.007123+0000","last_fullsized":"2020-11-16T22:26:52.007123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"23'32","reported_seq":"42","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:58.090375+0000","last_change":"2020-11-16T22:26:51.643344+0000","last_active":"2020-11-16T22:26:58.090375+0000","last_peered":"2020-11-16T22:26:58.090375+0000","last_clean":"2020-11-16T22:26:58.090375+0000","last_became_active":"2020-11-16T22:26:51.642906+0000","last_became_peered":"2020-11-16T22:26:51.642906+0000","last_unstale":"2020-11-16T22:26:58.090375+0000","last_undegraded":"2020-11-16T22:26:58.090375+0000","last_fullsized":"2020-11-16T22:26:58.090375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":890,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034598+0000","last_change":"2020-11-16T22:26:52.030213+0000","last_active":"2020-11-16T22:26:52.034598+0000","last_peered":"2020-11-16T22:26:52.034598+0000","last_clean":"2020-11-16T22:26:52.034598+0000","last_became_active":"2020-11-16T22:26:52.029810+0000","last_became_peered":"2020-11-16T22:26:52.029810+0000","last_unstale":"2020-11-16T22:26:52.034598+0000","last_undegraded":"2020-11-16T22:26:52.034598+0000","last_fullsized":"2020-11-16T22:26:52.034598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.017880+0000","last_change":"2020-11-16T22:26:48.145112+0000","last_active":"2020-11-16T22:26:52.017880+0000","last_peered":"2020-11-16T22:26:52.017880+0000","last_clean":"2020-11-16T22:26:52.017880+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:26:52.017880+0000","last_undegraded":"2020-11-16T22:26:52.017880+0000","last_fullsized":"2020-11-16T22:26:52.017880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"23'35","reported_seq":"45","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.099288+0000","last_change":"2020-11-16T22:26:51.648521+0000","last_active":"2020-11-16T22:26:59.099288+0000","last_peered":"2020-11-16T22:26:59.099288+0000","last_clean":"2020-11-16T22:26:59.099288+0000","last_became_active":"2020-11-16T22:26:51.648069+0000","last_became_peered":"2020-11-16T22:26:51.648069+0000","last_unstale":"2020-11-16T22:26:59.099288+0000","last_undegraded":"2020-11-16T22:26:59.099288+0000","last_fullsized":"2020-11-16T22:26:59.099288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":777,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.230134+0000","last_change":"2020-11-16T22:26:52.227373+0000","last_active":"2020-11-16T22:26:52.230134+0000","last_peered":"2020-11-16T22:26:52.230134+0000","last_clean":"2020-11-16T22:26:52.230134+0000","last_became_active":"2020-11-16T22:26:52.226925+0000","last_became_peered":"2020-11-16T22:26:52.226925+0000","last_unstale":"2020-11-16T22:26:52.230134+0000","last_undegraded":"2020-11-16T22:26:52.230134+0000","last_fullsized":"2020-11-16T22:26:52.230134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007029+0000","last_change":"2020-11-16T22:26:48.223877+0000","last_active":"2020-11-16T22:26:52.007029+0000","last_peered":"2020-11-16T22:26:52.007029+0000","last_clean":"2020-11-16T22:26:52.007029+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:26:52.007029+0000","last_undegraded":"2020-11-16T22:26:52.007029+0000","last_fullsized":"2020-11-16T22:26:52.007029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.223528+0000","last_change":"2020-11-16T22:26:44.683337+0000","last_active":"2020-11-16T22:26:52.223528+0000","last_peered":"2020-11-16T22:26:52.223528+0000","last_clean":"2020-11-16T22:26:52.223528+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:26:52.223528+0000","last_undegraded":"2020-11-16T22:26:52.223528+0000","last_fullsized":"2020-11-16T22:26:52.223528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_clean_scrub_stamp":"2020-11-16T22:26:39.662675+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'29","reported_seq":"39","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.099757+0000","last_change":"2020-11-16T22:26:51.650278+0000","last_active":"2020-11-16T22:26:59.099757+0000","last_peered":"2020-11-16T22:26:59.099757+0000","last_clean":"2020-11-16T22:26:59.099757+0000","last_became_active":"2020-11-16T22:26:51.649870+0000","last_became_peered":"2020-11-16T22:26:51.649870+0000","last_unstale":"2020-11-16T22:26:59.099757+0000","last_undegraded":"2020-11-16T22:26:59.099757+0000","last_fullsized":"2020-11-16T22:26:59.099757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":664,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.034073+0000","last_change":"2020-11-16T22:26:52.022648+0000","last_active":"2020-11-16T22:26:52.034073+0000","last_peered":"2020-11-16T22:26:52.034073+0000","last_clean":"2020-11-16T22:26:52.034073+0000","last_became_active":"2020-11-16T22:26:52.020827+0000","last_became_peered":"2020-11-16T22:26:52.020827+0000","last_unstale":"2020-11-16T22:26:52.034073+0000","last_undegraded":"2020-11-16T22:26:52.034073+0000","last_fullsized":"2020-11-16T22:26:52.034073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.006885+0000","last_change":"2020-11-16T22:26:48.143290+0000","last_active":"2020-11-16T22:26:52.006885+0000","last_peered":"2020-11-16T22:26:52.006885+0000","last_clean":"2020-11-16T22:26:52.006885+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:26:52.006885+0000","last_undegraded":"2020-11-16T22:26:52.006885+0000","last_fullsized":"2020-11-16T22:26:52.006885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"23'33","reported_seq":"43","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:27:01.126921+0000","last_change":"2020-11-16T22:26:51.648794+0000","last_active":"2020-11-16T22:27:01.126921+0000","last_peered":"2020-11-16T22:27:01.126921+0000","last_clean":"2020-11-16T22:27:01.126921+0000","last_became_active":"2020-11-16T22:26:51.648229+0000","last_became_peered":"2020-11-16T22:26:51.648229+0000","last_unstale":"2020-11-16T22:27:01.126921+0000","last_undegraded":"2020-11-16T22:27:01.126921+0000","last_fullsized":"2020-11-16T22:27:01.126921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":866,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.031582+0000","last_change":"2020-11-16T22:26:52.026962+0000","last_active":"2020-11-16T22:26:52.031582+0000","last_peered":"2020-11-16T22:26:52.031582+0000","last_clean":"2020-11-16T22:26:52.031582+0000","last_became_active":"2020-11-16T22:26:52.026541+0000","last_became_peered":"2020-11-16T22:26:52.026541+0000","last_unstale":"2020-11-16T22:26:52.031582+0000","last_undegraded":"2020-11-16T22:26:52.031582+0000","last_fullsized":"2020-11-16T22:26:52.031582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007997+0000","last_change":"2020-11-16T22:26:48.141598+0000","last_active":"2020-11-16T22:26:52.007997+0000","last_peered":"2020-11-16T22:26:52.007997+0000","last_clean":"2020-11-16T22:26:52.007997+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:26:52.007997+0000","last_undegraded":"2020-11-16T22:26:52.007997+0000","last_fullsized":"2020-11-16T22:26:52.007997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'33","reported_seq":"43","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:27:01.128471+0000","last_change":"2020-11-16T22:26:51.648997+0000","last_active":"2020-11-16T22:27:01.128471+0000","last_peered":"2020-11-16T22:27:01.128471+0000","last_clean":"2020-11-16T22:27:01.128471+0000","last_became_active":"2020-11-16T22:26:51.648228+0000","last_became_peered":"2020-11-16T22:26:51.648228+0000","last_unstale":"2020-11-16T22:27:01.128471+0000","last_undegraded":"2020-11-16T22:27:01.128471+0000","last_fullsized":"2020-11-16T22:27:01.128471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4827,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.229631+0000","last_change":"2020-11-16T22:26:52.228532+0000","last_active":"2020-11-16T22:26:52.229631+0000","last_peered":"2020-11-16T22:26:52.229631+0000","last_clean":"2020-11-16T22:26:52.229631+0000","last_became_active":"2020-11-16T22:26:52.227737+0000","last_became_peered":"2020-11-16T22:26:52.227737+0000","last_unstale":"2020-11-16T22:26:52.229631+0000","last_undegraded":"2020-11-16T22:26:52.229631+0000","last_fullsized":"2020-11-16T22:26:52.229631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'32","reported_seq":"42","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.109279+0000","last_change":"2020-11-16T22:26:51.650732+0000","last_active":"2020-11-16T22:26:59.109279+0000","last_peered":"2020-11-16T22:26:59.109279+0000","last_clean":"2020-11-16T22:26:59.109279+0000","last_became_active":"2020-11-16T22:26:51.650314+0000","last_became_peered":"2020-11-16T22:26:51.650314+0000","last_unstale":"2020-11-16T22:26:59.109279+0000","last_undegraded":"2020-11-16T22:26:59.109279+0000","last_fullsized":"2020-11-16T22:26:59.109279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":867,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.005506+0000","last_change":"2020-11-16T22:26:48.228444+0000","last_active":"2020-11-16T22:26:52.005506+0000","last_peered":"2020-11-16T22:26:52.005506+0000","last_clean":"2020-11-16T22:26:52.005506+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:26:52.005506+0000","last_undegraded":"2020-11-16T22:26:52.005506+0000","last_fullsized":"2020-11-16T22:26:52.005506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.018274+0000","last_change":"2020-11-16T22:26:52.014000+0000","last_active":"2020-11-16T22:26:52.018274+0000","last_peered":"2020-11-16T22:26:52.018274+0000","last_clean":"2020-11-16T22:26:52.018274+0000","last_became_active":"2020-11-16T22:26:52.013586+0000","last_became_peered":"2020-11-16T22:26:52.013586+0000","last_unstale":"2020-11-16T22:26:52.018274+0000","last_undegraded":"2020-11-16T22:26:52.018274+0000","last_fullsized":"2020-11-16T22:26:52.018274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:51.621960+0000","last_clean_scrub_stamp":"2020-11-16T22:26:51.621960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:26:52.007438+0000","last_change":"2020-11-16T22:26:48.225986+0000","last_active":"2020-11-16T22:26:52.007438+0000","last_peered":"2020-11-16T22:26:52.007438+0000","last_clean":"2020-11-16T22:26:52.007438+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:26:52.007438+0000","last_undegraded":"2020-11-16T22:26:52.007438+0000","last_fullsized":"2020-11-16T22:26:52.007438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'29","reported_seq":"39","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-16T22:26:59.104534+0000","last_change":"2020-11-16T22:26:51.648714+0000","last_active":"2020-11-16T22:26:59.104534+0000","last_peered":"2020-11-16T22:26:59.104534+0000","last_clean":"2020-11-16T22:26:59.104534+0000","last_became_active":"2020-11-16T22:26:51.648304+0000","last_became_peered":"2020-11-16T22:26:51.648304+0000","last_unstale":"2020-11-16T22:26:59.104534+0000","last_undegraded":"2020-11-16T22:26:59.104534+0000","last_fullsized":"2020-11-16T22:26:59.104534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:50.618017+0000","last_clean_scrub_stamp":"2020-11-16T22:26:50.618017+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1093,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":10809,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":91,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":21627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":91549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.48}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5212,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366628,"statfs":{"total":96636764160,"available":96631427072,"internally_reserved":0,"allocated":282624,"data_stored":94100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.782}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.331}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5420,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366420,"statfs":{"total":96636764160,"available":96631214080,"internally_reserved":0,"allocated":364544,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5208,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366632,"statfs":{"total":96636764160,"available":96631431168,"internally_reserved":0,"allocated":278528,"data_stored":91750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5830000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.073}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0139999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4489999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":90389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8919999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0179999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.982}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.835}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":89679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3929999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.048}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.452}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0739999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5312,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366528,"statfs":{"total":96636764160,"available":96631324672,"internally_reserved":0,"allocated":319488,"data_stored":92800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":89679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6629999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1960000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3610000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5470000000000002}]}]}]}} 2020-11-16T22:27:06.331 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T22:27:06.332 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-16T22:27:06.332 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-16T22:27:06.332 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-16T22:27:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 Processor -- start 2020-11-16T22:27:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 -- start start 2020-11-16T22:27:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55c413e870 0x7f55c413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c4061810 0x7f55c4160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c415d180 con 0x7f55c415fdc0 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.477+0000 7f55ca49b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c4160730 con 0x7f55c413e870 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c413f140 con 0x7f55c4061810 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55c413e870 0x7f55c413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55c413e870 0x7f55c413ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56950/0 (socket says 172.21.15.181:56950) 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 -- 172.21.15.181:0/298703493 learned_addr learned my addr 172.21.15.181:0/298703493 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9c9a700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c4061810 0x7f55c4160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 -- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c4061810 msgr2=0x7f55c4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c4061810 0x7f55c4160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 -- 172.21.15.181:0/298703493 >> v1:172.21.15.175:6789/0 conn(0x7f55c415fdc0 legacy=0x7f55c4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 -- 172.21.15.181:0/298703493 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c415d810 con 0x7f55c413e870 2020-11-16T22:27:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55c9499700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c413e870 0x7f55c413ec50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f55c0006fd0 tx=0x7f55c0006f40).ready entity=mon.1 client_cookie=f11f58663d4fca99 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55b7fff700 1 -- 172.21.15.181:0/298703493 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000cad0 con 0x7f55c413e870 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55b7fff700 1 -- 172.21.15.181:0/298703493 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000cc30 con 0x7f55c413e870 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55b7fff700 1 -- 172.21.15.181:0/298703493 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0010410 con 0x7f55c413e870 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c413e870 msgr2=0x7f55c413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c413e870 0x7f55c413ec50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f55c0006fd0 tx=0x7f55c0006f40).stop 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- 172.21.15.181:0/298703493 shutdown_connections 2020-11-16T22:27:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c4061810 0x7f55c4160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 --2- 172.21.15.181:0/298703493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c413e870 0x7f55c413ec50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- 172.21.15.181:0/298703493 >> 172.21.15.181:0/298703493 conn(0x7f55c40cf630 msgr2=0x7f55c40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- 172.21.15.181:0/298703493 shutdown_connections 2020-11-16T22:27:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.481+0000 7f55ca49b700 1 -- 172.21.15.181:0/298703493 wait complete. 2020-11-16T22:27:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 Processor -- start 2020-11-16T22:27:06.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- start start 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c4061810 0x7f55c40ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c413e870 0x7f55c40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b6f10 con 0x7f55c413e870 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b7100 con 0x7f55c4061810 2020-11-16T22:27:06.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b73a0 con 0x7f55c415fdc0 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36568/0 (socket says 172.21.15.181:36568) 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 -- 172.21.15.181:0/1197944137 learned_addr learned my addr 172.21.15.181:0/1197944137 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9499700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c4061810 0x7f55c40ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c8c98700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c413e870 0x7f55c40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 -- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c4061810 msgr2=0x7f55c40ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c4061810 0x7f55c40ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 -- 172.21.15.181:0/1197944137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c413e870 msgr2=0x7f55c40ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c413e870 0x7f55c40ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 -- 172.21.15.181:0/1197944137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40b7640 con 0x7f55c415fdc0 2020-11-16T22:27:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55c9c9a700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f55bc00ef50 tx=0x7f55bc0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00cb40 con 0x7f55c415fdc0 2020-11-16T22:27:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00cca0 con 0x7f55c415fdc0 2020-11-16T22:27:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c40b78e0 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0123f0 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.485+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c40b8510 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c4044f60 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f55bc022070 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55b57fa700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 0x7f559c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f55bc06e450 con 0x7f55c415fdc0 2020-11-16T22:27:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55c9499700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 0x7f559c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.489+0000 7f55c9499700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 0x7f559c057550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f55c40d1660 tx=0x7f55c41459d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.493+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f55bc0446b0 con 0x7f55c415fdc0 2020-11-16T22:27:06.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.701+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f55c405c310 con 0x7f55c415fdc0 2020-11-16T22:27:06.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55b57fa700 1 -- 172.21.15.181:0/1197944137 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f55bc016020 con 0x7f55c415fdc0 2020-11-16T22:27:06.694 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:06.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 msgr2=0x7f559c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 0x7f559c057550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f55c40d1660 tx=0x7f55c41459d0).stop 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 msgr2=0x7f55c40b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f55bc00ef50 tx=0x7f55bc0091d0).stop 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 shutdown_connections 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f559c055090 0x7f559c057550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415fdc0 0x7f55c40b8020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c4061810 0x7f55c40ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 --2- 172.21.15.181:0/1197944137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c413e870 0x7f55c40ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 >> 172.21.15.181:0/1197944137 conn(0x7f55c40cf630 msgr2=0x7f55c40b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:06.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 shutdown_connections 2020-11-16T22:27:06.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.709+0000 7f55ca49b700 1 -- 172.21.15.181:0/1197944137 wait complete. 2020-11-16T22:27:06.710 INFO:teuthology.orchestra.run.smithi181.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-16T22:27:06.710 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:06.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.841+0000 7f4bb1b35700 1 Processor -- start 2020-11-16T22:27:06.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.841+0000 7f4bb1b35700 1 -- start start 2020-11-16T22:27:06.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4bac1492c0 0x7f4bac0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac061b50 con 0x7f4bac145a30 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac149700 con 0x7f4bac1492c0 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0618b0 con 0x7f4bac0816e0 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36574/0 (socket says 172.21.15.181:36574) 2020-11-16T22:27:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 -- 172.21.15.181:0/2235192962 learned_addr learned my addr 172.21.15.181:0/2235192962 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1334700 1 --2- 172.21.15.181:0/2235192962 >> v2:172.21.15.175:3300/0 conn(0x7f4bac1492c0 0x7f4bac0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 -- 172.21.15.181:0/2235192962 >> v2:172.21.15.175:3300/0 conn(0x7f4bac1492c0 msgr2=0x7f4bac0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:06.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 --2- 172.21.15.181:0/2235192962 >> v2:172.21.15.175:3300/0 conn(0x7f4bac1492c0 0x7f4bac0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 -- 172.21.15.181:0/2235192962 >> v1:172.21.15.175:6789/0 conn(0x7f4bac145a30 legacy=0x7f4bac080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 -- 172.21.15.181:0/2235192962 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0cb930 con 0x7f4bac0816e0 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4babfff700 1 --2- 172.21.15.181:0/2235192962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4b94006fd0 tx=0x7f4b94006eb0).ready entity=mon.2 client_cookie=671e9c3678cabd46 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bab7fe700 1 -- 172.21.15.181:0/2235192962 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940098d0 con 0x7f4bac0816e0 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bab7fe700 1 -- 172.21.15.181:0/2235192962 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94009a30 con 0x7f4bac0816e0 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bab7fe700 1 -- 172.21.15.181:0/2235192962 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94009db0 con 0x7f4bac0816e0 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- 172.21.15.181:0/2235192962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 msgr2=0x7f4bac145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/2235192962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4b94006fd0 tx=0x7f4b94006eb0).stop 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- 172.21.15.181:0/2235192962 shutdown_connections 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/2235192962 >> v2:172.21.15.175:3300/0 conn(0x7f4bac1492c0 0x7f4bac0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/2235192962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac0816e0 0x7f4bac145e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- 172.21.15.181:0/2235192962 >> 172.21.15.181:0/2235192962 conn(0x7f4bac0d68c0 msgr2=0x7f4bac0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- 172.21.15.181:0/2235192962 shutdown_connections 2020-11-16T22:27:06.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- 172.21.15.181:0/2235192962 wait complete. 2020-11-16T22:27:06.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 Processor -- start 2020-11-16T22:27:06.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.845+0000 7f4bb1b35700 1 -- start start 2020-11-16T22:27:06.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bac0816e0 0x7f4bac160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 0x7f4bac171f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 0x7f4bac15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac160b30 con 0x7f4bac0816e0 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac16f720 con 0x7f4bac1492c0 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac16f9c0 con 0x7f4bac145a30 2020-11-16T22:27:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 0x7f4bac171f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1334700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 0x7f4bac15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1334700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 0x7f4bac15f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56974/0 (socket says 172.21.15.181:56974) 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1334700 1 -- 172.21.15.181:0/1950821642 learned_addr learned my addr 172.21.15.181:0/1950821642 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb0b33700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bac0816e0 0x7f4bac160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 -- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 msgr2=0x7f4bac15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 0x7f4bac15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 -- 172.21.15.181:0/1950821642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bac0816e0 msgr2=0x7f4bac160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bac0816e0 0x7f4bac160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 -- 172.21.15.181:0/1950821642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac16fc60 con 0x7f4bac145a30 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4babfff700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 0x7f4bac171f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4b94006fd0 tx=0x7f4b94015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94011c80 con 0x7f4bac145a30 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94011de0 con 0x7f4bac145a30 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac16ff00 con 0x7f4bac145a30 2020-11-16T22:27:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9401e670 con 0x7f4bac145a30 2020-11-16T22:27:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac16f310 con 0x7f4bac145a30 2020-11-16T22:27:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.849+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac16f590 con 0x7f4bac145a30 2020-11-16T22:27:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.853+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4b9401a040 con 0x7f4bac145a30 2020-11-16T22:27:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.853+0000 7f4ba8ff9700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 0x7f4b88057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.853+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b9406ef90 con 0x7f4bac145a30 2020-11-16T22:27:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.853+0000 7f4bb0b33700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 0x7f4b88057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.853+0000 7f4bb0b33700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 0x7f4b88057550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f4b9c004250 tx=0x7f4b9c007f10).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:06.857+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4b94045320 con 0x7f4bac145a30 2020-11-16T22:27:06.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.009+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bac044f60 con 0x7f4bac145a30 2020-11-16T22:27:06.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4ba8ff9700 1 -- 172.21.15.181:0/1950821642 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f4b94030200 con 0x7f4bac145a30 2020-11-16T22:27:06.998 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:06.998 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:06.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 msgr2=0x7f4b88057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:06.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 0x7f4b88057550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f4b9c004250 tx=0x7f4b9c007f10).stop 2020-11-16T22:27:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 msgr2=0x7f4bac171f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 0x7f4bac171f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4b94006fd0 tx=0x7f4b94015000).stop 2020-11-16T22:27:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 shutdown_connections 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4b88055090 0x7f4b88057550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bac1492c0 0x7f4bac15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bac0816e0 0x7f4bac160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 --2- 172.21.15.181:0/1950821642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bac145a30 0x7f4bac171f40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 >> 172.21.15.181:0/1950821642 conn(0x7f4bac0d68c0 msgr2=0x7f4bac080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 shutdown_connections 2020-11-16T22:27:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.013+0000 7f4bb1b35700 1 -- 172.21.15.181:0/1950821642 wait complete. 2020-11-16T22:27:07.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:07.410 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:07.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 Processor -- start 2020-11-16T22:27:07.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- start start 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 0x7f58f415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40cccd0 con 0x7f58f4156a90 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40ccf70 con 0x7f58f415f1a0 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415e030 con 0x7f58f4061810 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f9585700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 0x7f58f415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fa587700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f58f415f1a0 0x7f58f40cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43054/0 (socket says 172.21.15.181:43054) 2020-11-16T22:27:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fa587700 1 -- 172.21.15.181:0/2087329686 learned_addr learned my addr 172.21.15.181:0/2087329686 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f9585700 1 -- 172.21.15.181:0/2087329686 >> v1:172.21.15.181:6789/0 conn(0x7f58f415f1a0 legacy=0x7f58f40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f9585700 1 -- 172.21.15.181:0/2087329686 >> v1:172.21.15.175:6789/0 conn(0x7f58f4156a90 legacy=0x7f58f4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f9585700 1 -- 172.21.15.181:0/2087329686 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415e2d0 con 0x7f58f4061810 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f9585700 1 --2- 172.21.15.181:0/2087329686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 0x7f58f415ecb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f58e800b9d0 tx=0x7f58e8006df0).ready entity=mon.2 client_cookie=d9712d1dc4d0d434 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f8d84700 1 -- 172.21.15.181:0/2087329686 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8009830 con 0x7f58f4061810 2020-11-16T22:27:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f8d84700 1 -- 172.21.15.181:0/2087329686 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8009990 con 0x7f58f4061810 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58f8d84700 1 -- 172.21.15.181:0/2087329686 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8009d10 con 0x7f58f4061810 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- 172.21.15.181:0/2087329686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 msgr2=0x7f58f415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 --2- 172.21.15.181:0/2087329686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 0x7f58f415ecb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f58e800b9d0 tx=0x7f58e8006df0).stop 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- 172.21.15.181:0/2087329686 shutdown_connections 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 --2- 172.21.15.181:0/2087329686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f4061810 0x7f58f415ecb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.561+0000 7f58fad88700 1 -- 172.21.15.181:0/2087329686 >> 172.21.15.181:0/2087329686 conn(0x7f58f40cf630 msgr2=0x7f58f40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- 172.21.15.181:0/2087329686 shutdown_connections 2020-11-16T22:27:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- 172.21.15.181:0/2087329686 wait complete. 2020-11-16T22:27:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 Processor -- start 2020-11-16T22:27:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- start start 2020-11-16T22:27:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58f4061810 0x7f58f4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415ad90 con 0x7f58f4061810 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415aff0 con 0x7f58f4156a90 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fad88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415b290 con 0x7f58f415f1a0 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58f9585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.565+0000 7f58fa587700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56986/0 (socket says 172.21.15.181:56986) 2020-11-16T22:27:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58fa587700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36596/0 (socket says 172.21.15.181:36596) 2020-11-16T22:27:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 -- 172.21.15.181:0/3544360714 learned_addr learned my addr 172.21.15.181:0/3544360714 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9d86700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58f4061810 0x7f58f4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 -- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 msgr2=0x7f58f415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 -- 172.21.15.181:0/3544360714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58f4061810 msgr2=0x7f58f4159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58f4061810 0x7f58f4159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 -- 172.21.15.181:0/3544360714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415b530 con 0x7f58f4156a90 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58fa587700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9585700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f58e800b150 tx=0x7f58e8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8011dd0 con 0x7f58f4156a90 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e801e460 con 0x7f58f4156a90 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f415b7d0 con 0x7f58f4156a90 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e801e7e0 con 0x7f58f4156a90 2020-11-16T22:27:07.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f416e970 con 0x7f58f4156a90 2020-11-16T22:27:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4156720 con 0x7f58f4156a90 2020-11-16T22:27:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f58e8025030 con 0x7f58f4156a90 2020-11-16T22:27:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 0x7f58cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f58e806f5a0 con 0x7f58f4156a90 2020-11-16T22:27:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9d86700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 0x7f58cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.569+0000 7f58f9d86700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 0x7f58cc057520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f58ec0039b0 tx=0x7f58ec007ef0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.577+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f58f4156720 con 0x7f58f4156a90 2020-11-16T22:27:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f58f4156720 con 0x7f58f4156a90 2020-11-16T22:27:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58e5ffb700 1 -- 172.21.15.181:0/3544360714 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f58f4156720 con 0x7f58f4156a90 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 msgr2=0x7f58cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 0x7f58cc057520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f58ec0039b0 tx=0x7f58ec007ef0).stop 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 msgr2=0x7f58f414b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f58e800b150 tx=0x7f58e8015000).stop 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 shutdown_connections 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58cc055060 0x7f58cc057520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f415f1a0 0x7f58f415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58f4061810 0x7f58f4159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 --2- 172.21.15.181:0/3544360714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f4156a90 0x7f58f414b8d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 >> 172.21.15.181:0/3544360714 conn(0x7f58f40cf630 msgr2=0x7f58f4065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 shutdown_connections 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.733+0000 7f58fad88700 1 -- 172.21.15.181:0/3544360714 wait complete. 2020-11-16T22:27:07.765 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:07.766 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.909+0000 7fda6105c700 1 Processor -- start 2020-11-16T22:27:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.909+0000 7fda6105c700 1 -- start start 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0cf030 con 0x7fda5c0c85e0 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c160cc0 con 0x7fda5c160810 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c8a40 con 0x7fda5c160430 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36604/0 (socket says 172.21.15.181:36604) 2020-11-16T22:27:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1482393332 learned_addr learned my addr 172.21.15.181:0/1482393332 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1482393332 >> v1:172.21.15.181:6789/0 conn(0x7fda5c160810 legacy=0x7fda5c0600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1482393332 >> v1:172.21.15.175:6789/0 conn(0x7fda5c0c85e0 legacy=0x7fda5c162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda59d9b700 1 -- 172.21.15.181:0/1482393332 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737072613 0 0) 0x7fda5c160cc0 con 0x7fda5c160810 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1482393332 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c161a10 con 0x7fda5c160430 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/1482393332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fda4400ba20 tx=0x7fda44006eb0).ready entity=mon.2 client_cookie=4d4a7637f2062a11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda59d9b700 1 -- 172.21.15.181:0/1482393332 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440098d0 con 0x7fda5c160430 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda59d9b700 1 -- 172.21.15.181:0/1482393332 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44009a30 con 0x7fda5c160430 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda59d9b700 1 -- 172.21.15.181:0/1482393332 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44009db0 con 0x7fda5c160430 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- 172.21.15.181:0/1482393332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 msgr2=0x7fda5c161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 --2- 172.21.15.181:0/1482393332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fda4400ba20 tx=0x7fda44006eb0).stop 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- 172.21.15.181:0/1482393332 shutdown_connections 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 --2- 172.21.15.181:0/1482393332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160430 0x7fda5c161460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- 172.21.15.181:0/1482393332 >> 172.21.15.181:0/1482393332 conn(0x7fda5c0d3680 msgr2=0x7fda5c0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- 172.21.15.181:0/1482393332 shutdown_connections 2020-11-16T22:27:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.913+0000 7fda6105c700 1 -- 172.21.15.181:0/1482393332 wait complete. 2020-11-16T22:27:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 Processor -- start 2020-11-16T22:27:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- start start 2020-11-16T22:27:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5c0c85e0 0x7fda5c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160810 0x7fda5c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15aac0 con 0x7fda5c0c85e0 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c170c00 con 0x7fda5c159de0 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c170ea0 con 0x7fda5c160810 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57000/0 (socket says 172.21.15.181:57000) 2020-11-16T22:27:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 -- 172.21.15.181:0/391542349 learned_addr learned my addr 172.21.15.181:0/391542349 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:07.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5a59c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160810 0x7fda5c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5c0c85e0 0x7fda5c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 -- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160810 msgr2=0x7fda5c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160810 0x7fda5c159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 -- 172.21.15.181:0/391542349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5c0c85e0 msgr2=0x7fda5c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5c0c85e0 0x7fda5c159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 -- 172.21.15.181:0/391542349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c171140 con 0x7fda5c159de0 2020-11-16T22:27:07.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda5b59e700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fda4c00cae0 tx=0x7fda4c007cd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:07.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00a770 con 0x7fda5c159de0 2020-11-16T22:27:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00a8f0 con 0x7fda5c159de0 2020-11-16T22:27:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c1713e0 con 0x7fda5c159de0 2020-11-16T22:27:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00acb0 con 0x7fda5c159de0 2020-11-16T22:27:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.917+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c15f1d0 con 0x7fda5c159de0 2020-11-16T22:27:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c171570 con 0x7fda5c159de0 2020-11-16T22:27:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fda4c019030 con 0x7fda5c159de0 2020-11-16T22:27:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda52ffd700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 0x7fda38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 0x7fda38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda4c06d6f0 con 0x7fda5c159de0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.921+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 0x7fda38057520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fda5c0681b0 tx=0x7fda44015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:07.929+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fda4c0438f0 con 0x7fda5c159de0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda5c15ac80 con 0x7fda5c159de0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda52ffd700 1 -- 172.21.15.181:0/391542349 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda4c046f10 con 0x7fda5c159de0 2020-11-16T22:27:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 msgr2=0x7fda38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 0x7fda38057520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fda5c0681b0 tx=0x7fda44015000).stop 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 msgr2=0x7fda5c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.085+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fda4c00cae0 tx=0x7fda4c007cd0).stop 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 shutdown_connections 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fda38055060 0x7fda38057520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c159de0 0x7fda5c15a260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5c0c85e0 0x7fda5c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 --2- 172.21.15.181:0/391542349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5c160810 0x7fda5c159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 >> 172.21.15.181:0/391542349 conn(0x7fda5c0d3680 msgr2=0x7fda5c0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 shutdown_connections 2020-11-16T22:27:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.089+0000 7fda6105c700 1 -- 172.21.15.181:0/391542349 wait complete. 2020-11-16T22:27:08.115 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:08.116 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:08.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 Processor -- start 2020-11-16T22:27:08.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 -- start start 2020-11-16T22:27:08.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc0015e270 0x7fdc0015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 0x7fdc000667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc00064050 con 0x7fdc00063b80 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000642f0 con 0x7fdc0015e270 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc06ad4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00064590 con 0x7fdc000b6c20 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36618/0 (socket says 172.21.15.181:36618) 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 -- 172.21.15.181:0/2443350376 learned_addr learned my addr 172.21.15.181:0/2443350376 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:08.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc04870700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015e270 0x7fdc0015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdbfffff700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 0x7fdc000667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 -- 172.21.15.181:0/2443350376 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015e270 msgr2=0x7fdc0015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015e270 0x7fdc0015e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 -- 172.21.15.181:0/2443350376 >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 msgr2=0x7fdc000667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 0x7fdc000667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdc05071700 1 -- 172.21.15.181:0/2443350376 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00064830 con 0x7fdc000b6c20 2020-11-16T22:27:08.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.261+0000 7fdbfffff700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 0x7fdc000667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:08.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc05071700 1 --2- 172.21.15.181:0/2443350376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fdc000d46a0 tx=0x7fdbf8009290).ready entity=mon.2 client_cookie=c3491688d9abb5c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdbff7fe700 1 -- 172.21.15.181:0/2443350376 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800bbc0 con 0x7fdc000b6c20 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdbff7fe700 1 -- 172.21.15.181:0/2443350376 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800bd20 con 0x7fdc000b6c20 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdbff7fe700 1 -- 172.21.15.181:0/2443350376 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8011610 con 0x7fdc000b6c20 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- 172.21.15.181:0/2443350376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 msgr2=0x7fdc00066ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/2443350376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fdc000d46a0 tx=0x7fdbf8009290).stop 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- 172.21.15.181:0/2443350376 shutdown_connections 2020-11-16T22:27:08.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/2443350376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc000b6c20 0x7fdc00066ca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015e270 0x7fdc0015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/2443350376 >> v2:172.21.15.175:3300/0 conn(0x7fdc00063b80 0x7fdc000667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- 172.21.15.181:0/2443350376 >> 172.21.15.181:0/2443350376 conn(0x7fdc000d4790 msgr2=0x7fdc000d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- 172.21.15.181:0/2443350376 shutdown_connections 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- 172.21.15.181:0/2443350376 wait complete. 2020-11-16T22:27:08.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 Processor -- start 2020-11-16T22:27:08.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.265+0000 7fdc06ad4700 1 -- start start 2020-11-16T22:27:08.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc000b6c20 0x7fdc00155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc0015e270 0x7fdc001560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00170520 con 0x7fdc000b6c20 2020-11-16T22:27:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001707c0 con 0x7fdc00063b80 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00170a60 con 0x7fdc0015e270 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc05071700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc0015e270 0x7fdc001560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57012/0 (socket says 172.21.15.181:57012) 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 -- 172.21.15.181:0/1249030043 learned_addr learned my addr 172.21.15.181:0/1249030043 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdbfffff700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc000b6c20 0x7fdc00155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 -- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc0015e270 msgr2=0x7fdc001560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc0015e270 0x7fdc001560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 -- 172.21.15.181:0/1249030043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc000b6c20 msgr2=0x7fdc00155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc000b6c20 0x7fdc00155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 -- 172.21.15.181:0/1249030043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00170d00 con 0x7fdc00063b80 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc04870700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fdbf000eba0 tx=0x7fdbf0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf000c8e0 con 0x7fdc00063b80 2020-11-16T22:27:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf000ca40 con 0x7fdc00063b80 2020-11-16T22:27:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc00170fa0 con 0x7fdc00063b80 2020-11-16T22:27:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf000cdc0 con 0x7fdc00063b80 2020-11-16T22:27:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0016d010 con 0x7fdc00063b80 2020-11-16T22:27:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.269+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc00069eb0 con 0x7fdc00063b80 2020-11-16T22:27:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.273+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdbf0012600 con 0x7fdc00063b80 2020-11-16T22:27:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.273+0000 7fdbfcff9700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 0x7fdbd8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.273+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbf006e080 con 0x7fdc00063b80 2020-11-16T22:27:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.273+0000 7fdbfffff700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 0x7fdbd8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.277+0000 7fdbfffff700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 0x7fdbd8057520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fdc000d3cb0 tx=0x7fdbf4008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.281+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdbf0044280 con 0x7fdc00063b80 2020-11-16T22:27:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.429+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc0015eb90 con 0x7fdc00063b80 2020-11-16T22:27:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.429+0000 7fdbfcff9700 1 -- 172.21.15.181:0/1249030043 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdbf00478a0 con 0x7fdc00063b80 2020-11-16T22:27:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 msgr2=0x7fdbd8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 0x7fdbd8057520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fdc000d3cb0 tx=0x7fdbf4008040).stop 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 msgr2=0x7fdc001643c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fdbf000eba0 tx=0x7fdbf0006df0).stop 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 shutdown_connections 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdbd8055060 0x7fdbd8057520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc0015e270 0x7fdc001560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc00063b80 0x7fdc001643c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 --2- 172.21.15.181:0/1249030043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc000b6c20 0x7fdc00155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 >> 172.21.15.181:0/1249030043 conn(0x7fdc000d4790 msgr2=0x7fdc000d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 shutdown_connections 2020-11-16T22:27:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.433+0000 7fdc06ad4700 1 -- 172.21.15.181:0/1249030043 wait complete. 2020-11-16T22:27:08.668 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:08.669 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:08.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.821+0000 7f07465e3700 1 Processor -- start 2020-11-16T22:27:08.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.821+0000 7f07465e3700 1 -- start start 2020-11-16T22:27:08.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f07465e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f07401461a0 0x7f0740061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f07465e3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400c8420 con 0x7f0740145dc0 2020-11-16T22:27:08.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f07465e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400c86c0 con 0x7f07400615f0 2020-11-16T22:27:08.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f07465e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400c8200 con 0x7f07401461a0 2020-11-16T22:27:08.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0745de2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f07400615f0 0x7f07400cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43152/0 (socket says 172.21.15.181:43152) 2020-11-16T22:27:08.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0745de2700 1 -- 172.21.15.181:0/1069046133 learned_addr learned my addr 172.21.15.181:0/1069046133 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:08.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0744de0700 1 --2- 172.21.15.181:0/1069046133 >> v2:172.21.15.175:3300/0 conn(0x7f07401461a0 0x7f0740061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230364399 0 0) 0x7f07400c86c0 con 0x7f07400615f0 2020-11-16T22:27:08.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0724003240 con 0x7f07400615f0 2020-11-16T22:27:08.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0744de0700 1 -- 172.21.15.181:0/1069046133 >> v1:172.21.15.181:6789/0 conn(0x7f07400615f0 legacy=0x7f07400cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0744de0700 1 -- 172.21.15.181:0/1069046133 >> v1:172.21.15.175:6789/0 conn(0x7f0740145dc0 legacy=0x7f07400c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321311407 0 0) 0x7f07400c8420 con 0x7f0740145dc0 2020-11-16T22:27:08.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0744de0700 1 -- 172.21.15.181:0/1069046133 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400c8960 con 0x7f07401461a0 2020-11-16T22:27:08.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0744de0700 1 --2- 172.21.15.181:0/1069046133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07401461a0 0x7f0740061100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f073000b9a0 tx=0x7f0730006df0).ready entity=mon.0 client_cookie=b51ed28ea0aa8faa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:08.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300097b0 con 0x7f07401461a0 2020-11-16T22:27:08.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730009910 con 0x7f07401461a0 2020-11-16T22:27:08.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.825+0000 7f0737fff700 1 -- 172.21.15.181:0/1069046133 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730009c90 con 0x7f07401461a0 2020-11-16T22:27:08.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- 172.21.15.181:0/1069046133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07401461a0 msgr2=0x7f0740061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 --2- 172.21.15.181:0/1069046133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07401461a0 0x7f0740061100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f073000b9a0 tx=0x7f0730006df0).stop 2020-11-16T22:27:08.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- 172.21.15.181:0/1069046133 shutdown_connections 2020-11-16T22:27:08.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 --2- 172.21.15.181:0/1069046133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07401461a0 0x7f0740061100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- 172.21.15.181:0/1069046133 >> 172.21.15.181:0/1069046133 conn(0x7f07400cf630 msgr2=0x7f07400cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:08.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- 172.21.15.181:0/1069046133 shutdown_connections 2020-11-16T22:27:08.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- 172.21.15.181:0/1069046133 wait complete. 2020-11-16T22:27:08.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 Processor -- start 2020-11-16T22:27:08.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.829+0000 7f07465e3700 1 -- start start 2020-11-16T22:27:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0740145dc0 0x7f07400d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 0x7f07401411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400d15e0 con 0x7f0740145dc0 2020-11-16T22:27:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074016f580 con 0x7f07400615f0 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074016f820 con 0x7f07401461a0 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f0745de2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 0x7f07401411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57082/0 (socket says 172.21.15.181:57082) 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 -- 172.21.15.181:0/500766153 learned_addr learned my addr 172.21.15.181:0/500766153 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f0744de0700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0740145dc0 0x7f07400d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 -- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 msgr2=0x7f07401411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 0x7f07401411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 -- 172.21.15.181:0/500766153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0740145dc0 msgr2=0x7f07400d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0740145dc0 0x7f07400d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 -- 172.21.15.181:0/500766153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074016fac0 con 0x7f07400615f0 2020-11-16T22:27:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f0745de2700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 0x7f07401411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07455e1700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f073c007fd0 tx=0x7f073c007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00e490 con 0x7f07400615f0 2020-11-16T22:27:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c00e5f0 con 0x7f07400615f0 2020-11-16T22:27:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074016fd60 con 0x7f07400615f0 2020-11-16T22:27:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00e9b0 con 0x7f07400615f0 2020-11-16T22:27:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.833+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0740164050 con 0x7f07400615f0 2020-11-16T22:27:09.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0740164240 con 0x7f07400615f0 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f073c019030 con 0x7f07400615f0 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f07357fa700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 0x7f0718057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f073c06d3d0 con 0x7f07400615f0 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f0744de0700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 0x7f0718057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.837+0000 7f0744de0700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 0x7f0718057520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0730006e30 tx=0x7f0730015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:08.845+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f073c043520 con 0x7f07400615f0 2020-11-16T22:27:09.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.001+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0740044f60 con 0x7f07400615f0 2020-11-16T22:27:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.001+0000 7f07357fa700 1 -- 172.21.15.181:0/500766153 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f073c046b40 con 0x7f07400615f0 2020-11-16T22:27:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 msgr2=0x7f0718057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 0x7f0718057520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0730006e30 tx=0x7f0730015000).stop 2020-11-16T22:27:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 msgr2=0x7f074016efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f073c007fd0 tx=0x7f073c007ac0).stop 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 shutdown_connections 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0718055060 0x7f0718057520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07401461a0 0x7f07401411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400615f0 0x7f074016efa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 --2- 172.21.15.181:0/500766153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0740145dc0 0x7f07400d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 >> 172.21.15.181:0/500766153 conn(0x7f07400cf630 msgr2=0x7f0740143480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 shutdown_connections 2020-11-16T22:27:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.005+0000 7f07465e3700 1 -- 172.21.15.181:0/500766153 wait complete. 2020-11-16T22:27:09.236 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:09.236 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:27:09.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 Processor -- start 2020-11-16T22:27:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 -- start start 2020-11-16T22:27:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5900d0a00 0x7ff590140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900cf380 con 0x7ff590140550 2020-11-16T22:27:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900cf5d0 con 0x7ff5900cef10 2020-11-16T22:27:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595c01700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59015ebb0 con 0x7ff5900d0a00 2020-11-16T22:27:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595400700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff5900cef10 0x7ff590063ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43164/0 (socket says 172.21.15.181:43164) 2020-11-16T22:27:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595400700 1 -- 172.21.15.181:0/2053769996 learned_addr learned my addr 172.21.15.181:0/2053769996 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 --2- 172.21.15.181:0/2053769996 >> v2:172.21.15.175:3300/0 conn(0x7ff5900d0a00 0x7ff590140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512515460 0 0) 0x7ff5900cf5d0 con 0x7ff5900cef10 2020-11-16T22:27:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff57c003240 con 0x7ff5900cef10 2020-11-16T22:27:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1424991499 0 0) 0x7ff57c003240 con 0x7ff5900cef10 2020-11-16T22:27:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 >> v2:172.21.15.175:3300/0 conn(0x7ff5900d0a00 msgr2=0x7ff590140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/2053769996 >> v2:172.21.15.175:3300/0 conn(0x7ff5900d0a00 0x7ff590140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 >> v1:172.21.15.175:6789/0 conn(0x7ff590140550 legacy=0x7ff5900d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900cf870 con 0x7ff5900cef10 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7ff588001860 con 0x7ff5900cef10 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 >> v1:172.21.15.181:6789/0 conn(0x7ff5900cef10 legacy=0x7ff590063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c003730 0x7ff57c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 0x7ff57c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57c004e50 0x7ff57c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59015ebb0 con 0x7ff57c003730 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0042c0 con 0x7ff57c004e50 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595400700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57c004e50 0x7ff57c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c00bc20 con 0x7ff57c0044e0 2020-11-16T22:27:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 0x7ff57c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff594bff700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c003730 0x7ff57c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 -- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57c004e50 msgr2=0x7ff57c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57c004e50 0x7ff57c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 -- 172.21.15.181:0/2053769996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c003730 msgr2=0x7ff57c003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff57c003730 0x7ff57c003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 -- 172.21.15.181:0/2053769996 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c003240 con 0x7ff57c0044e0 2020-11-16T22:27:09.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff595400700 1 -- 172.21.15.181:0/2053769996 reap_dead start 2020-11-16T22:27:09.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58ffff700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 0x7ff57c004960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff58400bc20 tx=0x7ff584006df0).ready entity=mon.2 client_cookie=d25ce1eb567838d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400b300 con 0x7ff57c0044e0 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400ea30 con 0x7ff57c0044e0 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.381+0000 7ff58f7fe700 1 -- 172.21.15.181:0/2053769996 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400edb0 con 0x7ff57c0044e0 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 msgr2=0x7ff57c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 0x7ff57c004960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff58400bc20 tx=0x7ff584006df0).stop 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/2053769996 shutdown_connections 2020-11-16T22:27:09.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 --2- 172.21.15.181:0/2053769996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff57c0044e0 0x7ff57c004960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/2053769996 >> 172.21.15.181:0/2053769996 conn(0x7ff5900d4570 msgr2=0x7ff5900d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:09.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/2053769996 shutdown_connections 2020-11-16T22:27:09.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/2053769996 wait complete. 2020-11-16T22:27:09.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 Processor -- start 2020-11-16T22:27:09.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- start start 2020-11-16T22:27:09.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59015d8b0 0x7ff5900c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 0x7ff5900c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59016d7f0 con 0x7ff59015dc90 2020-11-16T22:27:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59016da90 con 0x7ff59015d8b0 2020-11-16T22:27:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff590066570 con 0x7ff5900671f0 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36716/0 (socket says 172.21.15.181:36716) 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 -- 172.21.15.181:0/66274481 learned_addr learned my addr 172.21.15.181:0/66274481 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff594bff700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59015d8b0 0x7ff5900c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff58ffff700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 0x7ff5900c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 -- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59015d8b0 msgr2=0x7ff5900c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59015d8b0 0x7ff5900c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 -- 172.21.15.181:0/66274481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 msgr2=0x7ff5900c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 0x7ff5900c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 -- 172.21.15.181:0/66274481 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590066810 con 0x7ff5900671f0 2020-11-16T22:27:09.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff58ffff700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 0x7ff5900c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:09.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595400700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff5900818a0 tx=0x7ff590065f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:09.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800fc90 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800fdf0 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588016650 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff590066ab0 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.385+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59016d120 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff590044f60 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff58801f3a0 con 0x7ff5900671f0 2020-11-16T22:27:09.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff58cff9700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 0x7ff56c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:09.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff58806de90 con 0x7ff5900671f0 2020-11-16T22:27:09.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff594bff700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 0x7ff56c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:09.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.389+0000 7ff594bff700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 0x7ff56c057520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff580007ad0 tx=0x7ff580007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:09.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.393+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff588044140 con 0x7ff5900671f0 2020-11-16T22:27:09.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.593+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff59005d4e0 con 0x7ff5900671f0 2020-11-16T22:27:09.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.593+0000 7ff58cff9700 1 -- 172.21.15.181:0/66274481 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4737 (secure 0 0 0) 0x7ff58801b020 con 0x7ff5900671f0 2020-11-16T22:27:09.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 msgr2=0x7ff56c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 0x7ff56c057520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff580007ad0 tx=0x7ff580007b00).stop 2020-11-16T22:27:09.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 msgr2=0x7ff59016cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff5900818a0 tx=0x7ff590065f00).stop 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 shutdown_connections 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff56c055060 0x7ff56c057520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5900671f0 0x7ff59016cc30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59015d8b0 0x7ff5900c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 --2- 172.21.15.181:0/66274481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59015dc90 0x7ff5900c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 >> 172.21.15.181:0/66274481 conn(0x7ff5900d4570 msgr2=0x7ff59013e600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:09.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 shutdown_connections 2020-11-16T22:27:09.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:09.597+0000 7ff595c01700 1 -- 172.21.15.181:0/66274481 wait complete. 2020-11-16T22:27:09.992 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 22 2020-11-16T22:27:10.194 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:27:02.107409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2,3,4,5,6,7,8],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299,"mds_3":4296,"mds_4":4276,"mds_5":4266,"mds_6":4256,"mds_7":4253,"mds_8":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":8,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4253":{"gid":4253,"name":"g","rank":7,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4256":{"gid":4256,"name":"a","rank":6,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4266":{"gid":4266,"name":"e","rank":5,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4276":{"gid":4276,"name":"b","rank":4,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4296":{"gid":4296,"name":"h","rank":3,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:27:11.303 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:26:51.998542+0000', 'modified': '2020-11-16T22:27:02.107409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 9, 'in': [0, 1, 2, 3, 4, 5, 6, 7, 8], 'up': {'mds_0': 4305, 'mds_1': 4315, 'mds_2': 4299, 'mds_3': 4296, 'mds_4': 4276, 'mds_5': 4266, 'mds_6': 4256, 'mds_7': 4253, 'mds_8': 4244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4244': {'gid': 4244, 'name': 'd', 'rank': 8, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6837/3313292089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3313292089}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3313292089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4253': {'gid': 4253, 'name': 'g', 'rank': 7, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6835/228556183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 228556183}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 228556183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4256': {'gid': 4256, 'name': 'a', 'rank': 6, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6840/2424566790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2424566790}, {'type': 'v1', 'addr': '172.21.15.181:6840', 'nonce': 2424566790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4266': {'gid': 4266, 'name': 'e', 'rank': 5, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6833/3836362112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 3836362112}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 3836362112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 4, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6835/3777546427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3777546427}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3777546427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4296': {'gid': 4296, 'name': 'h', 'rank': 3, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/390679745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6837', 'nonce': 390679745}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 390679745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4299': {'gid': 4299, 'name': 'f', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6838/2983366267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2983366267}, {'type': 'v1', 'addr': '172.21.15.175:6838', 'nonce': 2983366267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4305': {'gid': 4305, 'name': 'i', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.175:6841/1137266867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 1137266867}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 1137266867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4315': {'gid': 4315, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3996054507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3996054507}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3996054507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:27:11.303 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 9/9 2020-11-16T22:27:11.304 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:27:11.443 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":8,"id":4244,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.341612048,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.202058514000001} 2020-11-16T22:27:11.443 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4244, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 10.341612048, "state": "up:active", "uptime": 23.202058514, "want_state": "up:active", "whoami": 8 } 2020-11-16T22:27:11.444 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.g.asok --format=json status 2020-11-16T22:27:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":7,"id":4253,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.493243977000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.337926473} 2020-11-16T22:27:11.585 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4253, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 11.493243977, "state": "up:active", "uptime": 23.337926473, "want_state": "up:active", "whoami": 7 } 2020-11-16T22:27:11.585 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:27:11.721 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":6,"id":4256,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.643700319000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.473554813} 2020-11-16T22:27:11.722 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4256, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 12.643700319, "state": "up:active", "uptime": 23.473554813, "want_state": "up:active", "whoami": 6 } 2020-11-16T22:27:11.722 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:27:11.858 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":5,"id":4266,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.794771322000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.621273601999999} 2020-11-16T22:27:11.858 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4266, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 13.794771322, "state": "up:active", "uptime": 23.621273602, "want_state": "up:active", "whoami": 5 } 2020-11-16T22:27:11.859 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:27:11.995 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":4,"id":4276,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.943556051,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.765087554000001} 2020-11-16T22:27:11.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4276, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 14.943556051, "state": "up:active", "uptime": 23.765087554, "want_state": "up:active", "whoami": 4 } 2020-11-16T22:27:11.996 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok --format=json status 2020-11-16T22:27:12.139 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":3,"id":4296,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.098448450999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.883129414999999} 2020-11-16T22:27:12.139 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4296, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 16.098448451, "state": "up:active", "uptime": 23.883129415, "want_state": "up:active", "whoami": 3 } 2020-11-16T22:27:12.139 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2020-11-16T22:27:12.270 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":4299,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.240991622999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.015559054000001} 2020-11-16T22:27:12.270 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4299, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 17.240991623, "state": "up:active", "uptime": 24.015559054, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:27:12.271 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:27:12.403 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":4305,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.409534953000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.138314510000001} 2020-11-16T22:27:12.403 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4305, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 20.409534953, "state": "up:active", "uptime": 24.13831451, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:27:12.404 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:27:12.541 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":4315,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.525395957000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":24.294238791000001} 2020-11-16T22:27:12.542 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4315, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 18.525395957, "state": "up:active", "uptime": 24.294238791, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:27:12.542 INFO:teuthology.run_tasks:Running task kclient... 2020-11-16T22:27:12.614 INFO:tasks.kclient:Mounting kernel clients... 2020-11-16T22:27:12.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.615 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:27:12.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.635 INFO:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2020-11-16T22:27:12.671 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2020-11-16T22:27:12.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:27:12.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.675 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:27:12.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.688 INFO:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2020-11-16T22:27:12.726 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2020-11-16T22:27:12.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:27:12.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:27:12.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:27:12.731 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:27:12.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:27:12.731 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:27:12.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-16T22:27:12.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:27:12.732 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:27:12.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.732 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:27:12.748 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:27:12.748 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:27:12.748 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:27:12.748 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:12.749 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:27:12.750 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:12.751 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-16T22:27:12.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.752 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:12.752 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:12.752 INFO:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2020-11-16T22:27:12.753 INFO:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2020-11-16T22:27:12.753 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2020-11-16T22:27:12.753 INFO:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-16T22:27:12.753 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:12.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.964 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:27:12.995 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:27:13.035 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:27:13.036 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:27:13.036 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:27:13.036 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:27:13.036 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:27:13.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.037 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:13.038 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:13.038 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:27:13.038 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:27:13.038 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:27:13.039 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:13.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.227 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:27:13.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.240 INFO:teuthology.orchestra.run.smithi106:> ip netns list-id 2020-11-16T22:27:13.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.254 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:13.254 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:13.254 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-16T22:27:13.254 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2020-11-16T22:27:13.255 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:13.337 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-11-16T22:27:13.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.338 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:13.338 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:13.338 INFO:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2020-11-16T22:27:13.339 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-16T22:27:13.339 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2020-11-16T22:27:13.339 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2020-11-16T22:27:13.339 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2020-11-16T22:27:13.340 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:13.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:13.506 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:13.506 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:13.507 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:27:13.507 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2020-11-16T22:27:13.507 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:13.583 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:13.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.725+0000 7fee4bfff700 1 Processor -- start 2020-11-16T22:27:13.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.725+0000 7fee4bfff700 1 -- start start 2020-11-16T22:27:13.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:13.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4bfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c081f20 con 0x7fee4c0d2a30 2020-11-16T22:27:13.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:13.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4bfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c060470 con 0x7fee4c160450 2020-11-16T22:27:13.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c060710 con 0x7fee4c145770 2020-11-16T22:27:13.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36720/0 (socket says 172.21.15.181:36720) 2020-11-16T22:27:13.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 -- 172.21.15.181:0/1830142480 learned_addr learned my addr 172.21.15.181:0/1830142480 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 -- 172.21.15.181:0/1830142480 >> v1:172.21.15.181:6789/0 conn(0x7fee4c160450 legacy=0x7fee4c0d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 -- 172.21.15.181:0/1830142480 >> v1:172.21.15.175:6789/0 conn(0x7fee4c0d2a30 legacy=0x7fee4c1612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 -- 172.21.15.181:0/1830142480 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0609b0 con 0x7fee4c145770 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4affd700 1 --2- 172.21.15.181:0/1830142480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fee3c00ba20 tx=0x7fee3c006eb0).ready entity=mon.2 client_cookie=fe7bb677de25fddc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4a7fc700 1 -- 172.21.15.181:0/1830142480 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0098d0 con 0x7fee4c145770 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4a7fc700 1 -- 172.21.15.181:0/1830142480 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c009a30 con 0x7fee4c145770 2020-11-16T22:27:13.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.729+0000 7fee4a7fc700 1 -- 172.21.15.181:0/1830142480 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c009db0 con 0x7fee4c145770 2020-11-16T22:27:13.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- 172.21.15.181:0/1830142480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 msgr2=0x7fee4c145b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 --2- 172.21.15.181:0/1830142480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fee3c00ba20 tx=0x7fee3c006eb0).stop 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- 172.21.15.181:0/1830142480 shutdown_connections 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 --2- 172.21.15.181:0/1830142480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c145770 0x7fee4c145b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- 172.21.15.181:0/1830142480 >> 172.21.15.181:0/1830142480 conn(0x7fee4c0d6760 msgr2=0x7fee4c0d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- 172.21.15.181:0/1830142480 shutdown_connections 2020-11-16T22:27:13.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- 172.21.15.181:0/1830142480 wait complete. 2020-11-16T22:27:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 Processor -- start 2020-11-16T22:27:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.733+0000 7fee4bfff700 1 -- start start 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 0x7fee4c0d3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 0x7fee4c0d3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15e000 con 0x7fee4c160450 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15e2a0 con 0x7fee4c145770 2020-11-16T22:27:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15e540 con 0x7fee4c0d2a30 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee427fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 0x7fee4c0d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36726/0 (socket says 172.21.15.181:36726) 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 -- 172.21.15.181:0/3666106638 learned_addr learned my addr 172.21.15.181:0/3666106638 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4b7fe700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 0x7fee4c0d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 -- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 msgr2=0x7fee4c0d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 0x7fee4c0d3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 -- 172.21.15.181:0/3666106638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 msgr2=0x7fee4c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 0x7fee4c0d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 -- 172.21.15.181:0/3666106638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c15e7e0 con 0x7fee4c0d2a30 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee427fc700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 0x7fee4c0d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4b7fe700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 0x7fee4c0d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4affd700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fee3c006ef0 tx=0x7fee3c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011dd0 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c01d450 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c15ea80 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01d810 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1651a0 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fee3c00f070 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c044f60 con 0x7fee4c0d2a30 2020-11-16T22:27:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 0x7fee28057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fee3c06fcd0 con 0x7fee4c0d2a30 2020-11-16T22:27:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee427fc700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 0x7fee28057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.737+0000 7fee427fc700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 0x7fee28057520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fee4c165900 tx=0x7fee34009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.741+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fee3c049e60 con 0x7fee4c0d2a30 2020-11-16T22:27:13.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.897+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee4c165420 con 0x7fee4c0d2a30 2020-11-16T22:27:13.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee437fe700 1 -- 172.21.15.181:0/3666106638 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fee3c045f30 con 0x7fee4c0d2a30 2020-11-16T22:27:13.885 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:13.886 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:14.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 msgr2=0x7fee28057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 0x7fee28057520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fee4c165900 tx=0x7fee34009040).stop 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 msgr2=0x7fee4c15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fee3c006ef0 tx=0x7fee3c015000).stop 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 shutdown_connections 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fee28055060 0x7fee28057520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee4c160450 0x7fee4c0d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee4c0d2a30 0x7fee4c15d9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 --2- 172.21.15.181:0/3666106638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee4c145770 0x7fee4c0d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 >> 172.21.15.181:0/3666106638 conn(0x7fee4c0d6760 msgr2=0x7fee4c0d5770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 shutdown_connections 2020-11-16T22:27:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:13.901+0000 7fee4bfff700 1 -- 172.21.15.181:0/3666106638 wait complete. 2020-11-16T22:27:14.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:15.318 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:15.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.473+0000 7f645b59e700 1 Processor -- start 2020-11-16T22:27:15.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.473+0000 7f645b59e700 1 -- start start 2020-11-16T22:27:15.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 0x7f645c061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:15.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645b59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c15d180 con 0x7f645c13e870 2020-11-16T22:27:15.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c15d810 con 0x7f645c0615f0 2020-11-16T22:27:15.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15dab0 con 0x7f645c13ec50 2020-11-16T22:27:15.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f645c0615f0 0x7f645c15e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43200/0 (socket says 172.21.15.181:43200) 2020-11-16T22:27:15.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f6459d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 0x7f645c061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:15.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645ad9d700 1 -- 172.21.15.181:0/3462395012 learned_addr learned my addr 172.21.15.181:0/3462395012 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:15.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127067229 0 0) 0x7f645c15d810 con 0x7f645c0615f0 2020-11-16T22:27:15.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c003240 con 0x7f645c0615f0 2020-11-16T22:27:15.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f6459d9b700 1 -- 172.21.15.181:0/3462395012 >> v1:172.21.15.181:6789/0 conn(0x7f645c0615f0 legacy=0x7f645c15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:15.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218917329 0 0) 0x7f645c15d180 con 0x7f645c13e870 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f6459d9b700 1 -- 172.21.15.181:0/3462395012 >> v1:172.21.15.175:6789/0 conn(0x7f645c13e870 legacy=0x7f645c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f6459d9b700 1 -- 172.21.15.181:0/3462395012 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c15dd50 con 0x7f645c13ec50 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f6459d9b700 1 --2- 172.21.15.181:0/3462395012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 0x7f645c061100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f644400b9d0 tx=0x7f6444006df0).ready entity=mon.2 client_cookie=27b83e80433e3d0a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444014070 con 0x7f645c13ec50 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64440097b0 con 0x7f645c13ec50 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.477+0000 7f645959a700 1 -- 172.21.15.181:0/3462395012 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444009b30 con 0x7f645c13ec50 2020-11-16T22:27:15.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- 172.21.15.181:0/3462395012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 msgr2=0x7f645c061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:15.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 --2- 172.21.15.181:0/3462395012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 0x7f645c061100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f644400b9d0 tx=0x7f6444006df0).stop 2020-11-16T22:27:15.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- 172.21.15.181:0/3462395012 shutdown_connections 2020-11-16T22:27:15.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 --2- 172.21.15.181:0/3462395012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c13ec50 0x7f645c061100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:15.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- 172.21.15.181:0/3462395012 >> 172.21.15.181:0/3462395012 conn(0x7f645c0cf630 msgr2=0x7f645c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:15.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- 172.21.15.181:0/3462395012 shutdown_connections 2020-11-16T22:27:15.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- 172.21.15.181:0/3462395012 wait complete. 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 Processor -- start 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- start start 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 0x7f645c138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c13ec50 0x7f645c139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0b7100 con 0x7f645c13ec50 2020-11-16T22:27:15.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0b73a0 con 0x7f645c13e870 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0b7640 con 0x7f645c0615f0 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f6459d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 0x7f645c138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36740/0 (socket says 172.21.15.181:36740) 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 -- 172.21.15.181:0/313820713 learned_addr learned my addr 172.21.15.181:0/313820713 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:15.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f6459d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 0x7f645c138c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57132/0 (socket says 172.21.15.181:57132) 2020-11-16T22:27:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645ad9d700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c13ec50 0x7f645c139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 -- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 msgr2=0x7f645c138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 0x7f645c138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 -- 172.21.15.181:0/313820713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c13ec50 msgr2=0x7f645c139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c13ec50 0x7f645c139140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.481+0000 7f645a59c700 1 -- 172.21.15.181:0/313820713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0b78e0 con 0x7f645c0615f0 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f645a59c700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f644c001520 tx=0x7f644c007df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00a640 con 0x7f645c0615f0 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c00a7a0 con 0x7f645c0615f0 2020-11-16T22:27:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0b7b80 con 0x7f645c0615f0 2020-11-16T22:27:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00ab60 con 0x7f645c0615f0 2020-11-16T22:27:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c0b6c90 con 0x7f645c0615f0 2020-11-16T22:27:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c044f60 con 0x7f645c0615f0 2020-11-16T22:27:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f644c010070 con 0x7f645c0615f0 2020-11-16T22:27:16.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 0x7f6430057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f644c06dae0 con 0x7f645c0615f0 2020-11-16T22:27:16.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.485+0000 7f6459d9b700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 0x7f6430057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.489+0000 7f6459d9b700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 0x7f6430057550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f644400b150 tx=0x7f6444016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:16.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.493+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f644c043d40 con 0x7f645c0615f0 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.645+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f645c0b6ee0 con 0x7f645c0615f0 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.645+0000 7f64527fc700 1 -- 172.21.15.181:0/313820713 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f644c047360 con 0x7f645c0615f0 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 msgr2=0x7f6430057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 0x7f6430057550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f644400b150 tx=0x7f6444016000).stop 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 msgr2=0x7f645c0b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f644c001520 tx=0x7f644c007df0).stop 2020-11-16T22:27:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 shutdown_connections 2020-11-16T22:27:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6430055090 0x7f6430057550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c13ec50 0x7f645c139140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0615f0 0x7f645c0b67a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 --2- 172.21.15.181:0/313820713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c13e870 0x7f645c138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 >> 172.21.15.181:0/313820713 conn(0x7f645c0cf630 msgr2=0x7f645c15efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:16.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 shutdown_connections 2020-11-16T22:27:16.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:15.649+0000 7f645b59e700 1 -- 172.21.15.181:0/313820713 wait complete. 2020-11-16T22:27:16.487 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:16.488 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:16.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.637+0000 7f017d958700 1 Processor -- start 2020-11-16T22:27:16.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.637+0000 7f017d958700 1 -- start start 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0178161480 0x7f0178164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0178164e10 0x7f0178062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01780c8db0 con 0x7f0178164e10 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01780c8330 con 0x7f0178161480 2020-11-16T22:27:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780c85d0 con 0x7f0178161860 2020-11-16T22:27:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017c956700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0178161480 0x7f0178164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36750/0 (socket says 172.21.15.181:36750) 2020-11-16T22:27:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 -- 172.21.15.181:0/1551176470 learned_addr learned my addr 172.21.15.181:0/1551176470 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f0177fff700 1 --2- 172.21.15.181:0/1551176470 >> v2:172.21.15.175:3300/0 conn(0x7f0178164e10 0x7f0178062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 -- 172.21.15.181:0/1551176470 >> v2:172.21.15.181:3300/0 conn(0x7f0178161480 msgr2=0x7f0178164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 --2- 172.21.15.181:0/1551176470 >> v2:172.21.15.181:3300/0 conn(0x7f0178161480 0x7f0178164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 -- 172.21.15.181:0/1551176470 >> v2:172.21.15.175:3300/0 conn(0x7f0178164e10 msgr2=0x7f0178062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 --2- 172.21.15.181:0/1551176470 >> v2:172.21.15.175:3300/0 conn(0x7f0178164e10 0x7f0178062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 -- 172.21.15.181:0/1551176470 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780c8870 con 0x7f0178161860 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d157700 1 --2- 172.21.15.181:0/1551176470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f0178066500 tx=0x7f0160006eb0).ready entity=mon.2 client_cookie=f3d256ab1d2a191 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f01777fe700 1 -- 172.21.15.181:0/1551176470 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160009810 con 0x7f0178161860 2020-11-16T22:27:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f01777fe700 1 -- 172.21.15.181:0/1551176470 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0160009970 con 0x7f0178161860 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f01777fe700 1 -- 172.21.15.181:0/1551176470 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160009cf0 con 0x7f0178161860 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- 172.21.15.181:0/1551176470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 msgr2=0x7f0178062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- 172.21.15.181:0/1551176470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f0178066500 tx=0x7f0160006eb0).stop 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- 172.21.15.181:0/1551176470 shutdown_connections 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- 172.21.15.181:0/1551176470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161860 0x7f0178062740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- 172.21.15.181:0/1551176470 >> v2:172.21.15.181:3300/0 conn(0x7f0178161480 0x7f0178164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 --2- 172.21.15.181:0/1551176470 >> v2:172.21.15.175:3300/0 conn(0x7f0178164e10 0x7f0178062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- 172.21.15.181:0/1551176470 >> 172.21.15.181:0/1551176470 conn(0x7f01780d2dc0 msgr2=0x7f01780ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- 172.21.15.181:0/1551176470 shutdown_connections 2020-11-16T22:27:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.641+0000 7f017d958700 1 -- 172.21.15.181:0/1551176470 wait complete. 2020-11-16T22:27:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 Processor -- start 2020-11-16T22:27:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 -- start start 2020-11-16T22:27:16.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0178161860 0x7f0178177580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780d0120 0x7f0178140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781735b0 con 0x7f0178161860 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017c956700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0178177a70 con 0x7f01780d0120 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d958700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780d1c70 con 0x7f0178161480 2020-11-16T22:27:16.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017c956700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36752/0 (socket says 172.21.15.181:36752) 2020-11-16T22:27:16.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017c956700 1 -- 172.21.15.181:0/2424269235 learned_addr learned my addr 172.21.15.181:0/2424269235 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:16.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f017d157700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780d0120 0x7f0178140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.645+0000 7f0177fff700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0178161860 0x7f0178177580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 -- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780d0120 msgr2=0x7f0178140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780d0120 0x7f0178140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 -- 172.21.15.181:0/2424269235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0178161860 msgr2=0x7f0178177580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0178161860 0x7f0178177580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 -- 172.21.15.181:0/2424269235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780d1f10 con 0x7f0178161480 2020-11-16T22:27:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017c956700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f017000ebd0 tx=0x7f0170006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000b810 con 0x7f0178161480 2020-11-16T22:27:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017000b970 con 0x7f0178161480 2020-11-16T22:27:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01781737e0 con 0x7f0178161480 2020-11-16T22:27:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000bcf0 con 0x7f0178161480 2020-11-16T22:27:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0178173d20 con 0x7f0178161480 2020-11-16T22:27:17.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0170012600 con 0x7f0178161480 2020-11-16T22:27:17.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0178044f60 con 0x7f0178161480 2020-11-16T22:27:17.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 0x7f0148057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f017006e000 con 0x7f0178161480 2020-11-16T22:27:17.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.649+0000 7f0177fff700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 0x7f0148057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.653+0000 7f0177fff700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 0x7f0148057520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0178143ac0 tx=0x7f0168008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.657+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0170044200 con 0x7f0178161480 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.809+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01780671e0 con 0x7f0178161480 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f0174ff9700 1 -- 172.21.15.181:0/2424269235 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0170047820 con 0x7f0178161480 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 msgr2=0x7f0148057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 0x7f0148057520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0178143ac0 tx=0x7f0168008040).stop 2020-11-16T22:27:17.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 msgr2=0x7f01781741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f017000ebd0 tx=0x7f0170006df0).stop 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 shutdown_connections 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0148055060 0x7f0148057520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780d0120 0x7f0178140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0178161480 0x7f01781741c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 --2- 172.21.15.181:0/2424269235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0178161860 0x7f0178177580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 >> 172.21.15.181:0/2424269235 conn(0x7f01780d2dc0 msgr2=0x7f01781608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 shutdown_connections 2020-11-16T22:27:17.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:16.813+0000 7f017d958700 1 -- 172.21.15.181:0/2424269235 wait complete. 2020-11-16T22:27:17.045 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:17.045 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:17.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.197+0000 7f6a688dc700 1 Processor -- start 2020-11-16T22:27:17.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.197+0000 7f6a688dc700 1 -- start start 2020-11-16T22:27:17.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6a640615f0 0x7f6a640622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a64062790 0x7f6a64080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a64160e40 con 0x7f6a6415f5b0 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a641609a0 con 0x7f6a640615f0 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a64160ba0 con 0x7f6a64062790 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a64062790 0x7f6a64080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a64062790 0x7f6a64080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57150/0 (socket says 172.21.15.181:57150) 2020-11-16T22:27:17.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 -- 172.21.15.181:0/2280139554 learned_addr learned my addr 172.21.15.181:0/2280139554 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a6259c700 1 --1- 172.21.15.181:0/2280139554 >> v1:172.21.15.181:6789/0 conn(0x7f6a6415f5b0 0x7f6a640d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43224/0 (socket says 172.21.15.181:43224) 2020-11-16T22:27:17.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a61d9b700 1 --2- 172.21.15.181:0/2280139554 >> v2:172.21.15.175:3300/0 conn(0x7f6a640615f0 0x7f6a640622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 -- 172.21.15.181:0/2280139554 >> v2:172.21.15.175:3300/0 conn(0x7f6a640615f0 msgr2=0x7f6a640622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:17.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 --2- 172.21.15.181:0/2280139554 >> v2:172.21.15.175:3300/0 conn(0x7f6a640615f0 0x7f6a640622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 -- 172.21.15.181:0/2280139554 >> v1:172.21.15.181:6789/0 conn(0x7f6a6415f5b0 legacy=0x7f6a640d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 -- 172.21.15.181:0/2280139554 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64060970 con 0x7f6a64062790 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a62d9d700 1 --2- 172.21.15.181:0/2280139554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a64062790 0x7f6a64080fd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f6a640d2ea0 tx=0x7f6a5c009290).ready entity=mon.1 client_cookie=3f7885e5055cae4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a6159a700 1 -- 172.21.15.181:0/2280139554 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00bbc0 con 0x7f6a64062790 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a6159a700 1 -- 172.21.15.181:0/2280139554 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00bd20 con 0x7f6a64062790 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a6159a700 1 -- 172.21.15.181:0/2280139554 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011610 con 0x7f6a64062790 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- 172.21.15.181:0/2280139554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a64062790 msgr2=0x7f6a64080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 --2- 172.21.15.181:0/2280139554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a64062790 0x7f6a64080fd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f6a640d2ea0 tx=0x7f6a5c009290).stop 2020-11-16T22:27:17.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- 172.21.15.181:0/2280139554 shutdown_connections 2020-11-16T22:27:17.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 --2- 172.21.15.181:0/2280139554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a64062790 0x7f6a64080fd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 --2- 172.21.15.181:0/2280139554 >> v2:172.21.15.175:3300/0 conn(0x7f6a640615f0 0x7f6a640622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- 172.21.15.181:0/2280139554 >> 172.21.15.181:0/2280139554 conn(0x7f6a640d69a0 msgr2=0x7f6a640d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- 172.21.15.181:0/2280139554 shutdown_connections 2020-11-16T22:27:17.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.201+0000 7f6a688dc700 1 -- 172.21.15.181:0/2280139554 wait complete. 2020-11-16T22:27:17.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 Processor -- start 2020-11-16T22:27:17.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 -- start start 2020-11-16T22:27:17.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 0x7f6a64081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a64062790 0x7f6a64081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64170920 con 0x7f6a64062790 2020-11-16T22:27:17.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640bbb90 con 0x7f6a6415f5b0 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640bbe30 con 0x7f6a640615f0 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57160/0 (socket says 172.21.15.181:57160) 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a6259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 0x7f6a64081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 -- 172.21.15.181:0/3803912632 learned_addr learned my addr 172.21.15.181:0/3803912632 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a61d9b700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a64062790 0x7f6a64081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 -- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 msgr2=0x7f6a64081170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 0x7f6a64081170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 -- 172.21.15.181:0/3803912632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a64062790 msgr2=0x7f6a64081660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a64062790 0x7f6a64081660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 -- 172.21.15.181:0/3803912632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640bc0d0 con 0x7f6a6415f5b0 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a6259c700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 0x7f6a64081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a62d9d700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6a5c0092d0 tx=0x7f6a5c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00bbc0 con 0x7f6a6415f5b0 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00be90 con 0x7f6a6415f5b0 2020-11-16T22:27:17.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640bc370 con 0x7f6a6415f5b0 2020-11-16T22:27:17.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.205+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c010dc0 con 0x7f6a6415f5b0 2020-11-16T22:27:17.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a640bb520 con 0x7f6a6415f5b0 2020-11-16T22:27:17.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6a5c023030 con 0x7f6a6415f5b0 2020-11-16T22:27:17.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64069eb0 con 0x7f6a6415f5b0 2020-11-16T22:27:17.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a5a7fc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a5c06f740 con 0x7f6a6415f5b0 2020-11-16T22:27:17.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a6259c700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.209+0000 7f6a6259c700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6a640d4200 tx=0x7f6a54009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.213+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6a5c045940 con 0x7f6a6415f5b0 2020-11-16T22:27:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.365+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a640600f0 con 0x7f6a6415f5b0 2020-11-16T22:27:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.365+0000 7f6a5a7fc700 1 -- 172.21.15.181:0/3803912632 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6a5c021020 con 0x7f6a6415f5b0 2020-11-16T22:27:17.353 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:17.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 msgr2=0x7f6a40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6a640d4200 tx=0x7f6a54009040).stop 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 msgr2=0x7f6a640bb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6a5c0092d0 tx=0x7f6a5c016000).stop 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 shutdown_connections 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a40055060 0x7f6a40057520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a6415f5b0 0x7f6a640bb030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a640615f0 0x7f6a64081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 --2- 172.21.15.181:0/3803912632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a64062790 0x7f6a64081660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 >> 172.21.15.181:0/3803912632 conn(0x7f6a640d69a0 msgr2=0x7f6a64061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 shutdown_connections 2020-11-16T22:27:17.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.369+0000 7f6a688dc700 1 -- 172.21.15.181:0/3803912632 wait complete. 2020-11-16T22:27:17.368 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:17.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 Processor -- start 2020-11-16T22:27:17.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 -- start start 2020-11-16T22:27:17.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85e81426d0 0x7f85e8142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 0x7f85e8061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e8142fa0 con 0x7f85e8160740 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e8064af0 con 0x7f85e81426d0 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ee560700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8064d90 con 0x7f85e8160bf0 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ecafd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 0x7f85e8061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85e81426d0 0x7f85e8142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ecafd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 0x7f85e8061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36778/0 (socket says 172.21.15.181:36778) 2020-11-16T22:27:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.509+0000 7f85ecafd700 1 -- 172.21.15.181:0/1447572015 learned_addr learned my addr 172.21.15.181:0/1447572015 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e7fff700 1 -- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 msgr2=0x7f85e8061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e7fff700 1 --2- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 0x7f85e8061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e7fff700 1 -- 172.21.15.181:0/1447572015 >> v1:172.21.15.175:6789/0 conn(0x7f85e8160740 legacy=0x7f85e8061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e7fff700 1 -- 172.21.15.181:0/1447572015 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8065030 con 0x7f85e81426d0 2020-11-16T22:27:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e7fff700 1 --2- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e81426d0 0x7f85e8142ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f85d000b9a0 tx=0x7f85d0006df0).ready entity=mon.1 client_cookie=ed626fa0fd976d2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e6ffd700 1 -- 172.21.15.181:0/1447572015 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00097b0 con 0x7f85e81426d0 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e6ffd700 1 -- 172.21.15.181:0/1447572015 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0009910 con 0x7f85e81426d0 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85e6ffd700 1 -- 172.21.15.181:0/1447572015 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0009c90 con 0x7f85e81426d0 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 -- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e81426d0 msgr2=0x7f85e8142ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 --2- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e81426d0 0x7f85e8142ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f85d000b9a0 tx=0x7f85d0006df0).stop 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 -- 172.21.15.181:0/1447572015 shutdown_connections 2020-11-16T22:27:17.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 --2- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160bf0 0x7f85e8061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 --2- 172.21.15.181:0/1447572015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e81426d0 0x7f85e8142ab0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 -- 172.21.15.181:0/1447572015 >> 172.21.15.181:0/1447572015 conn(0x7f85e80cf890 msgr2=0x7f85e80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 -- 172.21.15.181:0/1447572015 shutdown_connections 2020-11-16T22:27:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.513+0000 7f85ee560700 1 -- 172.21.15.181:0/1447572015 wait complete. 2020-11-16T22:27:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 Processor -- start 2020-11-16T22:27:17.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- start start 2020-11-16T22:27:17.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85e81426d0 0x7f85e81419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 0x7f85e8158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e816f3e0 con 0x7f85e81426d0 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e816f630 con 0x7f85e8160bf0 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e816f8d0 con 0x7f85e8160740 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ecafd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 0x7f85e8158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ecafd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 0x7f85e8158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57172/0 (socket says 172.21.15.181:57172) 2020-11-16T22:27:17.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36782/0 (socket says 172.21.15.181:36782) 2020-11-16T22:27:17.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ecafd700 1 -- 172.21.15.181:0/3184567775 learned_addr learned my addr 172.21.15.181:0/3184567775 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e7fff700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85e81426d0 0x7f85e81419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 -- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 msgr2=0x7f85e8158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 0x7f85e8158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 -- 172.21.15.181:0/3184567775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85e81426d0 msgr2=0x7f85e81419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85e81426d0 0x7f85e81419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 -- 172.21.15.181:0/3184567775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e816fb70 con 0x7f85e8160740 2020-11-16T22:27:17.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85e77fe700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f85d800c6e0 tx=0x7f85d8007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d800f470 con 0x7f85e8160740 2020-11-16T22:27:17.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800f5d0 con 0x7f85e8160740 2020-11-16T22:27:17.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e816fe10 con 0x7f85e8160740 2020-11-16T22:27:17.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d800f950 con 0x7f85e8160740 2020-11-16T22:27:17.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.517+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e8141ec0 con 0x7f85e8160740 2020-11-16T22:27:17.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8069eb0 con 0x7f85e8160740 2020-11-16T22:27:17.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f85d8019030 con 0x7f85e8160740 2020-11-16T22:27:17.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85dffff700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 0x7f85c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85d806cfa0 con 0x7f85e8160740 2020-11-16T22:27:17.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85e7fff700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 0x7f85c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.521+0000 7f85e7fff700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 0x7f85c4057520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f85d0006e30 tx=0x7f85d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.525+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f85d80432b0 con 0x7f85e8160740 2020-11-16T22:27:17.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85e8163250 con 0x7f85e8160740 2020-11-16T22:27:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85dffff700 1 -- 172.21.15.181:0/3184567775 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f85d8017030 con 0x7f85e8160740 2020-11-16T22:27:17.666 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 msgr2=0x7f85c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 0x7f85c4057520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f85d0006e30 tx=0x7f85d0015000).stop 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 msgr2=0x7f85e8158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f85d800c6e0 tx=0x7f85d8007b50).stop 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 shutdown_connections 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85c4055060 0x7f85c4057520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8160bf0 0x7f85e8158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85e81426d0 0x7f85e81419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 --2- 172.21.15.181:0/3184567775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8160740 0x7f85e8158730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 >> 172.21.15.181:0/3184567775 conn(0x7f85e80cf890 msgr2=0x7f85e80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 shutdown_connections 2020-11-16T22:27:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.681+0000 7f85ee560700 1 -- 172.21.15.181:0/3184567775 wait complete. 2020-11-16T22:27:17.680 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:27:17.681 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:27:17.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.813+0000 7f66a5e79700 1 Processor -- start 2020-11-16T22:27:17.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.813+0000 7f66a5e79700 1 -- start start 2020-11-16T22:27:17.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66a0062740 0x7f66a00644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a00c8db0 con 0x7f66a0161480 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a0165030 con 0x7f66a0062740 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00c8330 con 0x7f66a0161860 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36788/0 (socket says 172.21.15.181:36788) 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5678700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66a0062740 0x7f66a00644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 -- 172.21.15.181:0/263892899 learned_addr learned my addr 172.21.15.181:0/263892899 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 -- 172.21.15.181:0/263892899 >> v2:172.21.15.181:3300/0 conn(0x7f66a0062740 msgr2=0x7f66a00644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 --2- 172.21.15.181:0/263892899 >> v2:172.21.15.181:3300/0 conn(0x7f66a0062740 0x7f66a00644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 -- 172.21.15.181:0/263892899 >> v1:172.21.15.175:6789/0 conn(0x7f66a0161480 legacy=0x7f66a0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 -- 172.21.15.181:0/263892899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00c85d0 con 0x7f66a0161860 2020-11-16T22:27:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f6697fff700 1 --2- 172.21.15.181:0/263892899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f66a00b8580 tx=0x7f6690006eb0).ready entity=mon.2 client_cookie=fb6bdbdbdeb2f5a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66977fe700 1 -- 172.21.15.181:0/263892899 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900097d0 con 0x7f66a0161860 2020-11-16T22:27:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66977fe700 1 -- 172.21.15.181:0/263892899 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690009930 con 0x7f66a0161860 2020-11-16T22:27:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66977fe700 1 -- 172.21.15.181:0/263892899 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690009cb0 con 0x7f66a0161860 2020-11-16T22:27:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- 172.21.15.181:0/263892899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 msgr2=0x7f66a0062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 --2- 172.21.15.181:0/263892899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f66a00b8580 tx=0x7f6690006eb0).stop 2020-11-16T22:27:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- 172.21.15.181:0/263892899 shutdown_connections 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 --2- 172.21.15.181:0/263892899 >> v2:172.21.15.181:3300/0 conn(0x7f66a0062740 0x7f66a00644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 --2- 172.21.15.181:0/263892899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0062250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- 172.21.15.181:0/263892899 >> 172.21.15.181:0/263892899 conn(0x7f66a00d2dc0 msgr2=0x7f66a00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- 172.21.15.181:0/263892899 shutdown_connections 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- 172.21.15.181:0/263892899 wait complete. 2020-11-16T22:27:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 Processor -- start 2020-11-16T22:27:17.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.817+0000 7f66a5e79700 1 -- start start 2020-11-16T22:27:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a0062740 0x7f66a00b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66a0161480 0x7f66a00d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00b6e20 con 0x7f66a0062740 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00b70c0 con 0x7f66a0161480 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00b7360 con 0x7f66a0161860 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36798/0 (socket says 172.21.15.181:36798) 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 -- 172.21.15.181:0/3141050723 learned_addr learned my addr 172.21.15.181:0/3141050723 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f6697fff700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66a0161480 0x7f66a00d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a4e77700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a0062740 0x7f66a00b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 -- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66a0161480 msgr2=0x7f66a00d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66a0161480 0x7f66a00d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 -- 172.21.15.181:0/3141050723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a0062740 msgr2=0x7f66a00b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a0062740 0x7f66a00b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 -- 172.21.15.181:0/3141050723 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00b7600 con 0x7f66a0161860 2020-11-16T22:27:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5678700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f669800ef20 tx=0x7f66980091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698009690 con 0x7f66a0161860 2020-11-16T22:27:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669800cbc0 con 0x7f66a0161860 2020-11-16T22:27:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66980123f0 con 0x7f66a0161860 2020-11-16T22:27:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a00b78a0 con 0x7f66a0161860 2020-11-16T22:27:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.821+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a00b6a00 con 0x7f66a0161860 2020-11-16T22:27:17.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a00b6c50 con 0x7f66a0161860 2020-11-16T22:27:17.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6698022070 con 0x7f66a0161860 2020-11-16T22:27:17.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f6694ff9700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 0x7f6678057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:17.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f669806e460 con 0x7f66a0161860 2020-11-16T22:27:17.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f66a4e77700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 0x7f6678057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:17.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.825+0000 7f66a4e77700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 0x7f6678057550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f66a0161030 tx=0x7f669c007b70).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:17.833+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f66980446c0 con 0x7f66a0161860 2020-11-16T22:27:18.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66a0044f60 con 0x7f66a0161860 2020-11-16T22:27:18.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f6694ff9700 1 -- 172.21.15.181:0/3141050723 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f6698016070 con 0x7f66a0161860 2020-11-16T22:27:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:18.019 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:27:02.107409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2,3,4,5,6,7,8],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299,"mds_3":4296,"mds_4":4276,"mds_5":4266,"mds_6":4256,"mds_7":4253,"mds_8":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":8,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4253":{"gid":4253,"name":"g","rank":7,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4256":{"gid":4256,"name":"a","rank":6,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4266":{"gid":4266,"name":"e","rank":5,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4276":{"gid":4276,"name":"b","rank":4,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4296":{"gid":4296,"name":"h","rank":3,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:27:18.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 msgr2=0x7f6678057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 0x7f6678057550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f66a0161030 tx=0x7f669c007b70).stop 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 msgr2=0x7f66a0165f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f669800ef20 tx=0x7f66980091d0).stop 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 shutdown_connections 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6678055090 0x7f6678057550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:18.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66a0161860 0x7f66a0165f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a0062740 0x7f66a00b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 --2- 172.21.15.181:0/3141050723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66a0161480 0x7f66a00d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 >> 172.21.15.181:0/3141050723 conn(0x7f66a00d2dc0 msgr2=0x7f66a00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 shutdown_connections 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:18.033+0000 7f66a5e79700 1 -- 172.21.15.181:0/3141050723 wait complete. 2020-11-16T22:27:18.022 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 22 2020-11-16T22:27:18.941 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:26:51.998542+0000', 'modified': '2020-11-16T22:27:02.107409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 9, 'in': [0, 1, 2, 3, 4, 5, 6, 7, 8], 'up': {'mds_0': 4305, 'mds_1': 4315, 'mds_2': 4299, 'mds_3': 4296, 'mds_4': 4276, 'mds_5': 4266, 'mds_6': 4256, 'mds_7': 4253, 'mds_8': 4244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4244': {'gid': 4244, 'name': 'd', 'rank': 8, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6837/3313292089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3313292089}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3313292089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4253': {'gid': 4253, 'name': 'g', 'rank': 7, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6835/228556183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 228556183}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 228556183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4256': {'gid': 4256, 'name': 'a', 'rank': 6, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6840/2424566790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2424566790}, {'type': 'v1', 'addr': '172.21.15.181:6840', 'nonce': 2424566790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4266': {'gid': 4266, 'name': 'e', 'rank': 5, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6833/3836362112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 3836362112}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 3836362112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 4, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6835/3777546427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3777546427}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3777546427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4296': {'gid': 4296, 'name': 'h', 'rank': 3, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/390679745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6837', 'nonce': 390679745}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 390679745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4299': {'gid': 4299, 'name': 'f', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6838/2983366267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2983366267}, {'type': 'v1', 'addr': '172.21.15.175:6838', 'nonce': 2983366267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4305': {'gid': 4305, 'name': 'i', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.175:6841/1137266867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 1137266867}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 1137266867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4315': {'gid': 4315, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3996054507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3996054507}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3996054507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:27:18.942 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 9/9 2020-11-16T22:27:18.942 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:27:19.093 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":8,"id":4244,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.991751236999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":30.852190163} 2020-11-16T22:27:19.094 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4244, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 17.991751237, "state": "up:active", "uptime": 30.852190163, "want_state": "up:active", "whoami": 8 } 2020-11-16T22:27:19.094 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.g.asok --format=json status 2020-11-16T22:27:19.233 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":7,"id":4253,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.143266357000002,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":30.987940784999999} 2020-11-16T22:27:19.233 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4253, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 19.143266357, "state": "up:active", "uptime": 30.987940785, "want_state": "up:active", "whoami": 7 } 2020-11-16T22:27:19.234 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:27:19.374 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":6,"id":4256,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.296505410999998,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.126352695000001} 2020-11-16T22:27:19.374 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4256, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 20.296505411, "state": "up:active", "uptime": 31.126352695, "want_state": "up:active", "whoami": 6 } 2020-11-16T22:27:19.375 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:27:19.510 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":5,"id":4266,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.446564645999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.273059022999998} 2020-11-16T22:27:19.510 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4266, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 21.446564646, "state": "up:active", "uptime": 31.273059023, "want_state": "up:active", "whoami": 5 } 2020-11-16T22:27:19.511 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:27:19.652 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":4,"id":4276,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.600406599999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.421945956999998} 2020-11-16T22:27:19.652 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4276, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 22.6004066, "state": "up:active", "uptime": 31.421945957, "want_state": "up:active", "whoami": 4 } 2020-11-16T22:27:19.653 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok --format=json status 2020-11-16T22:27:19.789 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":3,"id":4296,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.749504440999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.534167116999999} 2020-11-16T22:27:19.790 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4296, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 23.749504441, "state": "up:active", "uptime": 31.534167117, "want_state": "up:active", "whoami": 3 } 2020-11-16T22:27:19.790 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2020-11-16T22:27:19.918 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":4299,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.88863392,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.663184068} 2020-11-16T22:27:19.918 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4299, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 24.88863392, "state": "up:active", "uptime": 31.663184068, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:27:19.919 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:27:20.047 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":4305,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.054112077999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.782891201999998} 2020-11-16T22:27:20.048 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4305, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 28.054112078, "state": "up:active", "uptime": 31.782891202, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:27:20.048 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:27:20.188 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":4315,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":26.172039175999998,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.940874946000001} 2020-11-16T22:27:20.189 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4315, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 26.172039176, "state": "up:active", "uptime": 31.940874946, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:27:20.189 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:27:20.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.190 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:27:20.210 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:27:20.210 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:20.292 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:20.292 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:27:20.392 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:20.392 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:27:20.399 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:27:20.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.400 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:27:20.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:27:20.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:27:20.444 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-16T22:27:20.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-16T22:27:20.444 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:27:20.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-16T22:27:20.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:27:20.445 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:27:20.496 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:27:20.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.500 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:27:20.514 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:27:20.515 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:27:20.515 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:27:20.515 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:27:20.515 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:27:20.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.516 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:20.516 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:20.517 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:27:20.517 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:27:20.517 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:27:20.517 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:20.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.610 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:27:20.621 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:27:20.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.623 INFO:teuthology.orchestra.run.smithi106:> ip netns list-id 2020-11-16T22:27:20.632 INFO:teuthology.orchestra.run.smithi106.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-11-16T22:27:20.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.633 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:20.634 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:20.634 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2020-11-16T22:27:20.634 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1 2020-11-16T22:27:20.634 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.753 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-16T22:27:20.783 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:27:20.784 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:27:20.784 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:27:20.784 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.784 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:27:20.784 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether f2:b5:72:be:de:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::f0b5:72ff:febe:de11/64 scope link 2020-11-16T22:27:20.785 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:20.788 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-16T22:27:20.817 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:27:20.818 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:27:20.818 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:27:20.818 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.818 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:27:20.818 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.819 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:27:20.819 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether f2:b5:72:be:de:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:27:20.819 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-16T22:27:20.819 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:20.819 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::f0b5:72ff:febe:de11/64 scope link 2020-11-16T22:27:20.820 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:27:25.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:25.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:25.208 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:27:25.208 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:27:25.209 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.1 up 2020-11-16T22:27:25.209 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.1 master ceph-brx 2020-11-16T22:27:25.209 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:27:25.285 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 Processor -- start 2020-11-16T22:27:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 -- start start 2020-11-16T22:27:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc544167ac0 0x7fc544167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc544168390 con 0x7fc5441672c0 2020-11-16T22:27:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440b2060 con 0x7fc5440b3220 2020-11-16T22:27:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc54a0ee700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440b5f00 con 0x7fc544167ac0 2020-11-16T22:27:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc542ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc5440b3220 0x7fc544166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43268/0 (socket says 172.21.15.181:43268) 2020-11-16T22:27:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc542ffd700 1 -- 172.21.15.181:0/2803097898 learned_addr learned my addr 172.21.15.181:0/2803097898 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2803097898 >> v2:172.21.15.175:3300/0 conn(0x7fc544167ac0 0x7fc544167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5427fc700 1 -- 172.21.15.181:0/2803097898 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177181063 0 0) 0x7fc5440b2060 con 0x7fc5440b3220 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5427fc700 1 -- 172.21.15.181:0/2803097898 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc528003240 con 0x7fc5440b3220 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5437fe700 1 -- 172.21.15.181:0/2803097898 >> v1:172.21.15.181:6789/0 conn(0x7fc5440b3220 legacy=0x7fc544166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5437fe700 1 -- 172.21.15.181:0/2803097898 >> v1:172.21.15.175:6789/0 conn(0x7fc5441672c0 legacy=0x7fc544161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5437fe700 1 -- 172.21.15.181:0/2803097898 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440b2300 con 0x7fc544167ac0 2020-11-16T22:27:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2803097898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544167ac0 0x7fc544167ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc534009c90 tx=0x7fc5340091d0).ready entity=mon.0 client_cookie=84c52948c5205c30 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5427fc700 1 -- 172.21.15.181:0/2803097898 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53400ba70 con 0x7fc544167ac0 2020-11-16T22:27:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5427fc700 1 -- 172.21.15.181:0/2803097898 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53400bbd0 con 0x7fc544167ac0 2020-11-16T22:27:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.429+0000 7fc5427fc700 1 -- 172.21.15.181:0/2803097898 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534011400 con 0x7fc544167ac0 2020-11-16T22:27:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2803097898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544167ac0 msgr2=0x7fc544167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2803097898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544167ac0 0x7fc544167ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc534009c90 tx=0x7fc5340091d0).stop 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2803097898 shutdown_connections 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2803097898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544167ac0 0x7fc544167ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2803097898 >> 172.21.15.181:0/2803097898 conn(0x7fc544060bc0 msgr2=0x7fc5440d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2803097898 shutdown_connections 2020-11-16T22:27:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2803097898 wait complete. 2020-11-16T22:27:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 Processor -- start 2020-11-16T22:27:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- start start 2020-11-16T22:27:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 0x7fc5441740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc544167ac0 0x7fc5440a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544175600 con 0x7fc5440b3220 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441758a0 con 0x7fc5441672c0 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc54a0ee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc544175b40 con 0x7fc544167ac0 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc542ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc542ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57200/0 (socket says 172.21.15.181:57200) 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc542ffd700 1 -- 172.21.15.181:0/2256687158 learned_addr learned my addr 172.21.15.181:0/2256687158 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc543fff700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc544167ac0 0x7fc5440a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.433+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 0x7fc5441740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 -- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc544167ac0 msgr2=0x7fc5440a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc544167ac0 0x7fc5440a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 -- 172.21.15.181:0/2256687158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 msgr2=0x7fc5441740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 0x7fc5441740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 -- 172.21.15.181:0/2256687158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544175de0 con 0x7fc5441672c0 2020-11-16T22:27:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 0x7fc5441740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc542ffd700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fc52c00c6e0 tx=0x7fc52c007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0072e0 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c007440 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc544176080 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c005800 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc544174590 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54405d2e0 con 0x7fc5441672c0 2020-11-16T22:27:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc52c010070 con 0x7fc5441672c0 2020-11-16T22:27:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 0x7fc514057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc52c06d4c0 con 0x7fc5441672c0 2020-11-16T22:27:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.437+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 0x7fc514057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.441+0000 7fc5437fe700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 0x7fc514057520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc534006f30 tx=0x7fc534016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.449+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc54405d2e0 con 0x7fc5441672c0 2020-11-16T22:27:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.601+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc54405d2e0 con 0x7fc5441672c0 2020-11-16T22:27:25.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc53b7fe700 1 -- 172.21.15.181:0/2256687158 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fc54405d2e0 con 0x7fc5441672c0 2020-11-16T22:27:25.590 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:25.591 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 msgr2=0x7fc514057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 0x7fc514057520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc534006f30 tx=0x7fc534016000).stop 2020-11-16T22:27:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 msgr2=0x7fc544065880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fc52c00c6e0 tx=0x7fc52c007ad0).stop 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 shutdown_connections 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc514055060 0x7fc514057520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc544167ac0 0x7fc5440a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5440b3220 0x7fc5441740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 --2- 172.21.15.181:0/2256687158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5441672c0 0x7fc544065880 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 >> 172.21.15.181:0/2256687158 conn(0x7fc544060bc0 msgr2=0x7fc544162140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:25.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 shutdown_connections 2020-11-16T22:27:25.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:25.605+0000 7fc54a0ee700 1 -- 172.21.15.181:0/2256687158 wait complete. 2020-11-16T22:27:25.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:26.002 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:26.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.153+0000 7f1fcd0e7700 1 Processor -- start 2020-11-16T22:27:26.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 -- start start 2020-11-16T22:27:26.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 0x7f1fc8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 0x7f1fc8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc8061b50 con 0x7f1fc81492c0 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc80618b0 con 0x7f1fc80814c0 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcd0e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80cb930 con 0x7f1fc8145a30 2020-11-16T22:27:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcc8e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 0x7f1fc8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcc8e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57208/0 (socket says 172.21.15.181:57208) 2020-11-16T22:27:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcc8e6700 1 -- 172.21.15.181:0/238074665 learned_addr learned my addr 172.21.15.181:0/238074665 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc77fe700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 0x7f1fc8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 -- 172.21.15.181:0/238074665 >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 msgr2=0x7f1fc80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:26.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 -- 172.21.15.181:0/238074665 >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 msgr2=0x7f1fc8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 0x7f1fc8080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 -- 172.21.15.181:0/238074665 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80cbbd0 con 0x7f1fc8145a30 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fcc8e6700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc77fe700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 0x7f1fc8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:26.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/238074665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 0x7f1fc8145e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1fc8082560 tx=0x7f1fbc006df0).ready entity=mon.2 client_cookie=c27d22d641057c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.157+0000 7f1fc6ffd700 1 -- 172.21.15.181:0/238074665 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0097b0 con 0x7f1fc8145a30 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fc6ffd700 1 -- 172.21.15.181:0/238074665 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc009910 con 0x7f1fc8145a30 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fc6ffd700 1 -- 172.21.15.181:0/238074665 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc009c90 con 0x7f1fc8145a30 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/238074665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 msgr2=0x7f1fc8145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/238074665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 0x7f1fc8145e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1fc8082560 tx=0x7f1fbc006df0).stop 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/238074665 shutdown_connections 2020-11-16T22:27:26.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.181:3300/0 conn(0x7f1fc80814c0 0x7f1fc80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/238074665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc8145a30 0x7f1fc8145e10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/238074665 >> v2:172.21.15.175:3300/0 conn(0x7f1fc81492c0 0x7f1fc8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/238074665 >> 172.21.15.181:0/238074665 conn(0x7f1fc80d68c0 msgr2=0x7f1fc80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:26.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/238074665 shutdown_connections 2020-11-16T22:27:26.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.161+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/238074665 wait complete. 2020-11-16T22:27:26.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 Processor -- start 2020-11-16T22:27:26.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 -- start start 2020-11-16T22:27:26.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fc8145a30 0x7f1fc80d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 0x7f1fc813f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8133320 con 0x7f1fc8145a30 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc81475e0 con 0x7f1fc81492c0 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcd0e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8147880 con 0x7f1fc80814c0 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcc8e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 0x7f1fc813f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36822/0 (socket says 172.21.15.181:36822) 2020-11-16T22:27:26.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 -- 172.21.15.181:0/2524953282 learned_addr learned my addr 172.21.15.181:0/2524953282 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc77fe700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fc8145a30 0x7f1fc80d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 -- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 msgr2=0x7f1fc813f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 0x7f1fc813f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 -- 172.21.15.181:0/2524953282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fc8145a30 msgr2=0x7f1fc80d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fc8145a30 0x7f1fc80d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 -- 172.21.15.181:0/2524953282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8147b20 con 0x7f1fc80814c0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fcc8e6700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 0x7f1fc813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fc7fff700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1fbc006e30 tx=0x7f1fbc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.165+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc011c50 con 0x7f1fc80814c0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc011db0 con 0x7f1fc80814c0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc8147dc0 con 0x7f1fc80814c0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc01e690 con 0x7f1fc80814c0 2020-11-16T22:27:26.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc8148300 con 0x7f1fc80814c0 2020-11-16T22:27:26.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8145660 con 0x7f1fc80814c0 2020-11-16T22:27:26.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1fbc025030 con 0x7f1fc80814c0 2020-11-16T22:27:26.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fa7fff700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 0x7f1fa0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fc77fe700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 0x7f1fa0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1fbc06f050 con 0x7f1fc80814c0 2020-11-16T22:27:26.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.169+0000 7f1fc77fe700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 0x7f1fa0057520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1fc8062c50 tx=0x7f1fb8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.177+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1fc8145660 con 0x7f1fc80814c0 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.329+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1fc8145660 con 0x7f1fc80814c0 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.329+0000 7f1fa7fff700 1 -- 172.21.15.181:0/2524953282 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1fc8145660 con 0x7f1fc80814c0 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 msgr2=0x7f1fa0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 0x7f1fa0057520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1fc8062c50 tx=0x7f1fb8008040).stop 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 msgr2=0x7f1fc8146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1fbc006e30 tx=0x7f1fbc015000).stop 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 shutdown_connections 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1fa0055060 0x7f1fa0057520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fc81492c0 0x7f1fc813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fc80814c0 0x7f1fc8146f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 --2- 172.21.15.181:0/2524953282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fc8145a30 0x7f1fc80d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 >> 172.21.15.181:0/2524953282 conn(0x7f1fc80d68c0 msgr2=0x7f1fc8141570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:26.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 shutdown_connections 2020-11-16T22:27:26.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.333+0000 7f1fcd0e7700 1 -- 172.21.15.181:0/2524953282 wait complete. 2020-11-16T22:27:26.565 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:26.565 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.721+0000 7fd8bf59e700 1 Processor -- start 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.721+0000 7fd8bf59e700 1 -- start start 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.721+0000 7fd8bf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8c0145a30 0x7fd8c0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.721+0000 7fd8bf59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 0x7fd8c00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c0061b50 con 0x7fd8c01492c0 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00618b0 con 0x7fd8c00816e0 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00cb930 con 0x7fd8c0145a30 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8c0145a30 0x7fd8c0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8c0145a30 0x7fd8c0145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57220/0 (socket says 172.21.15.181:57220) 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 -- 172.21.15.181:0/2828780390 learned_addr learned my addr 172.21.15.181:0/2828780390 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bdd9b700 1 --1- 172.21.15.181:0/2828780390 >> v1:172.21.15.181:6789/0 conn(0x7fd8c01492c0 0x7fd8c0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43294/0 (socket says 172.21.15.181:43294) 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bed9d700 1 --2- 172.21.15.181:0/2828780390 >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 0x7fd8c00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 -- 172.21.15.181:0/2828780390 >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 msgr2=0x7fd8c00cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 --2- 172.21.15.181:0/2828780390 >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 0x7fd8c00cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 -- 172.21.15.181:0/2828780390 >> v1:172.21.15.181:6789/0 conn(0x7fd8c01492c0 legacy=0x7fd8c0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 -- 172.21.15.181:0/2828780390 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00cbbd0 con 0x7fd8c0145a30 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bed9d700 1 --2- 172.21.15.181:0/2828780390 >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 0x7fd8c00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8be59c700 1 --2- 172.21.15.181:0/2828780390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c0145a30 0x7fd8c0145e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fd8c00d43d0 tx=0x7fd8b0006eb0).ready entity=mon.1 client_cookie=2d9f42672f0498de server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bd59a700 1 -- 172.21.15.181:0/2828780390 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00098d0 con 0x7fd8c0145a30 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bd59a700 1 -- 172.21.15.181:0/2828780390 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0009a30 con 0x7fd8c0145a30 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bd59a700 1 -- 172.21.15.181:0/2828780390 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0009db0 con 0x7fd8c0145a30 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- 172.21.15.181:0/2828780390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c0145a30 msgr2=0x7fd8c0145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/2828780390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c0145a30 0x7fd8c0145e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fd8c00d43d0 tx=0x7fd8b0006eb0).stop 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- 172.21.15.181:0/2828780390 shutdown_connections 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/2828780390 >> v2:172.21.15.175:3300/0 conn(0x7fd8c00816e0 0x7fd8c00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/2828780390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c0145a30 0x7fd8c0145e10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- 172.21.15.181:0/2828780390 >> 172.21.15.181:0/2828780390 conn(0x7fd8c00d68c0 msgr2=0x7fd8c00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- 172.21.15.181:0/2828780390 shutdown_connections 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.725+0000 7fd8bf59e700 1 -- 172.21.15.181:0/2828780390 wait complete. 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 Processor -- start 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- start start 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c01492c0 0x7fd8c0147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0170750 con 0x7fd8c01492c0 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0170c00 con 0x7fd8c00816e0 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0170e30 con 0x7fd8c0145a30 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bdd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57226/0 (socket says 172.21.15.181:57226) 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bdd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36838/0 (socket says 172.21.15.181:36838) 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 -- 172.21.15.181:0/1536963701 learned_addr learned my addr 172.21.15.181:0/1536963701 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bed9d700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c01492c0 0x7fd8c0147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 -- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 msgr2=0x7fd8c015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 -- 172.21.15.181:0/1536963701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c01492c0 msgr2=0x7fd8c0147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c01492c0 0x7fd8c0147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 -- 172.21.15.181:0/1536963701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c01710d0 con 0x7fd8c00816e0 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bdd9b700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8be59c700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fd8c00cba70 tx=0x7fd8b0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0011de0 con 0x7fd8c00816e0 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b001e460 con 0x7fd8c00816e0 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b001e7e0 con 0x7fd8c00816e0 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c0171370 con 0x7fd8c00816e0 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.729+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c01718b0 con 0x7fd8c00816e0 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c0044f60 con 0x7fd8c00816e0 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd8b0025030 con 0x7fd8c00816e0 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8ae7fc700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 0x7fd894057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8b006f4a0 con 0x7fd8c00816e0 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8bdd9b700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 0x7fd894057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.733+0000 7fd8bdd9b700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 0x7fd894057520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd8c00cc580 tx=0x7fd8b4007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.741+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd8b0045750 con 0x7fd8c00816e0 2020-11-16T22:27:26.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8c0160af0 con 0x7fd8c00816e0 2020-11-16T22:27:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8ae7fc700 1 -- 172.21.15.181:0/1536963701 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8b0023070 con 0x7fd8c00816e0 2020-11-16T22:27:26.879 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:26.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 msgr2=0x7fd894057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 0x7fd894057520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd8c00cc580 tx=0x7fd8b4007b80).stop 2020-11-16T22:27:26.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 msgr2=0x7fd8c015d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:26.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.893+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fd8c00cba70 tx=0x7fd8b0015000).stop 2020-11-16T22:27:26.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 shutdown_connections 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd894055060 0x7fd894057520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8c01492c0 0x7fd8c0147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8c00816e0 0x7fd8c015d210 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 --2- 172.21.15.181:0/1536963701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c0145a30 0x7fd8c015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 >> 172.21.15.181:0/1536963701 conn(0x7fd8c00d68c0 msgr2=0x7fd8c00b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 shutdown_connections 2020-11-16T22:27:26.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:26.897+0000 7fd8bf59e700 1 -- 172.21.15.181:0/1536963701 wait complete. 2020-11-16T22:27:26.894 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:27.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 Processor -- start 2020-11-16T22:27:27.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 -- start start 2020-11-16T22:27:27.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f86300d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86300615f0 0x7f86300d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8630160e40 con 0x7f86300622a0 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86301609a0 con 0x7f86300615f0 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f8635bf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8630160ba0 con 0x7f863015f5b0 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f86300622a0 0x7f8630062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43308/0 (socket says 172.21.15.181:43308) 2020-11-16T22:27:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862ffff700 1 -- 172.21.15.181:0/733443970 learned_addr learned my addr 172.21.15.181:0/733443970 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862f7fe700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f86300d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862effd700 1 --2- 172.21.15.181:0/733443970 >> v2:172.21.15.175:3300/0 conn(0x7f86300615f0 0x7f86300d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751265604 0 0) 0x7f8630160e40 con 0x7f86300622a0 2020-11-16T22:27:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f861c003240 con 0x7f86300622a0 2020-11-16T22:27:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 692073017 0 0) 0x7f861c003240 con 0x7f86300622a0 2020-11-16T22:27:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 msgr2=0x7f86300d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f86300d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 >> v2:172.21.15.175:3300/0 conn(0x7f86300615f0 msgr2=0x7f86300d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 --2- 172.21.15.181:0/733443970 >> v2:172.21.15.175:3300/0 conn(0x7f86300615f0 0x7f86300d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630060970 con 0x7f86300622a0 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f8628001860 con 0x7f86300622a0 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 >> v1:172.21.15.181:6789/0 conn(0x7f86300622a0 legacy=0x7f8630062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c003730 0x7f861c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0044e0 0x7f861c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f861c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86301609a0 con 0x7f861c003730 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8630160ba0 con 0x7f861c0044e0 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c004270 con 0x7f863015f5b0 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862effd700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0044e0 0x7f861c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862ffff700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f861c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.037+0000 7f862f7fe700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c003730 0x7f861c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 -- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 msgr2=0x7f861c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f863015f5b0 0x7f861c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 -- 172.21.15.181:0/733443970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c003730 msgr2=0x7f861c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c003730 0x7f861c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 -- 172.21.15.181:0/733443970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c003240 con 0x7f861c0044e0 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0044e0 0x7f861c004960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f8624006fd0 tx=0x7f8624006df0).ready entity=mon.1 client_cookie=28f3e316a3b4d54f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400b320 con 0x7f861c0044e0 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400ea10 con 0x7f861c0044e0 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862e7fc700 1 -- 172.21.15.181:0/733443970 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400ed90 con 0x7f861c0044e0 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0044e0 msgr2=0x7f861c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 --2- 172.21.15.181:0/733443970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0044e0 0x7f861c004960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f8624006fd0 tx=0x7f8624006df0).stop 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862ffff700 1 -- 172.21.15.181:0/733443970 reap_dead start 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- 172.21.15.181:0/733443970 shutdown_connections 2020-11-16T22:27:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- 172.21.15.181:0/733443970 >> 172.21.15.181:0/733443970 conn(0x7f86300d69a0 msgr2=0x7f86300d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- 172.21.15.181:0/733443970 shutdown_connections 2020-11-16T22:27:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- 172.21.15.181:0/733443970 wait complete. 2020-11-16T22:27:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 Processor -- start 2020-11-16T22:27:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- start start 2020-11-16T22:27:27.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c0044e0 0x7f86300cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8630163990 0x7f86300d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8630170a60 con 0x7f861c0044e0 2020-11-16T22:27:27.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86300cc020 con 0x7f8630163990 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f8635bf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86300bacf0 con 0x7f8630163e10 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36860/0 (socket says 172.21.15.181:36860) 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862ffff700 1 -- 172.21.15.181:0/3160879174 learned_addr learned my addr 172.21.15.181:0/3160879174 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.041+0000 7f862effd700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8630163990 0x7f86300d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862f7fe700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c0044e0 0x7f86300cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 -- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8630163990 msgr2=0x7f86300d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8630163990 0x7f86300d47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 -- 172.21.15.181:0/3160879174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c0044e0 msgr2=0x7f86300cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c0044e0 0x7f86300cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 -- 172.21.15.181:0/3160879174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86300baf90 con 0x7f8630163e10 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f862ffff700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f8630083ad0 tx=0x7f8628000fa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800ca30 con 0x7f8630163e10 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862800cb90 con 0x7f8630163e10 2020-11-16T22:27:27.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86300bb230 con 0x7f8630163e10 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628014430 con 0x7f8630163e10 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8630170f30 con 0x7f8630163e10 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8628001570 con 0x7f8630163e10 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86300b92d0 con 0x7f8630163e10 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.045+0000 7f860f7fe700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 0x7f8608057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.049+0000 7f862f7fe700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 0x7f8608057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.049+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f862806dff0 con 0x7f8630163e10 2020-11-16T22:27:27.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.049+0000 7f862f7fe700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 0x7f8608057550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8620006800 tx=0x7f8620008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.053+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f86300b92d0 con 0x7f8630163e10 2020-11-16T22:27:27.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86300b92d0 con 0x7f8630163e10 2020-11-16T22:27:27.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f860f7fe700 1 -- 172.21.15.181:0/3160879174 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f86300b92d0 con 0x7f8630163e10 2020-11-16T22:27:27.195 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 msgr2=0x7f8608057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 0x7f8608057550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8620006800 tx=0x7f8620008040).stop 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 msgr2=0x7f86300d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f8630083ad0 tx=0x7f8628000fa0).stop 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 shutdown_connections 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8608055090 0x7f8608057550 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8630163e10 0x7f86300d4cc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f861c0044e0 0x7f86300cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 --2- 172.21.15.181:0/3160879174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8630163990 0x7f86300d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 >> 172.21.15.181:0/3160879174 conn(0x7f86300d69a0 msgr2=0x7f86300d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 shutdown_connections 2020-11-16T22:27:27.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.209+0000 7f8635bf3700 1 -- 172.21.15.181:0/3160879174 wait complete. 2020-11-16T22:27:27.208 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.345+0000 7f4bf759e700 1 Processor -- start 2020-11-16T22:27:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.345+0000 7f4bf759e700 1 -- start start 2020-11-16T22:27:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf8145a30 0x7f4bf8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4bf81492c0 0x7f4bf8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8061b50 con 0x7f4bf80814c0 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80618b0 con 0x7f4bf81492c0 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80cb930 con 0x7f4bf8145a30 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf8145a30 0x7f4bf8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf8145a30 0x7f4bf8145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57254/0 (socket says 172.21.15.181:57254) 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 -- 172.21.15.181:0/2224678419 learned_addr learned my addr 172.21.15.181:0/2224678419 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf5d9b700 1 --2- 172.21.15.181:0/2224678419 >> v2:172.21.15.175:3300/0 conn(0x7f4bf81492c0 0x7f4bf8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 -- 172.21.15.181:0/2224678419 >> v2:172.21.15.175:3300/0 conn(0x7f4bf81492c0 msgr2=0x7f4bf8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 --2- 172.21.15.181:0/2224678419 >> v2:172.21.15.175:3300/0 conn(0x7f4bf81492c0 0x7f4bf8080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 -- 172.21.15.181:0/2224678419 >> v1:172.21.15.175:6789/0 conn(0x7f4bf80814c0 legacy=0x7f4bf80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 -- 172.21.15.181:0/2224678419 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80cbbd0 con 0x7f4bf8145a30 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf659c700 1 --2- 172.21.15.181:0/2224678419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf8145a30 0x7f4bf8145e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4bf80d43d0 tx=0x7f4be8006eb0).ready entity=mon.1 client_cookie=7525ecb445190d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf559a700 1 -- 172.21.15.181:0/2224678419 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8009810 con 0x7f4bf8145a30 2020-11-16T22:27:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf559a700 1 -- 172.21.15.181:0/2224678419 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8009970 con 0x7f4bf8145a30 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf559a700 1 -- 172.21.15.181:0/2224678419 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8009cf0 con 0x7f4bf8145a30 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- 172.21.15.181:0/2224678419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf8145a30 msgr2=0x7f4bf8145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2224678419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf8145a30 0x7f4bf8145e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4bf80d43d0 tx=0x7f4be8006eb0).stop 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- 172.21.15.181:0/2224678419 shutdown_connections 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2224678419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf8145a30 0x7f4bf8145e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2224678419 >> v2:172.21.15.175:3300/0 conn(0x7f4bf81492c0 0x7f4bf8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- 172.21.15.181:0/2224678419 >> 172.21.15.181:0/2224678419 conn(0x7f4bf80d68c0 msgr2=0x7f4bf80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- 172.21.15.181:0/2224678419 shutdown_connections 2020-11-16T22:27:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.349+0000 7f4bf759e700 1 -- 172.21.15.181:0/2224678419 wait complete. 2020-11-16T22:27:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 Processor -- start 2020-11-16T22:27:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- start start 2020-11-16T22:27:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf80814c0 0x7f4bf815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf8170b00 0x7f4bf8170f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8171730 con 0x7f4bf80814c0 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81719d0 con 0x7f4bf81492c0 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8171c70 con 0x7f4bf8170b00 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57262/0 (socket says 172.21.15.181:57262) 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 -- 172.21.15.181:0/2699759726 learned_addr learned my addr 172.21.15.181:0/2699759726 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf6d9d700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf8170b00 0x7f4bf8170f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf659c700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf80814c0 0x7f4bf815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 -- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf8170b00 msgr2=0x7f4bf8170f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf8170b00 0x7f4bf8170f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 -- 172.21.15.181:0/2699759726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf80814c0 msgr2=0x7f4bf815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf80814c0 0x7f4bf815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 -- 172.21.15.181:0/2699759726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8171f10 con 0x7f4bf81492c0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf5d9b700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4be000ebd0 tx=0x7f4be0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000c8e0 con 0x7f4bf81492c0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000ca40 con 0x7f4bf81492c0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf81721b0 con 0x7f4bf81492c0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000cde0 con 0x7f4bf81492c0 2020-11-16T22:27:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.353+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf8171450 con 0x7f4bf81492c0 2020-11-16T22:27:27.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf8044f60 con 0x7f4bf81492c0 2020-11-16T22:27:27.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4be0012600 con 0x7f4bf81492c0 2020-11-16T22:27:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bee7fc700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4be006e180 con 0x7f4bf81492c0 2020-11-16T22:27:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bf659c700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.357+0000 7f4bf659c700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f4bf80d0ee0 tx=0x7f4be8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.361+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4be00443e0 con 0x7f4bf81492c0 2020-11-16T22:27:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bf8068e30 con 0x7f4bf81492c0 2020-11-16T22:27:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bee7fc700 1 -- 172.21.15.181:0/2699759726 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4be0047a00 con 0x7f4bf81492c0 2020-11-16T22:27:27.499 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 msgr2=0x7f4bc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f4bf80d0ee0 tx=0x7f4be8015000).stop 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 msgr2=0x7f4bf815d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4be000ebd0 tx=0x7f4be0006df0).stop 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 shutdown_connections 2020-11-16T22:27:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf8170b00 0x7f4bf8170f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf80814c0 0x7f4bf815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.513+0000 7f4bf759e700 1 --2- 172.21.15.181:0/2699759726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf81492c0 0x7f4bf815d470 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.517+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 >> 172.21.15.181:0/2699759726 conn(0x7f4bf80d68c0 msgr2=0x7f4bf8144b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.517+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 shutdown_connections 2020-11-16T22:27:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.517+0000 7f4bf759e700 1 -- 172.21.15.181:0/2699759726 wait complete. 2020-11-16T22:27:27.513 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:27:27.514 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:27:27.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 Processor -- start 2020-11-16T22:27:27.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 -- start start 2020-11-16T22:27:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7440cf6b0 0x7ff74415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440c8910 con 0x7ff7440cf6b0 2020-11-16T22:27:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b3460 con 0x7ff74415d2e0 2020-11-16T22:27:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff74a386700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440b31d0 con 0x7ff7440b2900 2020-11-16T22:27:27.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749b85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749b85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57270/0 (socket says 172.21.15.181:57270) 2020-11-16T22:27:27.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36882/0 (socket says 172.21.15.181:36882) 2020-11-16T22:27:27.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749b85700 1 -- 172.21.15.181:0/3479669090 learned_addr learned my addr 172.21.15.181:0/3479669090 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:27.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff748b83700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.175:3300/0 conn(0x7ff7440cf6b0 0x7ff74415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 -- 172.21.15.181:0/3479669090 >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 msgr2=0x7ff7440c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 -- 172.21.15.181:0/3479669090 >> v2:172.21.15.175:3300/0 conn(0x7ff7440cf6b0 msgr2=0x7ff74415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.175:3300/0 conn(0x7ff7440cf6b0 0x7ff74415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 -- 172.21.15.181:0/3479669090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440c8130 con 0x7ff7440b2900 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749b85700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff749384700 1 --2- 172.21.15.181:0/3479669090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7ff73c009cc0 tx=0x7ff73c0091d0).ready entity=mon.2 client_cookie=7ba3e9109da927e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.653+0000 7ff733fff700 1 -- 172.21.15.181:0/3479669090 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00ba50 con 0x7ff7440b2900 2020-11-16T22:27:27.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff733fff700 1 -- 172.21.15.181:0/3479669090 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00bbb0 con 0x7ff7440b2900 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff733fff700 1 -- 172.21.15.181:0/3479669090 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011400 con 0x7ff7440b2900 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- 172.21.15.181:0/3479669090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 msgr2=0x7ff7440b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- 172.21.15.181:0/3479669090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7ff73c009cc0 tx=0x7ff73c0091d0).stop 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- 172.21.15.181:0/3479669090 shutdown_connections 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.181:3300/0 conn(0x7ff74415d2e0 0x7ff7440c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- 172.21.15.181:0/3479669090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7440b2900 0x7ff7440b2ce0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- 172.21.15.181:0/3479669090 >> v2:172.21.15.175:3300/0 conn(0x7ff7440cf6b0 0x7ff74415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- 172.21.15.181:0/3479669090 >> 172.21.15.181:0/3479669090 conn(0x7ff7440d2dc0 msgr2=0x7ff7440ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- 172.21.15.181:0/3479669090 shutdown_connections 2020-11-16T22:27:27.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- 172.21.15.181:0/3479669090 wait complete. 2020-11-16T22:27:27.647 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:27.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 Processor -- start 2020-11-16T22:27:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- start start 2020-11-16T22:27:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 0x7ff74416d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74416db30 con 0x7ff7440b2900 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74416ddd0 con 0x7ff7440cf6b0 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff74a386700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74416e070 con 0x7ff74415d2e0 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff748b83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36888/0 (socket says 172.21.15.181:36888) 2020-11-16T22:27:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff748b83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57276/0 (socket says 172.21.15.181:57276) 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 -- 172.21.15.181:0/3944295025 learned_addr learned my addr 172.21.15.181:0/3944295025 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749384700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 0x7ff74416d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 -- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 msgr2=0x7ff744140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 -- 172.21.15.181:0/3944295025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 msgr2=0x7ff74416d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 0x7ff74416d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 -- 172.21.15.181:0/3944295025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440ccba0 con 0x7ff74415d2e0 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749384700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 0x7ff74416d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff748b83700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff749b85700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff73400ec20 tx=0x7ff734006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73400c9b0 con 0x7ff74415d2e0 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73400cb10 con 0x7ff74415d2e0 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.657+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340123f0 con 0x7ff74415d2e0 2020-11-16T22:27:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7440cce40 con 0x7ff74415d2e0 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74416d6a0 con 0x7ff74415d2e0 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744044f60 con 0x7ff74415d2e0 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff734022070 con 0x7ff74415d2e0 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff7317fa700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff73406e3b0 con 0x7ff74415d2e0 2020-11-16T22:27:27.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff749384700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:27.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.661+0000 7ff749384700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 0x7ff71c057520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff73c009c10 tx=0x7ff73c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:27.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.665+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff734044610 con 0x7ff74415d2e0 2020-11-16T22:27:27.659 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:27.661 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:27.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.865+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff74416d8c0 con 0x7ff74415d2e0 2020-11-16T22:27:27.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff7317fa700 1 -- 172.21.15.181:0/3944295025 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4737 (secure 0 0 0) 0x7ff734016090 con 0x7ff74415d2e0 2020-11-16T22:27:27.853 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:27.853 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:27:02.107409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2,3,4,5,6,7,8],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299,"mds_3":4296,"mds_4":4276,"mds_5":4266,"mds_6":4256,"mds_7":4253,"mds_8":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":8,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4253":{"gid":4253,"name":"g","rank":7,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4256":{"gid":4256,"name":"a","rank":6,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4266":{"gid":4266,"name":"e","rank":5,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4276":{"gid":4276,"name":"b","rank":4,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4296":{"gid":4296,"name":"h","rank":3,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:27:27.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 msgr2=0x7ff71c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 0x7ff71c057520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff73c009c10 tx=0x7ff73c016000).stop 2020-11-16T22:27:27.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 msgr2=0x7ff744140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff73400ec20 tx=0x7ff734006eb0).stop 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 shutdown_connections 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff74415d2e0 0x7ff744140720 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7440b2900 0x7ff74416d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 --2- 172.21.15.181:0/3944295025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7440cf6b0 0x7ff744140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:27.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 >> 172.21.15.181:0/3944295025 conn(0x7ff7440d2dc0 msgr2=0x7ff744156470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:27.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 shutdown_connections 2020-11-16T22:27:27.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:27.869+0000 7ff74a386700 1 -- 172.21.15.181:0/3944295025 wait complete. 2020-11-16T22:27:27.857 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 22 2020-11-16T22:27:28.776 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:26:51.998542+0000', 'modified': '2020-11-16T22:27:02.107409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 9, 'in': [0, 1, 2, 3, 4, 5, 6, 7, 8], 'up': {'mds_0': 4305, 'mds_1': 4315, 'mds_2': 4299, 'mds_3': 4296, 'mds_4': 4276, 'mds_5': 4266, 'mds_6': 4256, 'mds_7': 4253, 'mds_8': 4244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4244': {'gid': 4244, 'name': 'd', 'rank': 8, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6837/3313292089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3313292089}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3313292089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4253': {'gid': 4253, 'name': 'g', 'rank': 7, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6835/228556183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 228556183}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 228556183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4256': {'gid': 4256, 'name': 'a', 'rank': 6, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6840/2424566790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2424566790}, {'type': 'v1', 'addr': '172.21.15.181:6840', 'nonce': 2424566790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4266': {'gid': 4266, 'name': 'e', 'rank': 5, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.175:6833/3836362112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 3836362112}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 3836362112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 4, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6835/3777546427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3777546427}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3777546427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4296': {'gid': 4296, 'name': 'h', 'rank': 3, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6839/390679745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6837', 'nonce': 390679745}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 390679745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4299': {'gid': 4299, 'name': 'f', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6838/2983366267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2983366267}, {'type': 'v1', 'addr': '172.21.15.175:6838', 'nonce': 2983366267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4305': {'gid': 4305, 'name': 'i', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.175:6841/1137266867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 1137266867}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 1137266867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4315': {'gid': 4315, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3996054507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3996054507}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3996054507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:27:28.776 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 9/9 2020-11-16T22:27:28.777 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:27:28.913 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":8,"id":4244,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":27.812797385,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":40.673235726999998} 2020-11-16T22:27:28.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4244, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 27.812797385, "state": "up:active", "uptime": 40.673235727, "want_state": "up:active", "whoami": 8 } 2020-11-16T22:27:28.914 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.g.asok --format=json status 2020-11-16T22:27:29.052 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":7,"id":4253,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.962935599000001,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":40.807607687999997} 2020-11-16T22:27:29.052 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4253, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 28.962935599, "state": "up:active", "uptime": 40.807607688, "want_state": "up:active", "whoami": 7 } 2020-11-16T22:27:29.053 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:27:29.191 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":6,"id":4256,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":30.113850934999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":40.943697471999997} 2020-11-16T22:27:29.191 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4256, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 30.113850935, "state": "up:active", "uptime": 40.943697472, "want_state": "up:active", "whoami": 6 } 2020-11-16T22:27:29.192 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:27:29.331 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":5,"id":4266,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.268132877999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.094627262000003} 2020-11-16T22:27:29.332 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4266, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 31.268132878, "state": "up:active", "uptime": 41.094627262, "want_state": "up:active", "whoami": 5 } 2020-11-16T22:27:29.332 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:27:29.468 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":4,"id":4276,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.417099927000002,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.238628167999998} 2020-11-16T22:27:29.468 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4276, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 32.417099927, "state": "up:active", "uptime": 41.238628168, "want_state": "up:active", "whoami": 4 } 2020-11-16T22:27:29.469 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok --format=json status 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":3,"id":4296,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":33.566824468,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.351498855999999} 2020-11-16T22:27:29.607 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4296, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 33.566824468, "state": "up:active", "uptime": 41.351498856, "want_state": "up:active", "whoami": 3 } 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":4299,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.711936627,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.486488104999999} 2020-11-16T22:27:29.742 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4299, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 34.711936627, "state": "up:active", "uptime": 41.486488105, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:27:29.871 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":4305,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.877620405000002,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.606391680000002} 2020-11-16T22:27:29.871 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4305, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 37.877620405, "state": "up:active", "uptime": 41.60639168, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:27:29.872 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:27:30.011 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":4315,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.994922338999999,"mdsmap_epoch":22,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":41.763757446} 2020-11-16T22:27:30.011 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4315, "mdsmap_epoch": 22, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 35.994922339, "state": "up:active", "uptime": 41.763757446, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:27:30.012 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:27:30.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:30.012 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-16T22:27:30.031 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:27:30.031 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:30.097 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:30.097 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-16T22:27:30.120 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:27:30.121 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-16T22:27:30.125 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:27:30.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:30.126 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-16T22:27:30.155 INFO:teuthology.run_tasks:Running task check-counter... 2020-11-16T22:27:30.166 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-16T22:27:30.184 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.329+0000 7f2ec2c0f700 1 Processor -- start 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.329+0000 7f2ec2c0f700 1 -- start start 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ebc161480 0x7f2ebc164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0c8db0 con 0x7f2ebc161860 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0c8330 con 0x7f2ebc164e10 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0c85d0 con 0x7f2ebc161480 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec140c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2ebc164e10 0x7f2ebc062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43354/0 (socket says 172.21.15.181:43354) 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec140c700 1 -- 172.21.15.181:0/2680280348 learned_addr learned my addr 172.21.15.181:0/2680280348 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec1c0d700 1 --2- 172.21.15.181:0/2680280348 >> v2:172.21.15.181:3300/0 conn(0x7f2ebc161480 0x7f2ebc164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940765854 0 0) 0x7f2ebc0c8330 con 0x7f2ebc164e10 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c003240 con 0x7f2ebc164e10 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165335758 0 0) 0x7f2ebc0c8db0 con 0x7f2ebc161860 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec1c0d700 1 -- 172.21.15.181:0/2680280348 >> v1:172.21.15.181:6789/0 conn(0x7f2ebc164e10 legacy=0x7f2ebc062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec1c0d700 1 -- 172.21.15.181:0/2680280348 >> v1:172.21.15.175:6789/0 conn(0x7f2ebc161860 legacy=0x7f2ebc0644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec1c0d700 1 -- 172.21.15.181:0/2680280348 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0c8870 con 0x7f2ebc161480 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec1c0d700 1 --2- 172.21.15.181:0/2680280348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc161480 0x7f2ebc164920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2eac009c10 tx=0x7f2eac0091d0).ready entity=mon.1 client_cookie=58c9b8e0e3ee864 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40014e0 con 0x7f2ebc161480 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac00b790 con 0x7f2ebc161480 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec0c0b700 1 -- 172.21.15.181:0/2680280348 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00bb10 con 0x7f2ebc161480 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/2680280348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc161480 msgr2=0x7f2ebc164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/2680280348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc161480 0x7f2ebc164920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2eac009c10 tx=0x7f2eac0091d0).stop 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/2680280348 shutdown_connections 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/2680280348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc161480 0x7f2ebc164920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/2680280348 >> 172.21.15.181:0/2680280348 conn(0x7f2ebc0d2dc0 msgr2=0x7f2ebc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/2680280348 shutdown_connections 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.333+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/2680280348 wait complete. 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 Processor -- start 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 -- start start 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc161860 0x7f2ebc146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ebc144d30 0x7f2ebc1734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc174490 con 0x7f2ebc144d30 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc173f50 con 0x7f2ebc1749e0 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec2c0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc1741f0 con 0x7f2ebc161860 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57290/0 (socket says 172.21.15.181:57290) 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 -- 172.21.15.181:0/1265889971 learned_addr learned my addr 172.21.15.181:0/1265889971 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec1c0d700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc161860 0x7f2ebc146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec240e700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ebc144d30 0x7f2ebc1734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 -- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc161860 msgr2=0x7f2ebc146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc161860 0x7f2ebc146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 -- 172.21.15.181:0/1265889971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ebc144d30 msgr2=0x7f2ebc1734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ebc144d30 0x7f2ebc1734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.337+0000 7f2ec140c700 1 -- 172.21.15.181:0/1265889971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc1461b0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2ec140c700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2ea400cc00 tx=0x7f2ea4001bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4007170 con 0x7f2ebc1749e0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40072d0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc146450 con 0x7f2ebc1749e0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400ab10 con 0x7f2ebc1749e0 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc148d70 con 0x7f2ebc1749e0 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc068fb0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2ea4010070 con 0x7f2ebc1749e0 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 0x7f2e98057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2ec1c0d700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 0x7f2e98057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.341+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ea406dd50 con 0x7f2ebc1749e0 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.345+0000 7f2ec1c0d700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 0x7f2e98057520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f2ebc045680 tx=0x7f2ebc062130).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.349+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ebc068fb0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.501+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ebc068fb0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2eb9ffb700 1 -- 172.21.15.181:0/1265889971 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f2ebc068fb0 con 0x7f2ebc1749e0 2020-11-16T22:27:30.490 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:30.491 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 msgr2=0x7f2e98057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 0x7f2e98057520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f2ebc045680 tx=0x7f2ebc062130).stop 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 msgr2=0x7f2ebc144840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2ea400cc00 tx=0x7f2ea4001bf0).stop 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 shutdown_connections 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e98055060 0x7f2e98057520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ebc144d30 0x7f2ebc1734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc161860 0x7f2ebc146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 --2- 172.21.15.181:0/1265889971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc1749e0 0x7f2ebc144840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 >> 172.21.15.181:0/1265889971 conn(0x7f2ebc0d2dc0 msgr2=0x7f2ebc164410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 shutdown_connections 2020-11-16T22:27:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.505+0000 7f2ec2c0f700 1 -- 172.21.15.181:0/1265889971 wait complete. 2020-11-16T22:27:30.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:30.506 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:30.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 Processor -- start 2020-11-16T22:27:30.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- start start 2020-11-16T22:27:30.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabb815e270 0x7fabb815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fabb8063b80 0x7fabb80667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb8064050 con 0x7fabb80b6c20 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb80642f0 con 0x7fabb8063b80 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb8064590 con 0x7fabb815e270 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabb815e270 0x7fabb815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabb815e270 0x7fabb815e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57296/0 (socket says 172.21.15.181:57296) 2020-11-16T22:27:30.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 -- 172.21.15.181:0/2839465344 learned_addr learned my addr 172.21.15.181:0/2839465344 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:30.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabaeffd700 1 --2- 172.21.15.181:0/2839465344 >> v2:172.21.15.175:3300/0 conn(0x7fabb8063b80 0x7fabb80667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 -- 172.21.15.181:0/2839465344 >> v2:172.21.15.175:3300/0 conn(0x7fabb8063b80 msgr2=0x7fabb80667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 --2- 172.21.15.181:0/2839465344 >> v2:172.21.15.175:3300/0 conn(0x7fabb8063b80 0x7fabb80667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 -- 172.21.15.181:0/2839465344 >> v1:172.21.15.175:6789/0 conn(0x7fabb80b6c20 legacy=0x7fabb8066ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 -- 172.21.15.181:0/2839465344 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8064830 con 0x7fabb815e270 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb77fe700 1 --2- 172.21.15.181:0/2839465344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb815e650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fabb80d46a0 tx=0x7faba8006eb0).ready entity=mon.1 client_cookie=ee0ca2d446fd5398 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb6ffd700 1 -- 172.21.15.181:0/2839465344 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80098d0 con 0x7fabb815e270 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb6ffd700 1 -- 172.21.15.181:0/2839465344 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8009a30 con 0x7fabb815e270 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabb6ffd700 1 -- 172.21.15.181:0/2839465344 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8009db0 con 0x7fabb815e270 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- 172.21.15.181:0/2839465344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 msgr2=0x7fabb815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 --2- 172.21.15.181:0/2839465344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb815e650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fabb80d46a0 tx=0x7faba8006eb0).stop 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- 172.21.15.181:0/2839465344 shutdown_connections 2020-11-16T22:27:30.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 --2- 172.21.15.181:0/2839465344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb815e650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 --2- 172.21.15.181:0/2839465344 >> v2:172.21.15.175:3300/0 conn(0x7fabb8063b80 0x7fabb80667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- 172.21.15.181:0/2839465344 >> 172.21.15.181:0/2839465344 conn(0x7fabb80d4790 msgr2=0x7fabb80d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:30.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- 172.21.15.181:0/2839465344 shutdown_connections 2020-11-16T22:27:30.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.649+0000 7fabbde08700 1 -- 172.21.15.181:0/2839465344 wait complete. 2020-11-16T22:27:30.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 Processor -- start 2020-11-16T22:27:30.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 -- start start 2020-11-16T22:27:30.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8063b80 0x7fabb816d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb816dac0 con 0x7fabb8063b80 2020-11-16T22:27:30.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb816dd60 con 0x7fabb815e270 2020-11-16T22:27:30.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabbde08700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb816e000 con 0x7fabb80b6c20 2020-11-16T22:27:30.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabb7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabaeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8063b80 0x7fabb816d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabb7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57304/0 (socket says 172.21.15.181:57304) 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabaeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36916/0 (socket says 172.21.15.181:36916) 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.653+0000 7fabb7fff700 1 -- 172.21.15.181:0/2680972734 learned_addr learned my addr 172.21.15.181:0/2680972734 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 -- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 msgr2=0x7fabb80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 -- 172.21.15.181:0/2680972734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8063b80 msgr2=0x7fabb816d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8063b80 0x7fabb816d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 -- 172.21.15.181:0/2680972734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80b7d70 con 0x7fabb815e270 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaeffd700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb7fff700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fabb000ca90 tx=0x7fabb0007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000ce70 con 0x7fabb815e270 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb80b8010 con 0x7fabb815e270 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb000a730 con 0x7fabb815e270 2020-11-16T22:27:30.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000aab0 con 0x7fabb815e270 2020-11-16T22:27:30.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb816d5e0 con 0x7fabb815e270 2020-11-16T22:27:30.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb816d8c0 con 0x7fabb815e270 2020-11-16T22:27:30.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fabb0010070 con 0x7fabb815e270 2020-11-16T22:27:30.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 0x7fab9405b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabb006dc10 con 0x7fabb815e270 2020-11-16T22:27:30.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb77fe700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 0x7fab9405b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.657+0000 7fabb77fe700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 0x7fab9405b960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7faba8006ef0 tx=0x7faba8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.665+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fabb0043d60 con 0x7fabb815e270 2020-11-16T22:27:30.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fabb8069eb0 con 0x7fabb815e270 2020-11-16T22:27:30.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabaffff700 1 -- 172.21.15.181:0/2680972734 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fabb0047380 con 0x7fabb815e270 2020-11-16T22:27:30.807 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 msgr2=0x7fab9405b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 0x7fab9405b960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7faba8006ef0 tx=0x7faba8015000).stop 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 msgr2=0x7fabb80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fabb000ca90 tx=0x7fabb0007e80).stop 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 shutdown_connections 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab940594a0 0x7fab9405b960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb815e270 0x7fabb80ce120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb8063b80 0x7fabb816d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 --2- 172.21.15.181:0/2680972734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabb80b6c20 0x7fabb80cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.821+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 >> 172.21.15.181:0/2680972734 conn(0x7fabb80d4790 msgr2=0x7fabb815b830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:30.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.825+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 shutdown_connections 2020-11-16T22:27:30.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.825+0000 7fabbde08700 1 -- 172.21.15.181:0/2680972734 wait complete. 2020-11-16T22:27:30.827 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 Processor -- start 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 -- start start 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd94815d510 0x7fd94815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd9480b7c10 0x7fd9480ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9480ceb10 con 0x7fd9480ce020 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480cedb0 con 0x7fd9480b7c10 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94e951700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480cf050 con 0x7fd94815d510 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd94815d510 0x7fd94815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd94815d510 0x7fd94815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57310/0 (socket says 172.21.15.181:57310) 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 -- 172.21.15.181:0/3462986169 learned_addr learned my addr 172.21.15.181:0/3462986169 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd94ceee700 1 --2- 172.21.15.181:0/3462986169 >> v2:172.21.15.175:3300/0 conn(0x7fd9480b7c10 0x7fd9480ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 -- 172.21.15.181:0/3462986169 >> v2:172.21.15.175:3300/0 conn(0x7fd9480b7c10 msgr2=0x7fd9480ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 --2- 172.21.15.181:0/3462986169 >> v2:172.21.15.175:3300/0 conn(0x7fd9480b7c10 0x7fd9480ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 -- 172.21.15.181:0/3462986169 >> v1:172.21.15.175:6789/0 conn(0x7fd9480ce020 legacy=0x7fd9480b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 -- 172.21.15.181:0/3462986169 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480cf2f0 con 0x7fd94815d510 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.969+0000 7fd947fff700 1 --2- 172.21.15.181:0/3462986169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd94815d8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd9480681b0 tx=0x7fd938006eb0).ready entity=mon.1 client_cookie=34279b1057ccfd11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd946ffd700 1 -- 172.21.15.181:0/3462986169 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380098d0 con 0x7fd94815d510 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd946ffd700 1 -- 172.21.15.181:0/3462986169 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd938009a30 con 0x7fd94815d510 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd946ffd700 1 -- 172.21.15.181:0/3462986169 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938009db0 con 0x7fd94815d510 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 -- 172.21.15.181:0/3462986169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 msgr2=0x7fd94815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 --2- 172.21.15.181:0/3462986169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd94815d8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd9480681b0 tx=0x7fd938006eb0).stop 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 -- 172.21.15.181:0/3462986169 shutdown_connections 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 --2- 172.21.15.181:0/3462986169 >> v2:172.21.15.175:3300/0 conn(0x7fd9480b7c10 0x7fd9480ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 --2- 172.21.15.181:0/3462986169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd94815d8f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 -- 172.21.15.181:0/3462986169 >> 172.21.15.181:0/3462986169 conn(0x7fd9480d3680 msgr2=0x7fd9480d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 -- 172.21.15.181:0/3462986169 shutdown_connections 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.973+0000 7fd94e951700 1 -- 172.21.15.181:0/3462986169 wait complete. 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 Processor -- start 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- start start 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9480ce020 0x7fd9480ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd9480cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd948160e40 con 0x7fd9480ce020 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94815ee90 con 0x7fd94815d510 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd948170b90 con 0x7fd9480b7c10 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd947fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd9480cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd947fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36926/0 (socket says 172.21.15.181:36926) 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd947fff700 1 -- 172.21.15.181:0/3705076529 learned_addr learned my addr 172.21.15.181:0/3705076529 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd9477fe700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9480ce020 0x7fd9480ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 -- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 msgr2=0x7fd948158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 -- 172.21.15.181:0/3705076529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9480ce020 msgr2=0x7fd9480ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9480ce020 0x7fd9480ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 -- 172.21.15.181:0/3705076529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948170e30 con 0x7fd94815d510 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd947fff700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94ceee700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd9480cd1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd94000eb40 tx=0x7fd940008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940017070 con 0x7fd94815d510 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9400095b0 con 0x7fd94815d510 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9481710d0 con 0x7fd94815d510 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940009930 con 0x7fd94815d510 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.977+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd948171610 con 0x7fd94815d510 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9481717d0 con 0x7fd94815d510 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd940009d20 con 0x7fd94815d510 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd927fff700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 0x7fd920057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd947fff700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 0x7fd920057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd94006d8b0 con 0x7fd94815d510 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.981+0000 7fd947fff700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 0x7fd920057520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd93800b1b0 tx=0x7fd938015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:30.989+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd940043b10 con 0x7fd94815d510 2020-11-16T22:27:31.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd948069eb0 con 0x7fd94815d510 2020-11-16T22:27:31.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd927fff700 1 -- 172.21.15.181:0/3705076529 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd940047130 con 0x7fd94815d510 2020-11-16T22:27:31.130 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 msgr2=0x7fd920057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 0x7fd920057520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd93800b1b0 tx=0x7fd938015000).stop 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 msgr2=0x7fd9480cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd9480cd1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd94000eb40 tx=0x7fd940008040).stop 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 shutdown_connections 2020-11-16T22:27:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd920055060 0x7fd920057520 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd94815d510 0x7fd9480cd1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9480b7c10 0x7fd948158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 --2- 172.21.15.181:0/3705076529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9480ce020 0x7fd9480ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 >> 172.21.15.181:0/3705076529 conn(0x7fd9480d3680 msgr2=0x7fd94815bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 shutdown_connections 2020-11-16T22:27:31.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.145+0000 7fd94e951700 1 -- 172.21.15.181:0/3705076529 wait complete. 2020-11-16T22:27:31.144 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:31.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 Processor -- start 2020-11-16T22:27:31.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 -- start start 2020-11-16T22:27:31.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e8061b50 con 0x7f67e8145a30 2020-11-16T22:27:31.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e80618b0 con 0x7f67e81492c0 2020-11-16T22:27:31.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67edc87700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8149700 con 0x7f67e80816e0 2020-11-16T22:27:31.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67ed486700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f67e81492c0 0x7f67e80cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43400/0 (socket says 172.21.15.181:43400) 2020-11-16T22:27:31.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67ed486700 1 -- 172.21.15.181:0/789261476 learned_addr learned my addr 172.21.15.181:0/789261476 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:31.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.289+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36934/0 (socket says 172.21.15.181:36934) 2020-11-16T22:27:31.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e7fff700 1 -- 172.21.15.181:0/789261476 >> v1:172.21.15.181:6789/0 conn(0x7f67e81492c0 legacy=0x7f67e80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e77fe700 1 -- 172.21.15.181:0/789261476 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448045464 0 0) 0x7f67e80618b0 con 0x7f67e81492c0 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e7fff700 1 -- 172.21.15.181:0/789261476 >> v1:172.21.15.175:6789/0 conn(0x7f67e8145a30 legacy=0x7f67e8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e7fff700 1 -- 172.21.15.181:0/789261476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80cb930 con 0x7f67e80816e0 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e7fff700 1 --2- 172.21.15.181:0/789261476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f67d800b9d0 tx=0x7f67d8006df0).ready entity=mon.2 client_cookie=2c28817cfdf915ba server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e77fe700 1 -- 172.21.15.181:0/789261476 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8009870 con 0x7f67e80816e0 2020-11-16T22:27:31.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e77fe700 1 -- 172.21.15.181:0/789261476 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d80099d0 con 0x7f67e80816e0 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67e77fe700 1 -- 172.21.15.181:0/789261476 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8009d50 con 0x7f67e80816e0 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- 172.21.15.181:0/789261476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 msgr2=0x7f67e8145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 --2- 172.21.15.181:0/789261476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f67d800b9d0 tx=0x7f67d8006df0).stop 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- 172.21.15.181:0/789261476 shutdown_connections 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 --2- 172.21.15.181:0/789261476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e80816e0 0x7f67e8145e10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- 172.21.15.181:0/789261476 >> 172.21.15.181:0/789261476 conn(0x7f67e80d68c0 msgr2=0x7f67e80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- 172.21.15.181:0/789261476 shutdown_connections 2020-11-16T22:27:31.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- 172.21.15.181:0/789261476 wait complete. 2020-11-16T22:27:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 Processor -- start 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.293+0000 7f67edc87700 1 -- start start 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e80816e0 0x7f67e8142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 0x7f67e80d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81700d0 con 0x7f67e80816e0 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8170370 con 0x7f67e81492c0 2020-11-16T22:27:31.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8141f00 con 0x7f67e8145a30 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ed486700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ed486700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57332/0 (socket says 172.21.15.181:57332) 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ed486700 1 -- 172.21.15.181:0/2867911910 learned_addr learned my addr 172.21.15.181:0/2867911910 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 0x7f67e80d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ecc85700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e80816e0 0x7f67e8142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 -- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 msgr2=0x7f67e8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 -- 172.21.15.181:0/2867911910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e80816e0 msgr2=0x7f67e8142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e80816e0 0x7f67e8142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 -- 172.21.15.181:0/2867911910 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e81421a0 con 0x7f67e8145a30 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ed486700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e7fff700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 0x7f67e80d47c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f67d8012000 tx=0x7f67d8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8011dd0 con 0x7f67e8145a30 2020-11-16T22:27:31.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d801e460 con 0x7f67e8145a30 2020-11-16T22:27:31.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d801e7e0 con 0x7f67e8145a30 2020-11-16T22:27:31.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e8142440 con 0x7f67e8145a30 2020-11-16T22:27:31.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8144b80 con 0x7f67e8145a30 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f67d801a040 con 0x7f67e8145a30 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e8044f60 con 0x7f67e8145a30 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 0x7f67c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67d806f1d0 con 0x7f67e8145a30 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.297+0000 7f67ecc85700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 0x7f67c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.301+0000 7f67ecc85700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 0x7f67c0057520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f67dc003300 tx=0x7f67dc007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.305+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f67d80454e0 con 0x7f67e8145a30 2020-11-16T22:27:31.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.453+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67e8068e30 con 0x7f67e8145a30 2020-11-16T22:27:31.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67e4ff9700 1 -- 172.21.15.181:0/2867911910 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f67d80230a0 con 0x7f67e8145a30 2020-11-16T22:27:31.441 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 msgr2=0x7f67c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 0x7f67c0057520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f67dc003300 tx=0x7f67dc007b90).stop 2020-11-16T22:27:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 msgr2=0x7f67e80d47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 0x7f67e80d47c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f67d8012000 tx=0x7f67d8015000).stop 2020-11-16T22:27:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 shutdown_connections 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67c0055060 0x7f67c0057520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e81492c0 0x7f67e8140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e80816e0 0x7f67e8142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 --2- 172.21.15.181:0/2867911910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e8145a30 0x7f67e80d47c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 >> 172.21.15.181:0/2867911910 conn(0x7f67e80d68c0 msgr2=0x7f67e8080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 shutdown_connections 2020-11-16T22:27:31.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.457+0000 7f67edc87700 1 -- 172.21.15.181:0/2867911910 wait complete. 2020-11-16T22:27:31.456 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 Processor -- start 2020-11-16T22:27:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 -- start start 2020-11-16T22:27:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f33480cf390 0x7f33480cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3348163970 0x7f3348163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33480600f0 0x7f3348060580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480cfc60 con 0x7f33480cf390 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3348065190 con 0x7f33480600f0 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f334cbe1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3348065430 con 0x7f3348163970 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f3347fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33480600f0 0x7f3348060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f3346ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3348163970 0x7f3348163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f3347fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33480600f0 0x7f3348060580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57342/0 (socket says 172.21.15.181:57342) 2020-11-16T22:27:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f3347fff700 1 -- 172.21.15.181:0/2543740570 learned_addr learned my addr 172.21.15.181:0/2543740570 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.601+0000 7f33477fe700 1 --2- 172.21.15.181:0/2543740570 >> v2:172.21.15.175:3300/0 conn(0x7f33480cf390 0x7f33480cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 -- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3348163970 msgr2=0x7f3348163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 --2- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3348163970 0x7f3348163df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 -- 172.21.15.181:0/2543740570 >> v2:172.21.15.175:3300/0 conn(0x7f33480cf390 msgr2=0x7f33480cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 --2- 172.21.15.181:0/2543740570 >> v2:172.21.15.175:3300/0 conn(0x7f33480cf390 0x7f33480cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 -- 172.21.15.181:0/2543740570 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480656d0 con 0x7f33480600f0 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f3347fff700 1 --2- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f3348060580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3348142f80 tx=0x7f3340009290).ready entity=mon.1 client_cookie=17e35068b9db5382 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f33467fc700 1 -- 172.21.15.181:0/2543740570 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000bbc0 con 0x7f33480600f0 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f33467fc700 1 -- 172.21.15.181:0/2543740570 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334000bd20 con 0x7f33480600f0 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f33467fc700 1 -- 172.21.15.181:0/2543740570 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340011610 con 0x7f33480600f0 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 -- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 msgr2=0x7f3348060580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f3348060580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3348142f80 tx=0x7f3340009290).stop 2020-11-16T22:27:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 -- 172.21.15.181:0/2543740570 shutdown_connections 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f3348060580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2543740570 >> v2:172.21.15.175:3300/0 conn(0x7f33480cf390 0x7f33480cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2543740570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3348163970 0x7f3348163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 -- 172.21.15.181:0/2543740570 >> 172.21.15.181:0/2543740570 conn(0x7f33480cfe90 msgr2=0x7f33480d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 -- 172.21.15.181:0/2543740570 shutdown_connections 2020-11-16T22:27:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.605+0000 7f334cbe1700 1 -- 172.21.15.181:0/2543740570 wait complete. 2020-11-16T22:27:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 Processor -- start 2020-11-16T22:27:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- start start 2020-11-16T22:27:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 0x7f3348066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 0x7f3348066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334816fd90 con 0x7f3348163970 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481674a0 con 0x7f33480600f0 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b6380 con 0x7f33480cf390 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f33477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 0x7f3348066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f33477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57344/0 (socket says 172.21.15.181:57344) 2020-11-16T22:27:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f33477fe700 1 -- 172.21.15.181:0/2999933652 learned_addr learned my addr 172.21.15.181:0/2999933652 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3347fff700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 0x7f3348066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 -- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 msgr2=0x7f334816f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 -- 172.21.15.181:0/2999933652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 msgr2=0x7f3348066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 0x7f3348066660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 -- 172.21.15.181:0/2999933652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480b6620 con 0x7f33480cf390 2020-11-16T22:27:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f33477fe700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3347fff700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 0x7f3348066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f3346ffd700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 0x7f3348066170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f333800eba0 tx=0x7f3338006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800b2c0 con 0x7f33480cf390 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800c8e0 con 0x7f33480cf390 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33480b68c0 con 0x7f33480cf390 2020-11-16T22:27:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800cc80 con 0x7f33480cf390 2020-11-16T22:27:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.609+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33480b70a0 con 0x7f33480cf390 2020-11-16T22:27:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33480b6a50 con 0x7f33480cf390 2020-11-16T22:27:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f333801c460 con 0x7f33480cf390 2020-11-16T22:27:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f333f7fe700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 0x7f3324057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f33477fe700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 0x7f3324057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f333806e060 con 0x7f33480cf390 2020-11-16T22:27:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.613+0000 7f33477fe700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 0x7f3324057520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f333000bb50 tx=0x7f3330008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.617+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3338044260 con 0x7f33480cf390 2020-11-16T22:27:31.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.773+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3348044f60 con 0x7f33480cf390 2020-11-16T22:27:31.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f333f7fe700 1 -- 172.21.15.181:0/2999933652 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3338047880 con 0x7f33480cf390 2020-11-16T22:27:31.761 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:31.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 msgr2=0x7f3324057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 0x7f3324057520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f333000bb50 tx=0x7f3330008040).stop 2020-11-16T22:27:31.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 msgr2=0x7f3348066170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 0x7f3348066170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f333800eba0 tx=0x7f3338006df0).stop 2020-11-16T22:27:31.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 shutdown_connections 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3324055060 0x7f3324057520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3348163970 0x7f3348066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33480600f0 0x7f334816f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 --2- 172.21.15.181:0/2999933652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33480cf390 0x7f3348066170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 >> 172.21.15.181:0/2999933652 conn(0x7f33480cfe90 msgr2=0x7f33480d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 shutdown_connections 2020-11-16T22:27:31.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.777+0000 7f334cbe1700 1 -- 172.21.15.181:0/2999933652 wait complete. 2020-11-16T22:27:31.775 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:31.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 Processor -- start 2020-11-16T22:27:31.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 -- start start 2020-11-16T22:27:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1815e2e0 con 0x7fdd1815de80 2020-11-16T22:27:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd180ce460 con 0x7fdd180b2190 2020-11-16T22:27:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1f215700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18066a60 con 0x7fdd18160690 2020-11-16T22:27:31.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1e213700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdd180b2190 0x7fdd180cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43424/0 (socket says 172.21.15.181:43424) 2020-11-16T22:27:31.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1e213700 1 -- 172.21.15.181:0/909697494 learned_addr learned my addr 172.21.15.181:0/909697494 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:31.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1ea14700 1 --2- 172.21.15.181:0/909697494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1d211700 1 -- 172.21.15.181:0/909697494 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303235403 0 0) 0x7fdd180ce460 con 0x7fdd180b2190 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1ea14700 1 -- 172.21.15.181:0/909697494 >> v1:172.21.15.181:6789/0 conn(0x7fdd180b2190 legacy=0x7fdd180cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1ea14700 1 -- 172.21.15.181:0/909697494 >> v1:172.21.15.175:6789/0 conn(0x7fdd1815de80 legacy=0x7fdd1815ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1ea14700 1 -- 172.21.15.181:0/909697494 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180ce700 con 0x7fdd18160690 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1ea14700 1 --2- 172.21.15.181:0/909697494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18066570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fdd08009cc0 tx=0x7fdd080091d0).ready entity=mon.2 client_cookie=51bdc9663eed0593 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1d211700 1 -- 172.21.15.181:0/909697494 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800bbc0 con 0x7fdd18160690 2020-11-16T22:27:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1d211700 1 -- 172.21.15.181:0/909697494 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0800bd20 con 0x7fdd18160690 2020-11-16T22:27:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.909+0000 7fdd1d211700 1 -- 172.21.15.181:0/909697494 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08011610 con 0x7fdd18160690 2020-11-16T22:27:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- 172.21.15.181:0/909697494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 msgr2=0x7fdd18066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 --2- 172.21.15.181:0/909697494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18066570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fdd08009cc0 tx=0x7fdd080091d0).stop 2020-11-16T22:27:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- 172.21.15.181:0/909697494 shutdown_connections 2020-11-16T22:27:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 --2- 172.21.15.181:0/909697494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18066570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- 172.21.15.181:0/909697494 >> 172.21.15.181:0/909697494 conn(0x7fdd180cfe90 msgr2=0x7fdd180d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- 172.21.15.181:0/909697494 shutdown_connections 2020-11-16T22:27:31.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- 172.21.15.181:0/909697494 wait complete. 2020-11-16T22:27:31.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 Processor -- start 2020-11-16T22:27:31.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.913+0000 7fdd1f215700 1 -- start start 2020-11-16T22:27:31.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd180b2190 0x7fdd180b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1815de80 0x7fdd1814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b6f50 con 0x7fdd180b2190 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b7510 con 0x7fdd1815de80 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b77b0 con 0x7fdd18160690 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36972/0 (socket says 172.21.15.181:36972) 2020-11-16T22:27:31.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 -- 172.21.15.181:0/1117233476 learned_addr learned my addr 172.21.15.181:0/1117233476 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1da12700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1815de80 0x7fdd1814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1e213700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd180b2190 0x7fdd180b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 -- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1815de80 msgr2=0x7fdd1814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1815de80 0x7fdd1814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 -- 172.21.15.181:0/1117233476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd180b2190 msgr2=0x7fdd180b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd180b2190 0x7fdd180b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:31.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 -- 172.21.15.181:0/1117233476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180b7a50 con 0x7fdd18160690 2020-11-16T22:27:31.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1ea14700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fdd1815e470 tx=0x7fdd08016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08011e30 con 0x7fdd18160690 2020-11-16T22:27:31.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd08010aa0 con 0x7fdd18160690 2020-11-16T22:27:31.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08010e60 con 0x7fdd18160690 2020-11-16T22:27:31.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd180b7cf0 con 0x7fdd18160690 2020-11-16T22:27:31.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd180b6b80 con 0x7fdd18160690 2020-11-16T22:27:31.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.917+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd180b6da0 con 0x7fdd18160690 2020-11-16T22:27:31.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.921+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd18160690 2020-11-16T22:27:31.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.921+0000 7fdd167fc700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 0x7fdcf4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:31.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.921+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd08070e80 con 0x7fdd18160690 2020-11-16T22:27:31.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.921+0000 7fdd1e213700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 0x7fdcf4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:31.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.921+0000 7fdd1e213700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 0x7fdcf4057520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fdd00000bc0 tx=0x7fdd00007940).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:31.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:31.925+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdd0804aa80 con 0x7fdd18160690 2020-11-16T22:27:32.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.081+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd180b7e80 con 0x7fdd18160690 2020-11-16T22:27:32.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.081+0000 7fdd167fc700 1 -- 172.21.15.181:0/1117233476 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fdd08015030 con 0x7fdd18160690 2020-11-16T22:27:32.068 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:32.069 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 msgr2=0x7fdcf4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 0x7fdcf4057520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fdd00000bc0 tx=0x7fdd00007940).stop 2020-11-16T22:27:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 msgr2=0x7fdd18149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fdd1815e470 tx=0x7fdd08016000).stop 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 shutdown_connections 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdcf4055060 0x7fdcf4057520 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd18160690 0x7fdd18149f60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd180b2190 0x7fdd180b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 --2- 172.21.15.181:0/1117233476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1815de80 0x7fdd1814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 >> 172.21.15.181:0/1117233476 conn(0x7fdd180cfe90 msgr2=0x7fdd180d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 shutdown_connections 2020-11-16T22:27:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.085+0000 7fdd1f215700 1 -- 172.21.15.181:0/1117233476 wait complete. 2020-11-16T22:27:32.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:32.084 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:32.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 Processor -- start 2020-11-16T22:27:32.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 -- start start 2020-11-16T22:27:32.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f427813e870 0x7f427813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4278061100 0x7f4278061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4278160730 con 0x7f4278061100 2020-11-16T22:27:32.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427815d180 con 0x7f427813e870 2020-11-16T22:27:32.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427813f140 con 0x7f427815fdc0 2020-11-16T22:27:32.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f4275d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.225+0000 7f427659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f427813e870 0x7f427813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36978/0 (socket says 172.21.15.181:36978) 2020-11-16T22:27:32.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 -- 172.21.15.181:0/167774296 learned_addr learned my addr 172.21.15.181:0/167774296 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:32.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4276d9d700 1 --2- 172.21.15.181:0/167774296 >> v2:172.21.15.175:3300/0 conn(0x7f4278061100 0x7f4278061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 -- 172.21.15.181:0/167774296 >> v2:172.21.15.181:3300/0 conn(0x7f427813e870 msgr2=0x7f427813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 --2- 172.21.15.181:0/167774296 >> v2:172.21.15.181:3300/0 conn(0x7f427813e870 0x7f427813ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 -- 172.21.15.181:0/167774296 >> v2:172.21.15.175:3300/0 conn(0x7f4278061100 msgr2=0x7f4278061590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 --2- 172.21.15.181:0/167774296 >> v2:172.21.15.175:3300/0 conn(0x7f4278061100 0x7f4278061590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 -- 172.21.15.181:0/167774296 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427815d810 con 0x7f427815fdc0 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f4275d9b700 1 --2- 172.21.15.181:0/167774296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f42780675c0 tx=0x7f426c006eb0).ready entity=mon.2 client_cookie=ce524a213fbbb4a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427559a700 1 -- 172.21.15.181:0/167774296 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c009720 con 0x7f427815fdc0 2020-11-16T22:27:32.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427559a700 1 -- 172.21.15.181:0/167774296 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c009880 con 0x7f427815fdc0 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427559a700 1 -- 172.21.15.181:0/167774296 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c009c00 con 0x7f427815fdc0 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 -- 172.21.15.181:0/167774296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 msgr2=0x7f4278160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 --2- 172.21.15.181:0/167774296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f42780675c0 tx=0x7f426c006eb0).stop 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 -- 172.21.15.181:0/167774296 shutdown_connections 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 --2- 172.21.15.181:0/167774296 >> v2:172.21.15.175:3300/0 conn(0x7f4278061100 0x7f4278061590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 --2- 172.21.15.181:0/167774296 >> v2:172.21.15.181:3300/0 conn(0x7f427813e870 0x7f427813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 --2- 172.21.15.181:0/167774296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f4278160240 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 -- 172.21.15.181:0/167774296 >> 172.21.15.181:0/167774296 conn(0x7f42780cf630 msgr2=0x7f42780cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 -- 172.21.15.181:0/167774296 shutdown_connections 2020-11-16T22:27:32.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.229+0000 7f427759e700 1 -- 172.21.15.181:0/167774296 wait complete. 2020-11-16T22:27:32.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 Processor -- start 2020-11-16T22:27:32.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 -- start start 2020-11-16T22:27:32.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4278061100 0x7f42780b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 0x7f427815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427816e1d0 con 0x7f4278061100 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427816d750 con 0x7f427813e870 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.233+0000 7f427759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427816d9f0 con 0x7f427815fdc0 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4275d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 0x7f427815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:36986/0 (socket says 172.21.15.181:36986) 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f427659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4278061100 0x7f42780b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4275d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 0x7f427815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57374/0 (socket says 172.21.15.181:57374) 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 -- 172.21.15.181:0/969573722 learned_addr learned my addr 172.21.15.181:0/969573722 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 -- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 msgr2=0x7f427815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 0x7f427815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 -- 172.21.15.181:0/969573722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4278061100 msgr2=0x7f42780b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4278061100 0x7f42780b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 -- 172.21.15.181:0/969573722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427816dc90 con 0x7f427815fdc0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f4276d9d700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f427000e970 tx=0x7f4270006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000b6a0 con 0x7f427815fdc0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427000b800 con 0x7f427815fdc0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427816df30 con 0x7f427815fdc0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000bba0 con 0x7f427815fdc0 2020-11-16T22:27:32.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427816d260 con 0x7f427815fdc0 2020-11-16T22:27:32.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f427001c460 con 0x7f427815fdc0 2020-11-16T22:27:32.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.237+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4278044f60 con 0x7f427815fdc0 2020-11-16T22:27:32.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.241+0000 7f42667fc700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 0x7f424c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.241+0000 7f427659c700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 0x7f424c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.241+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f427006df80 con 0x7f427815fdc0 2020-11-16T22:27:32.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.241+0000 7f427659c700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 0x7f424c057520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f42680099f0 tx=0x7f42680091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.245+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4270044180 con 0x7f427815fdc0 2020-11-16T22:27:32.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f427805c310 con 0x7f427815fdc0 2020-11-16T22:27:32.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f42667fc700 1 -- 172.21.15.181:0/969573722 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f42700477a0 con 0x7f427815fdc0 2020-11-16T22:27:32.387 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:32.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 msgr2=0x7f424c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 0x7f424c057520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f42680099f0 tx=0x7f42680091d0).stop 2020-11-16T22:27:32.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 msgr2=0x7f42780ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f427000e970 tx=0x7f4270006df0).stop 2020-11-16T22:27:32.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.401+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 shutdown_connections 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f424c055060 0x7f424c057520 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427815fdc0 0x7f42780ca0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4278061100 0x7f42780b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 --2- 172.21.15.181:0/969573722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f427813e870 0x7f427815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 >> 172.21.15.181:0/969573722 conn(0x7f42780cf630 msgr2=0x7f42780d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 shutdown_connections 2020-11-16T22:27:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.405+0000 7f427759e700 1 -- 172.21.15.181:0/969573722 wait complete. 2020-11-16T22:27:32.402 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:32.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 Processor -- start 2020-11-16T22:27:32.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 -- start start 2020-11-16T22:27:32.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fccbc0615f0 0x7fccbc0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccbc0622a0 0x7fccbc062720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccbc160e40 con 0x7fccbc0615f0 2020-11-16T22:27:32.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccbc160ba0 con 0x7fccbc0622a0 2020-11-16T22:27:32.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc2703700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc060970 con 0x7fccbc15f5b0 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccbc0622a0 0x7fccbc062720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccbc0622a0 0x7fccbc062720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57382/0 (socket says 172.21.15.181:57382) 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 -- 172.21.15.181:0/4186083554 learned_addr learned my addr 172.21.15.181:0/4186083554 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccbbfff700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccbb7fe700 1 --2- 172.21.15.181:0/4186083554 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0615f0 0x7fccbc0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 -- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 msgr2=0x7fccbc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 -- 172.21.15.181:0/4186083554 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0615f0 msgr2=0x7fccbc0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 --2- 172.21.15.181:0/4186083554 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0615f0 0x7fccbc0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 -- 172.21.15.181:0/4186083554 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc060c10 con 0x7fccbc0622a0 2020-11-16T22:27:32.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccc0ca0700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0622a0 0x7fccbc062720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fccbc0d7a10 tx=0x7fccb4009290).ready entity=mon.1 client_cookie=6098f47b315347e2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccbaffd700 1 -- 172.21.15.181:0/4186083554 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb400bbc0 con 0x7fccbc0622a0 2020-11-16T22:27:32.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccbaffd700 1 -- 172.21.15.181:0/4186083554 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb400bd20 con 0x7fccbc0622a0 2020-11-16T22:27:32.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.545+0000 7fccbaffd700 1 -- 172.21.15.181:0/4186083554 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4011610 con 0x7fccbc0622a0 2020-11-16T22:27:32.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0622a0 msgr2=0x7fccbc062720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0622a0 0x7fccbc062720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fccbc0d7a10 tx=0x7fccb4009290).stop 2020-11-16T22:27:32.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/4186083554 shutdown_connections 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0622a0 0x7fccbc062720 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- 172.21.15.181:0/4186083554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- 172.21.15.181:0/4186083554 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0615f0 0x7fccbc0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/4186083554 >> 172.21.15.181:0/4186083554 conn(0x7fccbc0d69a0 msgr2=0x7fccbc0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/4186083554 shutdown_connections 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/4186083554 wait complete. 2020-11-16T22:27:32.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 Processor -- start 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- start start 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0622a0 0x7fccbc0bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc172930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc173290 con 0x7fccbc0622a0 2020-11-16T22:27:32.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc173530 con 0x7fccbc0615f0 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc1737d0 con 0x7fccbc15f5b0 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc0ca0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc172930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57388/0 (socket says 172.21.15.181:57388) 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 -- 172.21.15.181:0/475495705 learned_addr learned my addr 172.21.15.181:0/475495705 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbb7fe700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0622a0 0x7fccbc0bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 -- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 msgr2=0x7fccbc172930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc172930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 -- 172.21.15.181:0/475495705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0622a0 msgr2=0x7fccbc0bb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0622a0 0x7fccbc0bb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 -- 172.21.15.181:0/475495705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc163210 con 0x7fccbc0615f0 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccbbfff700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fccac00eba0 tx=0x7fccac008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00c970 con 0x7fccbc0615f0 2020-11-16T22:27:32.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac00cad0 con 0x7fccbc0615f0 2020-11-16T22:27:32.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccbc1634b0 con 0x7fccbc0615f0 2020-11-16T22:27:32.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00ce70 con 0x7fccbc0615f0 2020-11-16T22:27:32.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.549+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccbc164160 con 0x7fccbc0615f0 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccbc069eb0 con 0x7fccbc0615f0 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fccac012600 con 0x7fccbc0615f0 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fcc9bfff700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 0x7fcc94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fccac06e250 con 0x7fccbc0615f0 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fccbb7fe700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 0x7fcc94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.553+0000 7fccbb7fe700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 0x7fcc94057520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fccb0006fd0 tx=0x7fccb0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.561+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fccac0443a0 con 0x7fccbc0615f0 2020-11-16T22:27:32.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.713+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fccbc1642f0 con 0x7fccbc0615f0 2020-11-16T22:27:32.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.713+0000 7fcc9bfff700 1 -- 172.21.15.181:0/475495705 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fccac016020 con 0x7fccbc0615f0 2020-11-16T22:27:32.700 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:32.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 msgr2=0x7fcc94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 0x7fcc94057520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fccb0006fd0 tx=0x7fccb0008040).stop 2020-11-16T22:27:32.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 msgr2=0x7fccbc163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fccac00eba0 tx=0x7fccac008040).stop 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 shutdown_connections 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcc94055060 0x7fcc94057520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15f5b0 0x7fccbc172930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc0615f0 0x7fccbc163bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 --2- 172.21.15.181:0/475495705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0622a0 0x7fccbc0bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 >> 172.21.15.181:0/475495705 conn(0x7fccbc0d69a0 msgr2=0x7fccbc062680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 shutdown_connections 2020-11-16T22:27:32.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.717+0000 7fccc2703700 1 -- 172.21.15.181:0/475495705 wait complete. 2020-11-16T22:27:32.714 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:32.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.849+0000 7fbb22559700 1 Processor -- start 2020-11-16T22:27:32.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.849+0000 7fbb22559700 1 -- start start 2020-11-16T22:27:32.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb22559700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb1c148d30 0x7fbb1c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb22559700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0ce340 con 0x7fbb1c060800 2020-11-16T22:27:32.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb22559700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0ce5e0 con 0x7fbb1c149110 2020-11-16T22:27:32.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb22559700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c160de0 con 0x7fbb1c148d30 2020-11-16T22:27:32.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb20d56700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbb1c149110 0x7fbb1c0600f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43466/0 (socket says 172.21.15.181:43466) 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb20d56700 1 -- 172.21.15.181:0/3447343290 learned_addr learned my addr 172.21.15.181:0/3447343290 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21557700 1 --2- 172.21.15.181:0/3447343290 >> v2:172.21.15.175:3300/0 conn(0x7fbb1c148d30 0x7fbb1c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547182081 0 0) 0x7fbb1c0ce5e0 con 0x7fbb1c149110 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb00003240 con 0x7fbb1c149110 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118821831 0 0) 0x7fbb1c0ce340 con 0x7fbb1c060800 2020-11-16T22:27:32.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1c0ce5e0 con 0x7fbb1c060800 2020-11-16T22:27:32.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3701199233 0 0) 0x7fbb00003240 con 0x7fbb1c149110 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 >> v2:172.21.15.175:3300/0 conn(0x7fbb1c148d30 msgr2=0x7fbb1c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 --2- 172.21.15.181:0/3447343290 >> v2:172.21.15.175:3300/0 conn(0x7fbb1c148d30 0x7fbb1c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 >> v1:172.21.15.175:6789/0 conn(0x7fbb1c060800 legacy=0x7fbb1c0c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0ce880 con 0x7fbb1c149110 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7fbb0c0014e0 con 0x7fbb1c149110 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 >> v1:172.21.15.181:6789/0 conn(0x7fbb1c149110 legacy=0x7fbb1c0600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000038c0 0x7fbb00003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb000044e0 0x7fbb00004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 0x7fbb0000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21557700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000038c0 0x7fbb00003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00005380 con 0x7fbb000044e0 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000bc20 con 0x7fbb000038c0 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000bde0 con 0x7fbb00004e50 2020-11-16T22:27:32.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 0x7fbb0000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb20d56700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb000044e0 0x7fbb00004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 -- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000038c0 msgr2=0x7fbb00003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000038c0 0x7fbb00003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 -- 172.21.15.181:0/3447343290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb000044e0 msgr2=0x7fbb00004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb000044e0 0x7fbb00004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 -- 172.21.15.181:0/3447343290 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00003240 con 0x7fbb00004e50 2020-11-16T22:27:32.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 -- 172.21.15.181:0/3447343290 reap_dead start 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.853+0000 7fbb21d58700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 0x7fbb0000b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fbb14001fa0 tx=0x7fbb14007e80).ready entity=mon.2 client_cookie=e1b72641f7884d18 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400cb10 con 0x7fbb00004e50 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400cc70 con 0x7fbb00004e50 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb13fff700 1 -- 172.21.15.181:0/3447343290 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14006870 con 0x7fbb00004e50 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 -- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 msgr2=0x7fbb0000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 0x7fbb0000b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fbb14001fa0 tx=0x7fbb14007e80).stop 2020-11-16T22:27:32.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 -- 172.21.15.181:0/3447343290 shutdown_connections 2020-11-16T22:27:32.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 --2- 172.21.15.181:0/3447343290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb00004e50 0x7fbb0000b730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 -- 172.21.15.181:0/3447343290 >> 172.21.15.181:0/3447343290 conn(0x7fbb1c0d33f0 msgr2=0x7fbb1c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:32.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 -- 172.21.15.181:0/3447343290 shutdown_connections 2020-11-16T22:27:32.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.857+0000 7fbb22559700 1 -- 172.21.15.181:0/3447343290 wait complete. 2020-11-16T22:27:32.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 Processor -- start 2020-11-16T22:27:32.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 -- start start 2020-11-16T22:27:32.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 0x7fbb1c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cf920 con 0x7fbb00004e50 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c146b20 con 0x7fbb1c146660 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb22559700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c146dc0 con 0x7fbb1c0cf4e0 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb21d58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb20d56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb20d56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57408/0 (socket says 172.21.15.181:57408) 2020-11-16T22:27:32.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb21d58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37020/0 (socket says 172.21.15.181:37020) 2020-11-16T22:27:32.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb20d56700 1 -- 172.21.15.181:0/146240669 learned_addr learned my addr 172.21.15.181:0/146240669 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:32.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb21557700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 0x7fbb1c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.861+0000 7fbb20d56700 1 -- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 msgr2=0x7fbb1c15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb20d56700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb20d56700 1 -- 172.21.15.181:0/146240669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 msgr2=0x7fbb1c146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:32.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb20d56700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 0x7fbb1c146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb20d56700 1 -- 172.21.15.181:0/146240669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c147060 con 0x7fbb1c146660 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb21d58700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb21557700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 0x7fbb1c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb20d56700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fbb0c0019c0 tx=0x7fbb1c0b5890).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00f690 con 0x7fbb1c146660 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00f7f0 con 0x7fbb1c146660 2020-11-16T22:27:32.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c147300 con 0x7fbb1c146660 2020-11-16T22:27:32.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00fb70 con 0x7fbb1c146660 2020-11-16T22:27:32.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c15efd0 con 0x7fbb1c146660 2020-11-16T22:27:32.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c044f60 con 0x7fbb1c146660 2020-11-16T22:27:32.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb0c01b030 con 0x7fbb1c146660 2020-11-16T22:27:32.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 0x7fbaf4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:32.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb21557700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 0x7fbaf4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:32.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb0c06d4c0 con 0x7fbb1c146660 2020-11-16T22:27:32.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.865+0000 7fbb21557700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 0x7fbaf4057520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fbb18006d00 tx=0x7fbb18006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:32.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:32.873+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbb0c0437d0 con 0x7fbb1c146660 2020-11-16T22:27:33.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb1c0b63d0 con 0x7fbb1c146660 2020-11-16T22:27:33.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb117fa700 1 -- 172.21.15.181:0/146240669 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbb0c019030 con 0x7fbb1c146660 2020-11-16T22:27:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:33.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 msgr2=0x7fbaf4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 0x7fbaf4057520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fbb18006d00 tx=0x7fbb18006df0).stop 2020-11-16T22:27:33.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 msgr2=0x7fbb1c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.025+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fbb0c0019c0 tx=0x7fbb1c0b5890).stop 2020-11-16T22:27:33.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 shutdown_connections 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbaf4055060 0x7fbaf4057520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb1c0cf4e0 0x7fbb1c15eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb00004e50 0x7fbb1c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 --2- 172.21.15.181:0/146240669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c146660 0x7fbb1c0ceff0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 >> 172.21.15.181:0/146240669 conn(0x7fbb1c0d33f0 msgr2=0x7fbb1c0600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 shutdown_connections 2020-11-16T22:27:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.029+0000 7fbb22559700 1 -- 172.21.15.181:0/146240669 wait complete. 2020-11-16T22:27:33.025 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:33.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 Processor -- start 2020-11-16T22:27:33.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 -- start start 2020-11-16T22:27:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f20b80816e0 0x7f20b80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b8061b50 con 0x7f20b81492c0 2020-11-16T22:27:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b80618b0 con 0x7f20b80816e0 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bda92700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80cb930 con 0x7f20b8145a30 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20b7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f20b81492c0 0x7f20b8080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43486/0 (socket says 172.21.15.181:43486) 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20b7fff700 1 -- 172.21.15.181:0/3876552858 learned_addr learned my addr 172.21.15.181:0/3876552858 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bca90700 1 --2- 172.21.15.181:0/3876552858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20bd291700 1 --2- 172.21.15.181:0/3876552858 >> v2:172.21.15.175:3300/0 conn(0x7f20b80816e0 0x7f20b80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371464863 0 0) 0x7f20b8061b50 con 0x7f20b81492c0 2020-11-16T22:27:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.169+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c003240 con 0x7f20b81492c0 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3415208201 0 0) 0x7f209c003240 con 0x7f20b81492c0 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bca90700 1 -- 172.21.15.181:0/3876552858 >> v2:172.21.15.175:3300/0 conn(0x7f20b80816e0 msgr2=0x7f20b80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bca90700 1 --2- 172.21.15.181:0/3876552858 >> v2:172.21.15.175:3300/0 conn(0x7f20b80816e0 0x7f20b80cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bca90700 1 -- 172.21.15.181:0/3876552858 >> v1:172.21.15.181:6789/0 conn(0x7f20b81492c0 legacy=0x7f20b8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bca90700 1 -- 172.21.15.181:0/3876552858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80cbbd0 con 0x7f20b8145a30 2020-11-16T22:27:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bca90700 1 --2- 172.21.15.181:0/3876552858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b8145e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f20a400b9a0 tx=0x7f20a4006df0).ready entity=mon.2 client_cookie=15537eece4164f09 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0014e0 con 0x7f20b8145a30 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4011400 con 0x7f20b8145a30 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20b77fe700 1 -- 172.21.15.181:0/3876552858 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4011780 con 0x7f20b8145a30 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 -- 172.21.15.181:0/3876552858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 msgr2=0x7f20b8145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 --2- 172.21.15.181:0/3876552858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b8145e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f20a400b9a0 tx=0x7f20a4006df0).stop 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 -- 172.21.15.181:0/3876552858 shutdown_connections 2020-11-16T22:27:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 --2- 172.21.15.181:0/3876552858 >> v2:172.21.15.175:3300/0 conn(0x7f20b80816e0 0x7f20b80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 --2- 172.21.15.181:0/3876552858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b8145e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 -- 172.21.15.181:0/3876552858 >> 172.21.15.181:0/3876552858 conn(0x7f20b80d68c0 msgr2=0x7f20b80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 -- 172.21.15.181:0/3876552858 shutdown_connections 2020-11-16T22:27:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.173+0000 7f20bda92700 1 -- 172.21.15.181:0/3876552858 wait complete. 2020-11-16T22:27:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 Processor -- start 2020-11-16T22:27:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- start start 2020-11-16T22:27:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 0x7f20b813da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 0x7f20b80d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8170170 con 0x7f20b81492c0 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8170410 con 0x7f20b80816e0 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80d4e10 con 0x7f20b8145a30 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37034/0 (socket says 172.21.15.181:37034) 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bca90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 0x7f20b813da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 -- 172.21.15.181:0/2165417129 learned_addr learned my addr 172.21.15.181:0/2165417129 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bd291700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 0x7f20b80d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 -- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 msgr2=0x7f20b813da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 0x7f20b813da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 -- 172.21.15.181:0/2165417129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 msgr2=0x7f20b80d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 0x7f20b80d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 -- 172.21.15.181:0/2165417129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80d50b0 con 0x7f20b8145a30 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bd291700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 0x7f20b80d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bca90700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 0x7f20b813da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:33.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b7fff700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f20ac00ca90 tx=0x7f20ac0011f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac007320 con 0x7f20b8145a30 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b80d5350 con 0x7f20b8145a30 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac007480 con 0x7f20b8145a30 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac005800 con 0x7f20b8145a30 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.177+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b80baf60 con 0x7f20b8145a30 2020-11-16T22:27:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b80cd510 con 0x7f20b8145a30 2020-11-16T22:27:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f20ac010030 con 0x7f20b8145a30 2020-11-16T22:27:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20b4ff9700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 0x7f2090057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20ac06dde0 con 0x7f20b8145a30 2020-11-16T22:27:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20bca90700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 0x7f2090057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.181+0000 7f20bca90700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 0x7f2090057520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f20a4006e30 tx=0x7f20a4013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.193+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f20b80cd510 con 0x7f20b8145a30 2020-11-16T22:27:33.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.337+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20b80cd510 con 0x7f20b8145a30 2020-11-16T22:27:33.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.337+0000 7f20b4ff9700 1 -- 172.21.15.181:0/2165417129 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f20b80cd510 con 0x7f20b8145a30 2020-11-16T22:27:33.323 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.337+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 msgr2=0x7f2090057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 0x7f2090057520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f20a4006e30 tx=0x7f20a4013000).stop 2020-11-16T22:27:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 msgr2=0x7f20b813df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f20ac00ca90 tx=0x7f20ac0011f0).stop 2020-11-16T22:27:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 shutdown_connections 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2090055060 0x7f2090057520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b81492c0 0x7f20b80d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b80816e0 0x7f20b813da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 --2- 172.21.15.181:0/2165417129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b8145a30 0x7f20b813df80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 >> 172.21.15.181:0/2165417129 conn(0x7f20b80d68c0 msgr2=0x7f20b8080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 shutdown_connections 2020-11-16T22:27:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.341+0000 7f20bda92700 1 -- 172.21.15.181:0/2165417129 wait complete. 2020-11-16T22:27:33.337 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:33.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 Processor -- start 2020-11-16T22:27:33.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 -- start start 2020-11-16T22:27:33.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40c8db0 con 0x7f67e4164e10 2020-11-16T22:27:33.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40c8330 con 0x7f67e4161860 2020-11-16T22:27:33.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e9192700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40c85d0 con 0x7f67e4161480 2020-11-16T22:27:33.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e8991700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f67e4161860 0x7f67e40644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43500/0 (socket says 172.21.15.181:43500) 2020-11-16T22:27:33.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.477+0000 7f67e8991700 1 -- 172.21.15.181:0/3739112942 learned_addr learned my addr 172.21.15.181:0/3739112942 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e3fff700 1 --2- 172.21.15.181:0/3739112942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780566750 0 0) 0x7f67e40c8330 con 0x7f67e4161860 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c8003240 con 0x7f67e4161860 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737339912 0 0) 0x7f67e40c8db0 con 0x7f67e4164e10 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e40c8330 con 0x7f67e4164e10 2020-11-16T22:27:33.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e3fff700 1 -- 172.21.15.181:0/3739112942 >> v1:172.21.15.181:6789/0 conn(0x7f67e4161860 legacy=0x7f67e40644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e3fff700 1 -- 172.21.15.181:0/3739112942 >> v1:172.21.15.175:6789/0 conn(0x7f67e4164e10 legacy=0x7f67e4062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e3fff700 1 -- 172.21.15.181:0/3739112942 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40c8870 con 0x7f67e4161480 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e3fff700 1 --2- 172.21.15.181:0/3739112942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4164920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f67d400b9d0 tx=0x7f67d4006df0).ready entity=mon.2 client_cookie=43bab49a9e64903b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4009680 con 0x7f67e4161480 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d40097e0 con 0x7f67e4161480 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3739112942 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4009b60 con 0x7f67e4161480 2020-11-16T22:27:33.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 -- 172.21.15.181:0/3739112942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 msgr2=0x7f67e4164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 --2- 172.21.15.181:0/3739112942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4164920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f67d400b9d0 tx=0x7f67d4006df0).stop 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 -- 172.21.15.181:0/3739112942 shutdown_connections 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 --2- 172.21.15.181:0/3739112942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4164920 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 -- 172.21.15.181:0/3739112942 >> 172.21.15.181:0/3739112942 conn(0x7f67e40d2dc0 msgr2=0x7f67e40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 -- 172.21.15.181:0/3739112942 shutdown_connections 2020-11-16T22:27:33.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.481+0000 7f67e9192700 1 -- 172.21.15.181:0/3739112942 wait complete. 2020-11-16T22:27:33.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 Processor -- start 2020-11-16T22:27:33.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- start start 2020-11-16T22:27:33.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 0x7f67e40b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4136990 0x7f67e4136e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4173760 con 0x7f67e4161860 2020-11-16T22:27:33.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4173a00 con 0x7f67e4136990 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4173ca0 con 0x7f67e4161480 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37044/0 (socket says 172.21.15.181:37044) 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e8991700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4136990 0x7f67e4136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 -- 172.21.15.181:0/373325932 learned_addr learned my addr 172.21.15.181:0/373325932 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e37fe700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 0x7f67e40b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 -- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4136990 msgr2=0x7f67e4136e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4136990 0x7f67e4136e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 -- 172.21.15.181:0/373325932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 msgr2=0x7f67e40b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 0x7f67e40b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 -- 172.21.15.181:0/373325932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4173f40 con 0x7f67e4161480 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e37fe700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 0x7f67e40b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e3fff700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f67d400b150 tx=0x7f67d4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4011d70 con 0x7f67e4161480 2020-11-16T22:27:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d401e460 con 0x7f67e4161480 2020-11-16T22:27:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e41741e0 con 0x7f67e4161480 2020-11-16T22:27:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d401e7e0 con 0x7f67e4161480 2020-11-16T22:27:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.485+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e40b8170 con 0x7f67e4161480 2020-11-16T22:27:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4044f60 con 0x7f67e4161480 2020-11-16T22:27:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f67d4025030 con 0x7f67e4161480 2020-11-16T22:27:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67dbfff700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 0x7f67b4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67e37fe700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 0x7f67b4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67d406f520 con 0x7f67e4161480 2020-11-16T22:27:33.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.489+0000 7f67e37fe700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 0x7f67b4057550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f67e4162650 tx=0x7f67cc009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.501+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f67d4045670 con 0x7f67e4161480 2020-11-16T22:27:33.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.653+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e40671e0 con 0x7f67e4161480 2020-11-16T22:27:33.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.653+0000 7f67dbfff700 1 -- 172.21.15.181:0/373325932 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f67d4023020 con 0x7f67e4161480 2020-11-16T22:27:33.639 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 msgr2=0x7f67b4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 0x7f67b4057550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f67e4162650 tx=0x7f67cc009040).stop 2020-11-16T22:27:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 msgr2=0x7f67e4161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f67d400b150 tx=0x7f67d4015000).stop 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 shutdown_connections 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055090 0x7f67b4057550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4136990 0x7f67e4136e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161480 0x7f67e4161e50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 --2- 172.21.15.181:0/373325932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161860 0x7f67e40b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 >> 172.21.15.181:0/373325932 conn(0x7f67e40d2dc0 msgr2=0x7f67e4147650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 shutdown_connections 2020-11-16T22:27:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.657+0000 7f67e9192700 1 -- 172.21.15.181:0/373325932 wait complete. 2020-11-16T22:27:33.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:33.655 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:33.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 Processor -- start 2020-11-16T22:27:33.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 -- start start 2020-11-16T22:27:33.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc38815dfd0 0x7fc38815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3881611b0 con 0x7fc388061100 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38815f0b0 con 0x7fc38815dfd0 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc38ca58700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815dbe0 con 0x7fc388157a80 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc386d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc386d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37052/0 (socket says 172.21.15.181:37052) 2020-11-16T22:27:33.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.793+0000 7fc386d9d700 1 -- 172.21.15.181:0/1952345010 learned_addr learned my addr 172.21.15.181:0/1952345010 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:33.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38759e700 1 --2- 172.21.15.181:0/1952345010 >> v2:172.21.15.175:3300/0 conn(0x7fc38815dfd0 0x7fc38815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc386d9d700 1 -- 172.21.15.181:0/1952345010 >> v2:172.21.15.175:3300/0 conn(0x7fc38815dfd0 msgr2=0x7fc38815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc386d9d700 1 --2- 172.21.15.181:0/1952345010 >> v2:172.21.15.175:3300/0 conn(0x7fc38815dfd0 0x7fc38815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38659c700 1 -- 172.21.15.181:0/1952345010 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547451228 0 0) 0x7fc3881611b0 con 0x7fc388061100 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc386d9d700 1 -- 172.21.15.181:0/1952345010 >> v1:172.21.15.175:6789/0 conn(0x7fc388061100 legacy=0x7fc388061580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc386d9d700 1 -- 172.21.15.181:0/1952345010 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388161450 con 0x7fc388157a80 2020-11-16T22:27:33.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc386d9d700 1 --2- 172.21.15.181:0/1952345010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fc378006fd0 tx=0x7fc378006eb0).ready entity=mon.2 client_cookie=d2d9a06bd0030bfd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38659c700 1 -- 172.21.15.181:0/1952345010 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378009810 con 0x7fc388157a80 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38659c700 1 -- 172.21.15.181:0/1952345010 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378009970 con 0x7fc388157a80 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38659c700 1 -- 172.21.15.181:0/1952345010 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378009cf0 con 0x7fc388157a80 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- 172.21.15.181:0/1952345010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 msgr2=0x7fc3880cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 --2- 172.21.15.181:0/1952345010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fc378006fd0 tx=0x7fc378006eb0).stop 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- 172.21.15.181:0/1952345010 shutdown_connections 2020-11-16T22:27:33.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 --2- 172.21.15.181:0/1952345010 >> v2:172.21.15.175:3300/0 conn(0x7fc38815dfd0 0x7fc38815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 --2- 172.21.15.181:0/1952345010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc388157a80 0x7fc3880cc0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- 172.21.15.181:0/1952345010 >> 172.21.15.181:0/1952345010 conn(0x7fc3880cf630 msgr2=0x7fc3880cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- 172.21.15.181:0/1952345010 shutdown_connections 2020-11-16T22:27:33.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- 172.21.15.181:0/1952345010 wait complete. 2020-11-16T22:27:33.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 Processor -- start 2020-11-16T22:27:33.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.797+0000 7fc38ca58700 1 -- start start 2020-11-16T22:27:33.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc388061100 0x7fc388154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc388157a80 0x7fc38815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880d1a40 con 0x7fc388061100 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815d820 con 0x7fc388157a80 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38816d580 con 0x7fc38815dfd0 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc386d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc388061100 0x7fc388154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37062/0 (socket says 172.21.15.181:37062) 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 -- 172.21.15.181:0/59438299 learned_addr learned my addr 172.21.15.181:0/59438299 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc37e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc388157a80 0x7fc38815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 -- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc388157a80 msgr2=0x7fc38815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc388157a80 0x7fc38815ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 -- 172.21.15.181:0/59438299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc388061100 msgr2=0x7fc388154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc388061100 0x7fc388154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 -- 172.21.15.181:0/59438299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38816d820 con 0x7fc38815dfd0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38759e700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc38000ef20 tx=0x7fc3800091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380009690 con 0x7fc38815dfd0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000cb00 con 0x7fc38815dfd0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38816dac0 con 0x7fc38815dfd0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000cea0 con 0x7fc38815dfd0 2020-11-16T22:27:33.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38816e000 con 0x7fc38815dfd0 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.801+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc388044f60 con 0x7fc38815dfd0 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.805+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc380022070 con 0x7fc38815dfd0 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.805+0000 7fc37f7fe700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 0x7fc364057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.805+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc38006e400 con 0x7fc38815dfd0 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.805+0000 7fc386d9d700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 0x7fc364057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:33.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.805+0000 7fc386d9d700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 0x7fc364057520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fc37800b1b0 tx=0x7fc378015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.809+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc380044660 con 0x7fc38815dfd0 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.957+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc3880663b0 con 0x7fc38815dfd0 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc37f7fe700 1 -- 172.21.15.181:0/59438299 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc380016070 con 0x7fc38815dfd0 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 msgr2=0x7fc364057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 0x7fc364057520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fc37800b1b0 tx=0x7fc378015000).stop 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 msgr2=0x7fc38815d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fc38000ef20 tx=0x7fc3800091d0).stop 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 shutdown_connections 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc364055060 0x7fc364057520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc38815dfd0 0x7fc38815d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc388061100 0x7fc388154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 --2- 172.21.15.181:0/59438299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc388157a80 0x7fc38815ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 >> 172.21.15.181:0/59438299 conn(0x7fc3880cf630 msgr2=0x7fc3880d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 shutdown_connections 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:33.961+0000 7fc38ca58700 1 -- 172.21.15.181:0/59438299 wait complete. 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:34.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.097+0000 7fe54fbf1700 1 Processor -- start 2020-11-16T22:27:34.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.097+0000 7fe54fbf1700 1 -- start start 2020-11-16T22:27:34.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5480615f0 0x7fe5480622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe548160e40 con 0x7fe548062790 2020-11-16T22:27:34.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe548160ba0 con 0x7fe54815f5b0 2020-11-16T22:27:34.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5481609a0 con 0x7fe5480615f0 2020-11-16T22:27:34.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d98d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe54815f5b0 0x7fe5480d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43530/0 (socket says 172.21.15.181:43530) 2020-11-16T22:27:34.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d98d700 1 -- 172.21.15.181:0/2286626327 learned_addr learned my addr 172.21.15.181:0/2286626327 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:34.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d18c700 1 --2- 172.21.15.181:0/2286626327 >> v2:172.21.15.175:3300/0 conn(0x7fe5480615f0 0x7fe5480622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484106368 0 0) 0x7fe548160ba0 con 0x7fe54815f5b0 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe530003240 con 0x7fe54815f5b0 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d18c700 1 -- 172.21.15.181:0/2286626327 >> v1:172.21.15.181:6789/0 conn(0x7fe54815f5b0 legacy=0x7fe5480d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2617946070 0 0) 0x7fe530003240 con 0x7fe54815f5b0 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d18c700 1 -- 172.21.15.181:0/2286626327 >> v1:172.21.15.175:6789/0 conn(0x7fe548062790 legacy=0x7fe548080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d18c700 1 -- 172.21.15.181:0/2286626327 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe548060970 con 0x7fe5480615f0 2020-11-16T22:27:34.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54d18c700 1 --2- 172.21.15.181:0/2286626327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5480615f0 0x7fe5480622a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe53800b920 tx=0x7fe538006df0).ready entity=mon.0 client_cookie=12c250f1db9790d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544001660 con 0x7fe5480615f0 2020-11-16T22:27:34.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538011400 con 0x7fe5480615f0 2020-11-16T22:27:34.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54c98b700 1 -- 172.21.15.181:0/2286626327 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538011780 con 0x7fe5480615f0 2020-11-16T22:27:34.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- 172.21.15.181:0/2286626327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5480615f0 msgr2=0x7fe5480622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/2286626327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5480615f0 0x7fe5480622a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe53800b920 tx=0x7fe538006df0).stop 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- 172.21.15.181:0/2286626327 shutdown_connections 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/2286626327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5480615f0 0x7fe5480622a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- 172.21.15.181:0/2286626327 >> 172.21.15.181:0/2286626327 conn(0x7fe5480d69a0 msgr2=0x7fe5480d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- 172.21.15.181:0/2286626327 shutdown_connections 2020-11-16T22:27:34.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.101+0000 7fe54fbf1700 1 -- 172.21.15.181:0/2286626327 wait complete. 2020-11-16T22:27:34.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 Processor -- start 2020-11-16T22:27:34.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 -- start start 2020-11-16T22:27:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe548062790 0x7fe548149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814acc0 con 0x7fe548062790 2020-11-16T22:27:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814a780 con 0x7fe54815f5b0 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54fbf1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814aa20 con 0x7fe5480615f0 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54e18e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54e18e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57464/0 (socket says 172.21.15.181:57464) 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37072/0 (socket says 172.21.15.181:37072) 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54e18e700 1 -- 172.21.15.181:0/4239145390 learned_addr learned my addr 172.21.15.181:0/4239145390 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:34.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d18c700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe548062790 0x7fe548149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 -- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 msgr2=0x7fe548080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 -- 172.21.15.181:0/4239145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe548062790 msgr2=0x7fe548149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe548062790 0x7fe548149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 -- 172.21.15.181:0/4239145390 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480bad10 con 0x7fe5480615f0 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54e18e700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.105+0000 7fe54d98d700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe54400c8e0 tx=0x7fe5440012c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544007320 con 0x7fe5480615f0 2020-11-16T22:27:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5440054c0 con 0x7fe5480615f0 2020-11-16T22:27:34.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5480bafb0 con 0x7fe5480615f0 2020-11-16T22:27:34.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544005880 con 0x7fe5480615f0 2020-11-16T22:27:34.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe548164030 con 0x7fe5480615f0 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe544010030 con 0x7fe5480615f0 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe548069eb0 con 0x7fe5480615f0 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 0x7fe524057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe54406db80 con 0x7fe5480615f0 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe54d18c700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 0x7fe524057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.109+0000 7fe54d18c700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 0x7fe524057550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fe548061a50 tx=0x7fe538015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.117+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe544043de0 con 0x7fe5480615f0 2020-11-16T22:27:34.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.269+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe548164280 con 0x7fe5480615f0 2020-11-16T22:27:34.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.269+0000 7fe535ffb700 1 -- 172.21.15.181:0/4239145390 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe544047400 con 0x7fe5480615f0 2020-11-16T22:27:34.255 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 msgr2=0x7fe524057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 0x7fe524057550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fe548061a50 tx=0x7fe538015000).stop 2020-11-16T22:27:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 msgr2=0x7fe548163b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe54400c8e0 tx=0x7fe5440012c0).stop 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 shutdown_connections 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe524055090 0x7fe524057550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54815f5b0 0x7fe548080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5480615f0 0x7fe548163b40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 --2- 172.21.15.181:0/4239145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe548062790 0x7fe548149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 >> 172.21.15.181:0/4239145390 conn(0x7fe5480d69a0 msgr2=0x7fe548061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 shutdown_connections 2020-11-16T22:27:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.273+0000 7fe54fbf1700 1 -- 172.21.15.181:0/4239145390 wait complete. 2020-11-16T22:27:34.269 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:34.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.413+0000 7f00abfff700 1 Processor -- start 2020-11-16T22:27:34.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.413+0000 7f00abfff700 1 -- start start 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00abfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac1461a0 0x7f00ac061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00ac0615f0 0x7f00ac0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00abfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0c8420 con 0x7f00ac145dc0 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00abfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0c8200 con 0x7f00ac0615f0 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00abfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c86c0 con 0x7f00ac1461a0 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00ac0615f0 0x7f00ac0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00aa7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac1461a0 0x7f00ac061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00ac0615f0 0x7f00ac0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57474/0 (socket says 172.21.15.181:57474) 2020-11-16T22:27:34.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 -- 172.21.15.181:0/3194692650 learned_addr learned my addr 172.21.15.181:0/3194692650 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:34.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 -- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac1461a0 msgr2=0x7f00ac061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:34.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 --2- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac1461a0 0x7f00ac061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 -- 172.21.15.181:0/3194692650 >> v1:172.21.15.175:6789/0 conn(0x7f00ac145dc0 legacy=0x7f00ac0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 -- 172.21.15.181:0/3194692650 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0c8960 con 0x7f00ac0615f0 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00ab7fe700 1 --2- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0615f0 0x7f00ac0cdc70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f00a4009d50 tx=0x7f00a4009290).ready entity=mon.1 client_cookie=3f48d33a0f24c2a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00a9ffb700 1 -- 172.21.15.181:0/3194692650 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400bbc0 con 0x7f00ac0615f0 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00a9ffb700 1 -- 172.21.15.181:0/3194692650 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a400bd20 con 0x7f00ac0615f0 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.417+0000 7f00a9ffb700 1 -- 172.21.15.181:0/3194692650 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011610 con 0x7f00ac0615f0 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0615f0 msgr2=0x7f00ac0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0615f0 0x7f00ac0cdc70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f00a4009d50 tx=0x7f00a4009290).stop 2020-11-16T22:27:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/3194692650 shutdown_connections 2020-11-16T22:27:34.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0615f0 0x7f00ac0cdc70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- 172.21.15.181:0/3194692650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac1461a0 0x7f00ac061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/3194692650 >> 172.21.15.181:0/3194692650 conn(0x7f00ac0cf630 msgr2=0x7f00ac0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/3194692650 shutdown_connections 2020-11-16T22:27:34.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/3194692650 wait complete. 2020-11-16T22:27:34.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 Processor -- start 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- start start 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ac0615f0 0x7f00ac148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac1461a0 0x7f00ac0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0b5b20 con 0x7f00ac0615f0 2020-11-16T22:27:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0b5d50 con 0x7f00ac1461a0 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0b5ff0 con 0x7f00ac145dc0 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37086/0 (socket says 172.21.15.181:37086) 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 -- 172.21.15.181:0/2206903160 learned_addr learned my addr 172.21.15.181:0/2206903160 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac1461a0 0x7f00ac0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aaffd700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ac0615f0 0x7f00ac148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 -- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac1461a0 msgr2=0x7f00ac0b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac1461a0 0x7f00ac0b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 -- 172.21.15.181:0/2206903160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ac0615f0 msgr2=0x7f00ac148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ac0615f0 0x7f00ac148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 -- 172.21.15.181:0/2206903160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0b6290 con 0x7f00ac145dc0 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00aa7fc700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f00a000eba0 tx=0x7f00a0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000b3a0 con 0x7f00ac145dc0 2020-11-16T22:27:34.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000c9b0 con 0x7f00ac145dc0 2020-11-16T22:27:34.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000cd70 con 0x7f00ac145dc0 2020-11-16T22:27:34.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0b6530 con 0x7f00ac145dc0 2020-11-16T22:27:34.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.421+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac149310 con 0x7f00ac145dc0 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac0b69d0 con 0x7f00ac145dc0 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f00a0012600 con 0x7f00ac145dc0 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f0092ffd700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 0x7f0088057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f00aaffd700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 0x7f0088057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00a006e170 con 0x7f00ac145dc0 2020-11-16T22:27:34.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.425+0000 7f00aaffd700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 0x7f0088057520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f009c007f80 tx=0x7f009c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.433+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f00ac0b69d0 con 0x7f00ac145dc0 2020-11-16T22:27:34.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.585+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00ac0b69d0 con 0x7f00ac145dc0 2020-11-16T22:27:34.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.585+0000 7f0092ffd700 1 -- 172.21.15.181:0/2206903160 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f00ac0b69d0 con 0x7f00ac145dc0 2020-11-16T22:27:34.571 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:34.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 msgr2=0x7f0088057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 0x7f0088057520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f009c007f80 tx=0x7f009c009040).stop 2020-11-16T22:27:34.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 msgr2=0x7f00ac1411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f00a000eba0 tx=0x7f00a0006df0).stop 2020-11-16T22:27:34.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 shutdown_connections 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0088055060 0x7f0088057520 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac1461a0 0x7f00ac0b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ac0615f0 0x7f00ac148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 --2- 172.21.15.181:0/2206903160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac145dc0 0x7f00ac1411c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 >> 172.21.15.181:0/2206903160 conn(0x7f00ac0cf630 msgr2=0x7f00ac0ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 shutdown_connections 2020-11-16T22:27:34.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.589+0000 7f00abfff700 1 -- 172.21.15.181:0/2206903160 wait complete. 2020-11-16T22:27:34.591 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:34.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.737+0000 7f4e232a6700 1 Processor -- start 2020-11-16T22:27:34.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.737+0000 7f4e232a6700 1 -- start start 2020-11-16T22:27:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e232a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e232a6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4e1c061810 0x7f4e1c160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e232a6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c15d180 con 0x7f4e1c15fdc0 2020-11-16T22:27:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e232a6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c160730 con 0x7f4e1c061810 2020-11-16T22:27:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e232a6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c13f140 con 0x7f4e1c13e870 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37094/0 (socket says 172.21.15.181:37094) 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 -- 172.21.15.181:0/4081234964 learned_addr learned my addr 172.21.15.181:0/4081234964 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e22aa5700 1 --2- 172.21.15.181:0/4081234964 >> v2:172.21.15.181:3300/0 conn(0x7f4e1c061810 0x7f4e1c160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 -- 172.21.15.181:0/4081234964 >> v2:172.21.15.181:3300/0 conn(0x7f4e1c061810 msgr2=0x7f4e1c160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:34.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 --2- 172.21.15.181:0/4081234964 >> v2:172.21.15.181:3300/0 conn(0x7f4e1c061810 0x7f4e1c160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 -- 172.21.15.181:0/4081234964 >> v1:172.21.15.181:6789/0 conn(0x7f4e1c15fdc0 legacy=0x7f4e1c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 -- 172.21.15.181:0/4081234964 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c15d810 con 0x7f4e1c13e870 2020-11-16T22:27:34.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e222a4700 1 --2- 172.21.15.181:0/4081234964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f4e1800bbe0 tx=0x7f4e1800b120).ready entity=mon.2 client_cookie=81fa2f9298ac08b7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e212a2700 1 -- 172.21.15.181:0/4081234964 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800d9c0 con 0x7f4e1c13e870 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e212a2700 1 -- 172.21.15.181:0/4081234964 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1800db20 con 0x7f4e1c13e870 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.741+0000 7f4e212a2700 1 -- 172.21.15.181:0/4081234964 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800dea0 con 0x7f4e1c13e870 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- 172.21.15.181:0/4081234964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 msgr2=0x7f4e1c13ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 --2- 172.21.15.181:0/4081234964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f4e1800bbe0 tx=0x7f4e1800b120).stop 2020-11-16T22:27:34.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- 172.21.15.181:0/4081234964 shutdown_connections 2020-11-16T22:27:34.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 --2- 172.21.15.181:0/4081234964 >> v2:172.21.15.181:3300/0 conn(0x7f4e1c061810 0x7f4e1c160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 --2- 172.21.15.181:0/4081234964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c13e870 0x7f4e1c13ec50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- 172.21.15.181:0/4081234964 >> 172.21.15.181:0/4081234964 conn(0x7f4e1c0cf630 msgr2=0x7f4e1c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- 172.21.15.181:0/4081234964 shutdown_connections 2020-11-16T22:27:34.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- 172.21.15.181:0/4081234964 wait complete. 2020-11-16T22:27:34.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 Processor -- start 2020-11-16T22:27:34.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.745+0000 7f4e232a6700 1 -- start start 2020-11-16T22:27:34.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 0x7f4e1c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e1c13e870 0x7f4e1c0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 0x7f4e1c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b4410 con 0x7f4e1c13e870 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b3250 con 0x7f4e1c15fdc0 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b34f0 con 0x7f4e1c061810 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e22aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 0x7f4e1c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 0x7f4e1c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e22aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 0x7f4e1c0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57494/0 (socket says 172.21.15.181:57494) 2020-11-16T22:27:34.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e22aa5700 1 -- 172.21.15.181:0/1272974628 learned_addr learned my addr 172.21.15.181:0/1272974628 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e21aa3700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e1c13e870 0x7f4e1c0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 -- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 msgr2=0x7f4e1c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 0x7f4e1c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 -- 172.21.15.181:0/1272974628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e1c13e870 msgr2=0x7f4e1c0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e1c13e870 0x7f4e1c0c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 -- 172.21.15.181:0/1272974628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0b3790 con 0x7f4e1c061810 2020-11-16T22:27:34.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e222a4700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 0x7f4e1c0b2810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f4e18015000 tx=0x7f4e18018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18008740 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e180058e0 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18005c60 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c0b3a30 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c0b2d00 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4e18019040 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c044f60 con 0x7f4e1c061810 2020-11-16T22:27:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 0x7f4df4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:34.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e21aa3700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 0x7f4df4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:34.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e1806e6f0 con 0x7f4e1c061810 2020-11-16T22:27:34.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.749+0000 7f4e21aa3700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 0x7f4df4057520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4e1c0b8a70 tx=0x7f4e0c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:34.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.753+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4e18044a00 con 0x7f4e1c061810 2020-11-16T22:27:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.905+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e1c05c310 con 0x7f4e1c061810 2020-11-16T22:27:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.905+0000 7f4e0a7fc700 1 -- 172.21.15.181:0/1272974628 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4e180220a0 con 0x7f4e1c061810 2020-11-16T22:27:34.892 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 msgr2=0x7f4df4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 0x7f4df4057520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4e1c0b8a70 tx=0x7f4e0c009040).stop 2020-11-16T22:27:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 msgr2=0x7f4e1c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 0x7f4e1c0b2810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f4e18015000 tx=0x7f4e18018000).stop 2020-11-16T22:27:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 shutdown_connections 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4df4055060 0x7f4df4057520 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e1c15fdc0 0x7f4e1c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e1c061810 0x7f4e1c0b2810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 --2- 172.21.15.181:0/1272974628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e1c13e870 0x7f4e1c0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 >> 172.21.15.181:0/1272974628 conn(0x7f4e1c0cf630 msgr2=0x7f4e1c0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 shutdown_connections 2020-11-16T22:27:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:34.909+0000 7f4e232a6700 1 -- 172.21.15.181:0/1272974628 wait complete. 2020-11-16T22:27:34.906 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:35.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 Processor -- start 2020-11-16T22:27:35.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- start start 2020-11-16T22:27:35.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb924161480 0x7fb924164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb924164e10 0x7fb924062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9240c8db0 con 0x7fb924161860 2020-11-16T22:27:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9240c8330 con 0x7fb924164e10 2020-11-16T22:27:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9240c85d0 con 0x7fb924161480 2020-11-16T22:27:35.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb924161480 0x7fb924164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb924161480 0x7fb924164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57498/0 (socket says 172.21.15.181:57498) 2020-11-16T22:27:35.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 -- 172.21.15.181:0/3507052002 learned_addr learned my addr 172.21.15.181:0/3507052002 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:35.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929634700 1 --1- 172.21.15.181:0/3507052002 >> v1:172.21.15.181:6789/0 conn(0x7fb924161860 0x7fb924062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43574/0 (socket says 172.21.15.181:43574) 2020-11-16T22:27:35.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb923fff700 1 --2- 172.21.15.181:0/3507052002 >> v2:172.21.15.175:3300/0 conn(0x7fb924164e10 0x7fb924062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 -- 172.21.15.181:0/3507052002 >> v2:172.21.15.175:3300/0 conn(0x7fb924164e10 msgr2=0x7fb924062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:35.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 --2- 172.21.15.181:0/3507052002 >> v2:172.21.15.175:3300/0 conn(0x7fb924164e10 0x7fb924062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 -- 172.21.15.181:0/3507052002 >> v1:172.21.15.181:6789/0 conn(0x7fb924161860 legacy=0x7fb924062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 -- 172.21.15.181:0/3507052002 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240c8870 con 0x7fb924161480 2020-11-16T22:27:35.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb928e33700 1 --2- 172.21.15.181:0/3507052002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb924161480 0x7fb924164920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb9240c8260 tx=0x7fb914009290).ready entity=mon.1 client_cookie=c515854cfb7249e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:35.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb9237fe700 1 -- 172.21.15.181:0/3507052002 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91400bbc0 con 0x7fb924161480 2020-11-16T22:27:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb9237fe700 1 -- 172.21.15.181:0/3507052002 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91400bd20 con 0x7fb924161480 2020-11-16T22:27:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb9237fe700 1 -- 172.21.15.181:0/3507052002 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914011610 con 0x7fb924161480 2020-11-16T22:27:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- 172.21.15.181:0/3507052002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb924161480 msgr2=0x7fb924164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 --2- 172.21.15.181:0/3507052002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb924161480 0x7fb924164920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb9240c8260 tx=0x7fb914009290).stop 2020-11-16T22:27:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- 172.21.15.181:0/3507052002 shutdown_connections 2020-11-16T22:27:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 --2- 172.21.15.181:0/3507052002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb924161480 0x7fb924164920 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 --2- 172.21.15.181:0/3507052002 >> v2:172.21.15.175:3300/0 conn(0x7fb924164e10 0x7fb924062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- 172.21.15.181:0/3507052002 >> 172.21.15.181:0/3507052002 conn(0x7fb9240d2dc0 msgr2=0x7fb9240ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- 172.21.15.181:0/3507052002 shutdown_connections 2020-11-16T22:27:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.041+0000 7fb929e35700 1 -- 172.21.15.181:0/3507052002 wait complete. 2020-11-16T22:27:35.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 Processor -- start 2020-11-16T22:27:35.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 -- start start 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb924161480 0x7fb92416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb924161860 0x7fb9240ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92416a580 con 0x7fb924161480 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb924165df0 con 0x7fb9240cf230 2020-11-16T22:27:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929e35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb924166090 con 0x7fb924161860 2020-11-16T22:27:35.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57506/0 (socket says 172.21.15.181:57506) 2020-11-16T22:27:35.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 -- 172.21.15.181:0/1415471648 learned_addr learned my addr 172.21.15.181:0/1415471648 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:35.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb923fff700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb924161860 0x7fb9240ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb928e33700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb924161480 0x7fb92416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 -- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb924161860 msgr2=0x7fb9240ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb924161860 0x7fb9240ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 -- 172.21.15.181:0/1415471648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb924161480 msgr2=0x7fb92416a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb924161480 0x7fb92416a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.045+0000 7fb929634700 1 -- 172.21.15.181:0/1415471648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924166330 con 0x7fb9240cf230 2020-11-16T22:27:35.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb929634700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb90c00c710 tx=0x7fb90c007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:35.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0072e0 con 0x7fb9240cf230 2020-11-16T22:27:35.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c005530 con 0x7fb9240cf230 2020-11-16T22:27:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9241665d0 con 0x7fb9240cf230 2020-11-16T22:27:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0058b0 con 0x7fb9240cf230 2020-11-16T22:27:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb924166b10 con 0x7fb9240cf230 2020-11-16T22:27:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb924044f60 con 0x7fb9240cf230 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb90c01a060 con 0x7fb9240cf230 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 0x7fb900057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb90c06d5a0 con 0x7fb9240cf230 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb928e33700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 0x7fb900057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.049+0000 7fb928e33700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 0x7fb900057520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb914009fd0 tx=0x7fb914016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.053+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb90c0436f0 con 0x7fb9240cf230 2020-11-16T22:27:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.205+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9240671e0 con 0x7fb9240cf230 2020-11-16T22:27:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb920ff9700 1 -- 172.21.15.181:0/1415471648 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fb90c018020 con 0x7fb9240cf230 2020-11-16T22:27:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 msgr2=0x7fb900057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 0x7fb900057520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb914009fd0 tx=0x7fb914016000).stop 2020-11-16T22:27:35.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 msgr2=0x7fb924166fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:35.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb90c00c710 tx=0x7fb90c007b00).stop 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 shutdown_connections 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb900055060 0x7fb900057520 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9240cf230 0x7fb924166fb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb924161480 0x7fb92416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 --2- 172.21.15.181:0/1415471648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb924161860 0x7fb9240ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:35.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 >> 172.21.15.181:0/1415471648 conn(0x7fb9240d2dc0 msgr2=0x7fb924161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:35.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 shutdown_connections 2020-11-16T22:27:35.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:35.209+0000 7fb929e35700 1 -- 172.21.15.181:0/1415471648 wait complete. 2020-11-16T22:27:35.852 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:35.852 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:36.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:37.074 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:37.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 Processor -- start 2020-11-16T22:27:37.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 -- start start 2020-11-16T22:27:37.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 0x7fe608163e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 0x7fe6080813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe608061900 con 0x7fe608062af0 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe608061bd0 con 0x7fe608163a50 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60eb48700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe608061e70 con 0x7fe608081890 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60c8e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 0x7fe608163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37124/0 (socket says 172.21.15.181:37124) 2020-11-16T22:27:37.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60c8e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 0x7fe608163e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57512/0 (socket says 172.21.15.181:57512) 2020-11-16T22:27:37.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 -- 172.21.15.181:0/2301639022 learned_addr learned my addr 172.21.15.181:0/2301639022 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe607fff700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 0x7fe6080813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 -- 172.21.15.181:0/2301639022 >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 msgr2=0x7fe608163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 0x7fe608163e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 -- 172.21.15.181:0/2301639022 >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 msgr2=0x7fe6080813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 0x7fe6080813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 -- 172.21.15.181:0/2301639022 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608062110 con 0x7fe608081890 2020-11-16T22:27:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe607fff700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 0x7fe6080813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe60d0e5700 1 --2- 172.21.15.181:0/2301639022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe608083ad0 tx=0x7fe600009290).ready entity=mon.2 client_cookie=6ecec2c2d56fcf46 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe6077fe700 1 -- 172.21.15.181:0/2301639022 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000bbc0 con 0x7fe608081890 2020-11-16T22:27:37.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe6077fe700 1 -- 172.21.15.181:0/2301639022 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60000bd20 con 0x7fe608081890 2020-11-16T22:27:37.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.229+0000 7fe6077fe700 1 -- 172.21.15.181:0/2301639022 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600011610 con 0x7fe608081890 2020-11-16T22:27:37.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- 172.21.15.181:0/2301639022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 msgr2=0x7fe6080d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:37.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 --2- 172.21.15.181:0/2301639022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe608083ad0 tx=0x7fe600009290).stop 2020-11-16T22:27:37.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- 172.21.15.181:0/2301639022 shutdown_connections 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 --2- 172.21.15.181:0/2301639022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe6080d1340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.181:3300/0 conn(0x7fe608163a50 0x7fe608163e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 --2- 172.21.15.181:0/2301639022 >> v2:172.21.15.175:3300/0 conn(0x7fe608062af0 0x7fe6080813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- 172.21.15.181:0/2301639022 >> 172.21.15.181:0/2301639022 conn(0x7fe6080d69a0 msgr2=0x7fe6080d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- 172.21.15.181:0/2301639022 shutdown_connections 2020-11-16T22:27:37.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- 172.21.15.181:0/2301639022 wait complete. 2020-11-16T22:27:37.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 Processor -- start 2020-11-16T22:27:37.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.233+0000 7fe60eb48700 1 -- start start 2020-11-16T22:27:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe608176900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 0x7fe60815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081733d0 con 0x7fe608163a50 2020-11-16T22:27:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081777a0 con 0x7fe608062af0 2020-11-16T22:27:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe608177260 con 0x7fe608081890 2020-11-16T22:27:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57518/0 (socket says 172.21.15.181:57518) 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 -- 172.21.15.181:0/3233538967 learned_addr learned my addr 172.21.15.181:0/3233538967 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe607fff700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe608176900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60d0e5700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 0x7fe60815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 -- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 msgr2=0x7fe608176900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe608176900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 -- 172.21.15.181:0/3233538967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 msgr2=0x7fe60815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 0x7fe60815fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 -- 172.21.15.181:0/3233538967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608177500 con 0x7fe608062af0 2020-11-16T22:27:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60d0e5700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 0x7fe60815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:37.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60c8e4700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe5f800ebd0 tx=0x7fe5f8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:37.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800c8e0 con 0x7fe608062af0 2020-11-16T22:27:37.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f800ca40 con 0x7fe608062af0 2020-11-16T22:27:37.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6081735b0 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe608176df0 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800cdc0 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.237+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6081770a0 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.241+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe5f8012600 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.241+0000 7fe604ff9700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 0x7fe5e0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.241+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe5f806e080 con 0x7fe608062af0 2020-11-16T22:27:37.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.241+0000 7fe607fff700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 0x7fe5e0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:37.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.241+0000 7fe607fff700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 0x7fe5e0057550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe6080b6a70 tx=0x7fe5fc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:37.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.249+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe5f80442e0 con 0x7fe608062af0 2020-11-16T22:27:37.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.401+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe608069eb0 con 0x7fe608062af0 2020-11-16T22:27:38.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.401+0000 7fe604ff9700 1 -- 172.21.15.181:0/3233538967 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe5f8016020 con 0x7fe608062af0 2020-11-16T22:27:38.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.401+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 msgr2=0x7fe5e0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 0x7fe5e0057550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe6080b6a70 tx=0x7fe5fc008040).stop 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 msgr2=0x7fe608173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fe5f800ebd0 tx=0x7fe5f8008040).stop 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 shutdown_connections 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5e0055090 0x7fe5e0057550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe608163a50 0x7fe60815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe608062af0 0x7fe608173f90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 --2- 172.21.15.181:0/3233538967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe608081890 0x7fe608176900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 >> 172.21.15.181:0/3233538967 conn(0x7fe6080d69a0 msgr2=0x7fe6080b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 shutdown_connections 2020-11-16T22:27:38.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:37.405+0000 7fe60eb48700 1 -- 172.21.15.181:0/3233538967 wait complete. 2020-11-16T22:27:38.040 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:38.041 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.193+0000 7f33c7fff700 1 Processor -- start 2020-11-16T22:27:38.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.193+0000 7f33c7fff700 1 -- start start 2020-11-16T22:27:38.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.193+0000 7f33c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33c81606c0 0x7f33c8160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.193+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 0x7f33c80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.193+0000 7f33c7fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c8160f90 con 0x7f33c80c8e30 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80c9800 con 0x7f33c81606c0 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80c9aa0 con 0x7f33c80c9290 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 0x7f33c80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33c81606c0 0x7f33c8160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 0x7f33c80b78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37140/0 (socket says 172.21.15.181:37140) 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c77fe700 1 -- 172.21.15.181:0/832989751 learned_addr learned my addr 172.21.15.181:0/832989751 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 -- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 msgr2=0x7f33c80b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 0x7f33c80b78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 -- 172.21.15.181:0/832989751 >> v1:172.21.15.175:6789/0 conn(0x7f33c80c8e30 legacy=0x7f33c80b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 -- 172.21.15.181:0/832989751 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80c9d40 con 0x7f33c81606c0 2020-11-16T22:27:38.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c81606c0 0x7f33c8160aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f33b400b9a0 tx=0x7f33b4006df0).ready entity=mon.1 client_cookie=54a6a0c63c716a4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c5ffb700 1 -- 172.21.15.181:0/832989751 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40097b0 con 0x7f33c81606c0 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c5ffb700 1 -- 172.21.15.181:0/832989751 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4009910 con 0x7f33c81606c0 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c5ffb700 1 -- 172.21.15.181:0/832989751 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009c90 con 0x7f33c81606c0 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c81606c0 msgr2=0x7f33c8160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 --2- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c81606c0 0x7f33c8160aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f33b400b9a0 tx=0x7f33b4006df0).stop 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- 172.21.15.181:0/832989751 shutdown_connections 2020-11-16T22:27:38.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 --2- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c9290 0x7f33c80b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 --2- 172.21.15.181:0/832989751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c81606c0 0x7f33c8160aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- 172.21.15.181:0/832989751 >> 172.21.15.181:0/832989751 conn(0x7f33c80d33f0 msgr2=0x7f33c80d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- 172.21.15.181:0/832989751 shutdown_connections 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.197+0000 7f33c7fff700 1 -- 172.21.15.181:0/832989751 wait complete. 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 Processor -- start 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- start start 2020-11-16T22:27:38.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 0x7f33c8154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33c81606c0 0x7f33c80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c814ba30 con 0x7f33c81606c0 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8149cd0 con 0x7f33c80c9290 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8149f70 con 0x7f33c80c8e30 2020-11-16T22:27:38.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57532/0 (socket says 172.21.15.181:57532) 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c67fc700 1 -- 172.21.15.181:0/3604132243 learned_addr learned my addr 172.21.15.181:0/3604132243 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 0x7f33c8154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c77fe700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33c81606c0 0x7f33c80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 -- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 msgr2=0x7f33c8154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 -- 172.21.15.181:0/3604132243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33c81606c0 msgr2=0x7f33c80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33c81606c0 0x7f33c80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 -- 172.21.15.181:0/3604132243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c814a210 con 0x7f33c80c8e30 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c67fc700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c6ffd700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 0x7f33c8154670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f33b4012000 tx=0x7f33b4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4011d50 con 0x7f33c80c8e30 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b401e460 con 0x7f33c80c8e30 2020-11-16T22:27:38.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b401e7e0 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c814a4b0 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.201+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c814abf0 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f33b401a040 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8044f60 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33beffd700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 0x7f339c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f33b406ef80 con 0x7f33c80c8e30 2020-11-16T22:27:38.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33c67fc700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 0x7f339c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.205+0000 7f33c67fc700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 0x7f339c057520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f33c80b50c0 tx=0x7f33b8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.213+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f33b4045290 con 0x7f33c80c8e30 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33c8068fb0 con 0x7f33c80c8e30 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33beffd700 1 -- 172.21.15.181:0/3604132243 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33b4023030 con 0x7f33c80c8e30 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 msgr2=0x7f339c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 0x7f339c057520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f33c80b50c0 tx=0x7f33b8009040).stop 2020-11-16T22:27:38.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 msgr2=0x7f33c8154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 0x7f33c8154670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f33b4012000 tx=0x7f33b4015000).stop 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 shutdown_connections 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f339c055060 0x7f339c057520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33c81606c0 0x7f33c80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33c80c8e30 0x7f33c8154670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 --2- 172.21.15.181:0/3604132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33c80c9290 0x7f33c8154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.369+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 >> 172.21.15.181:0/3604132243 conn(0x7f33c80d33f0 msgr2=0x7f33c80d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.373+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 shutdown_connections 2020-11-16T22:27:38.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.373+0000 7f33c7fff700 1 -- 172.21.15.181:0/3604132243 wait complete. 2020-11-16T22:27:38.397 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:38.397 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 Processor -- start 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 -- start start 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2040ce960 0x7ff20415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff2040ced40 0x7ff20415f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20415d480 con 0x7ff20405fbc0 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20415d720 con 0x7ff2040ced40 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff203fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20415d9c0 con 0x7ff2040ce960 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2040ce960 0x7ff20415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff2027fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff20405fbc0 0x7ff20415f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43614/0 (socket says 172.21.15.181:43614) 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff2027fc700 1 -- 172.21.15.181:0/3829385771 learned_addr learned my addr 172.21.15.181:0/3829385771 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff2037fe700 1 --2- 172.21.15.181:0/3829385771 >> v2:172.21.15.175:3300/0 conn(0x7ff2040ced40 0x7ff20415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 -- 172.21.15.181:0/3829385771 >> v2:172.21.15.175:3300/0 conn(0x7ff2040ced40 msgr2=0x7ff20415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 --2- 172.21.15.181:0/3829385771 >> v2:172.21.15.175:3300/0 conn(0x7ff2040ced40 0x7ff20415f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 -- 172.21.15.181:0/3829385771 >> v1:172.21.15.181:6789/0 conn(0x7ff20405fbc0 legacy=0x7ff20415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 -- 172.21.15.181:0/3829385771 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20415dc60 con 0x7ff2040ce960 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.545+0000 7ff202ffd700 1 --2- 172.21.15.181:0/3829385771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ce960 0x7ff20415e640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff20406b040 tx=0x7ff1ec006df0).ready entity=mon.1 client_cookie=6ff10e78b6179366 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff201ffb700 1 -- 172.21.15.181:0/3829385771 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0097b0 con 0x7ff2040ce960 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff201ffb700 1 -- 172.21.15.181:0/3829385771 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ec009910 con 0x7ff2040ce960 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff201ffb700 1 -- 172.21.15.181:0/3829385771 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec009c90 con 0x7ff2040ce960 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 -- 172.21.15.181:0/3829385771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ce960 msgr2=0x7ff20415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 --2- 172.21.15.181:0/3829385771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ce960 0x7ff20415e640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff20406b040 tx=0x7ff1ec006df0).stop 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 -- 172.21.15.181:0/3829385771 shutdown_connections 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 --2- 172.21.15.181:0/3829385771 >> v2:172.21.15.175:3300/0 conn(0x7ff2040ced40 0x7ff20415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 --2- 172.21.15.181:0/3829385771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ce960 0x7ff20415e640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 -- 172.21.15.181:0/3829385771 >> 172.21.15.181:0/3829385771 conn(0x7ff204060bc0 msgr2=0x7ff2040d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 -- 172.21.15.181:0/3829385771 shutdown_connections 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.549+0000 7ff203fff700 1 -- 172.21.15.181:0/3829385771 wait complete. 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 Processor -- start 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 -- start start 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff20405fbc0 0x7ff20413de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 0x7ff204163630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040d1040 con 0x7ff20405fbc0 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040d12e0 con 0x7ff2040ced40 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff203fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040d1580 con 0x7ff2040ce960 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 0x7ff204163630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37158/0 (socket says 172.21.15.181:37158) 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 -- 172.21.15.181:0/2182473617 learned_addr learned my addr 172.21.15.181:0/2182473617 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff202ffd700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff20405fbc0 0x7ff20413de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 -- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 msgr2=0x7ff204163630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 0x7ff204163630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 -- 172.21.15.181:0/2182473617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff20405fbc0 msgr2=0x7ff20413de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff20405fbc0 0x7ff20413de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 -- 172.21.15.181:0/2182473617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040d1820 con 0x7ff2040ce960 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2037fe700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 0x7ff204163630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff2027fc700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff1f400ca60 tx=0x7ff1f4007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.553+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4005b80 con 0x7ff2040ce960 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4005ce0 con 0x7ff2040ce960 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2040d1ac0 con 0x7ff2040ce960 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4007340 con 0x7ff2040ce960 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff204163b20 con 0x7ff2040ce960 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff204044f60 con 0x7ff2040ce960 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff1f4010030 con 0x7ff2040ce960 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff1faffd700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 0x7ff1e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff1f406dba0 con 0x7ff2040ce960 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff202ffd700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 0x7ff1e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.557+0000 7ff202ffd700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 0x7ff1e0057520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff1ec00b150 tx=0x7ff1ec015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.565+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff1f4043e00 con 0x7ff2040ce960 2020-11-16T22:27:38.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.713+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2040611b0 con 0x7ff2040ce960 2020-11-16T22:27:38.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff1faffd700 1 -- 172.21.15.181:0/2182473617 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff1f4047420 con 0x7ff2040ce960 2020-11-16T22:27:38.701 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:38.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 msgr2=0x7ff1e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 0x7ff1e0057520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff1ec00b150 tx=0x7ff1ec015000).stop 2020-11-16T22:27:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 msgr2=0x7ff20413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff1f400ca60 tx=0x7ff1f4007ef0).stop 2020-11-16T22:27:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 shutdown_connections 2020-11-16T22:27:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff1e0055060 0x7ff1e0057520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2040ced40 0x7ff204163630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff20405fbc0 0x7ff20413de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 --2- 172.21.15.181:0/2182473617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2040ce960 0x7ff20413e370 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 >> 172.21.15.181:0/2182473617 conn(0x7ff204060bc0 msgr2=0x7ff20415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 shutdown_connections 2020-11-16T22:27:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.717+0000 7ff203fff700 1 -- 172.21.15.181:0/2182473617 wait complete. 2020-11-16T22:27:38.720 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:38.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 Processor -- start 2020-11-16T22:27:38.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 -- start start 2020-11-16T22:27:38.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 0x7f49c40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c40d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c4160e40 con 0x7f49c40622a0 2020-11-16T22:27:38.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c41609a0 con 0x7f49c415f5b0 2020-11-16T22:27:38.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49cab3c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4160ba0 con 0x7f49c40615f0 2020-11-16T22:27:38.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c90d9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f49c40622a0 0x7f49c4062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43632/0 (socket says 172.21.15.181:43632) 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c90d9700 1 -- 172.21.15.181:0/3230832038 learned_addr learned my addr 172.21.15.181:0/3230832038 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 --2- 172.21.15.181:0/3230832038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c40d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c88d8700 1 --2- 172.21.15.181:0/3230832038 >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 0x7f49c40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630326429 0 0) 0x7f49c4160e40 con 0x7f49c40622a0 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b0003240 con 0x7f49c40622a0 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 -- 172.21.15.181:0/3230832038 >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 msgr2=0x7f49c40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:38.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 --2- 172.21.15.181:0/3230832038 >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 0x7f49c40d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3832461868 0 0) 0x7f49b0003240 con 0x7f49c40622a0 2020-11-16T22:27:38.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 -- 172.21.15.181:0/3230832038 >> v1:172.21.15.181:6789/0 conn(0x7f49c40622a0 legacy=0x7f49c4062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 -- 172.21.15.181:0/3230832038 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4060970 con 0x7f49c40615f0 2020-11-16T22:27:38.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c88d8700 1 --2- 172.21.15.181:0/3230832038 >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 0x7f49c40d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:38.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c3fff700 1 --2- 172.21.15.181:0/3230832038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c40d1830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f49b800b9a0 tx=0x7f49b8006df0).ready entity=mon.2 client_cookie=aa9229b8199bf357 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.857+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc001860 con 0x7f49c40615f0 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8011400 con 0x7f49c40615f0 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49c37fe700 1 -- 172.21.15.181:0/3230832038 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8011780 con 0x7f49c40615f0 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- 172.21.15.181:0/3230832038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 msgr2=0x7f49c40d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 --2- 172.21.15.181:0/3230832038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c40d1830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f49b800b9a0 tx=0x7f49b8006df0).stop 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- 172.21.15.181:0/3230832038 shutdown_connections 2020-11-16T22:27:38.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 --2- 172.21.15.181:0/3230832038 >> v2:172.21.15.175:3300/0 conn(0x7f49c415f5b0 0x7f49c40d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 --2- 172.21.15.181:0/3230832038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c40d1830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- 172.21.15.181:0/3230832038 >> 172.21.15.181:0/3230832038 conn(0x7f49c40d69a0 msgr2=0x7f49c40d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- 172.21.15.181:0/3230832038 shutdown_connections 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- 172.21.15.181:0/3230832038 wait complete. 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 Processor -- start 2020-11-16T22:27:38.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.861+0000 7f49cab3c700 1 -- start start 2020-11-16T22:27:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49c40622a0 0x7f49c4081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c415f5b0 0x7f49c416fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40ba810 con 0x7f49c415f5b0 2020-11-16T22:27:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40baab0 con 0x7f49c40622a0 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4170c70 con 0x7f49c40615f0 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37174/0 (socket says 172.21.15.181:37174) 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49c40622a0 0x7f49c4081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 -- 172.21.15.181:0/1068145277 learned_addr learned my addr 172.21.15.181:0/1068145277 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c90d9700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c415f5b0 0x7f49c416fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 -- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49c40622a0 msgr2=0x7f49c4081660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49c40622a0 0x7f49c4081660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 -- 172.21.15.181:0/1068145277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c415f5b0 msgr2=0x7f49c416fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c415f5b0 0x7f49c416fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 -- 172.21.15.181:0/1068145277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4170f10 con 0x7f49c40615f0 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c88d8700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f49b400ed20 tx=0x7f49b4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:38.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400b530 con 0x7f49c40615f0 2020-11-16T22:27:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400cb00 con 0x7f49c40615f0 2020-11-16T22:27:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400ce80 con 0x7f49c40615f0 2020-11-16T22:27:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c41711b0 con 0x7f49c40615f0 2020-11-16T22:27:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4170320 con 0x7f49c40615f0 2020-11-16T22:27:38.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f49b4018070 con 0x7f49c40615f0 2020-11-16T22:27:38.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 0x7f499c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:38.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c40bac70 con 0x7f49c40615f0 2020-11-16T22:27:38.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.865+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49b406dda0 con 0x7f49c40615f0 2020-11-16T22:27:38.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.873+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f49b4044000 con 0x7f49c40615f0 2020-11-16T22:27:38.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.893+0000 7f49c3fff700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 0x7f499c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:38.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:38.893+0000 7f49c3fff700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 0x7f499c057550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f49c4061a50 tx=0x7f49b8013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.025+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49c4061330 con 0x7f49c40615f0 2020-11-16T22:27:39.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.025+0000 7f49c0ff9700 1 -- 172.21.15.181:0/1068145277 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f49b4047620 con 0x7f49c40615f0 2020-11-16T22:27:39.011 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:39.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 msgr2=0x7f499c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 0x7f499c057550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f49c4061a50 tx=0x7f49b8013000).stop 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 msgr2=0x7f49c4081170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f49b400ed20 tx=0x7f49b4008040).stop 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 shutdown_connections 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f499c055090 0x7f499c057550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c415f5b0 0x7f49c416fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49c40615f0 0x7f49c4081170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 --2- 172.21.15.181:0/1068145277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49c40622a0 0x7f49c4081660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 >> 172.21.15.181:0/1068145277 conn(0x7f49c40d69a0 msgr2=0x7f49c4061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:39.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 shutdown_connections 2020-11-16T22:27:39.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.029+0000 7f49cab3c700 1 -- 172.21.15.181:0/1068145277 wait complete. 2020-11-16T22:27:39.037 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:27:39.038 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2020-11-16T22:27:39.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 Processor -- start 2020-11-16T22:27:39.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 -- start start 2020-11-16T22:27:39.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 0x7fdddc065900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdddc063f30 0x7fdddc167640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc164180 con 0x7fdddc062180 2020-11-16T22:27:39.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc167c90 con 0x7fdddc063f30 2020-11-16T22:27:39.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc167f30 con 0x7fdddc062560 2020-11-16T22:27:39.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fddd3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdddc063f30 0x7fdddc167640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37186/0 (socket says 172.21.15.181:37186) 2020-11-16T22:27:39.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 -- 172.21.15.181:0/1267588931 learned_addr learned my addr 172.21.15.181:0/1267588931 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:39.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde0fa4700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 0x7fdddc065900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 -- 172.21.15.181:0/1267588931 >> v2:172.21.15.181:3300/0 conn(0x7fdddc063f30 msgr2=0x7fdddc167640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:39.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.181:3300/0 conn(0x7fdddc063f30 0x7fdddc167640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 -- 172.21.15.181:0/1267588931 >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 msgr2=0x7fdddc065900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:39.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 0x7fdddc065900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 -- 172.21.15.181:0/1267588931 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc1681d0 con 0x7fdddc062560 2020-11-16T22:27:39.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde0fa4700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 0x7fdddc065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde17a5700 1 --2- 172.21.15.181:0/1267588931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdddc065df0 tx=0x7fddd4009290).ready entity=mon.2 client_cookie=791d4a7dff97c6e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fddd37fe700 1 -- 172.21.15.181:0/1267588931 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400bc50 con 0x7fdddc062560 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fddd37fe700 1 -- 172.21.15.181:0/1267588931 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd400bdb0 con 0x7fdddc062560 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fddd37fe700 1 -- 172.21.15.181:0/1267588931 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011610 con 0x7fdddc062560 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 -- 172.21.15.181:0/1267588931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 msgr2=0x7fdddc164850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.177+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/1267588931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdddc065df0 tx=0x7fddd4009290).stop 2020-11-16T22:27:39.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/1267588931 shutdown_connections 2020-11-16T22:27:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/1267588931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc164850 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.175:3300/0 conn(0x7fdddc062180 0x7fdddc065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/1267588931 >> v2:172.21.15.181:3300/0 conn(0x7fdddc063f30 0x7fdddc167640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/1267588931 >> 172.21.15.181:0/1267588931 conn(0x7fdddc0d2310 msgr2=0x7fdddc0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/1267588931 shutdown_connections 2020-11-16T22:27:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/1267588931 wait complete. 2020-11-16T22:27:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 Processor -- start 2020-11-16T22:27:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- start start 2020-11-16T22:27:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 0x7fdddc0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 0x7fdddc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cc490 con 0x7fdddc063f30 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cb770 con 0x7fdddc062180 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cba10 con 0x7fdddc062560 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde0fa4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 0x7fdddc0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde0fa4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 0x7fdddc0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57580/0 (socket says 172.21.15.181:57580) 2020-11-16T22:27:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde0fa4700 1 -- 172.21.15.181:0/2379201651 learned_addr learned my addr 172.21.15.181:0/2379201651 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde17a5700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 0x7fdddc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 -- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 msgr2=0x7fdddc0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 0x7fdddc0caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 -- 172.21.15.181:0/2379201651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 msgr2=0x7fdddc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 0x7fdddc0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 -- 172.21.15.181:0/2379201651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0cbcb0 con 0x7fdddc062560 2020-11-16T22:27:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde17a5700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 0x7fdddc0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd3fff700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc0ccbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fddcc00ebd0 tx=0x7fddcc006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00c820 con 0x7fdddc062560 2020-11-16T22:27:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc0cbf50 con 0x7fdddc062560 2020-11-16T22:27:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc00c980 con 0x7fdddc062560 2020-11-16T22:27:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00cd20 con 0x7fdddc062560 2020-11-16T22:27:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.181+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc0cd5c0 con 0x7fdddc062560 2020-11-16T22:27:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc044f60 con 0x7fdddc062560 2020-11-16T22:27:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fddcc012600 con 0x7fdddc062560 2020-11-16T22:27:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fddd0ff9700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 0x7fddb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fddcc06ebf0 con 0x7fdddc062560 2020-11-16T22:27:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fdde0fa4700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 0x7fddb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.185+0000 7fdde0fa4700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 0x7fddb4057520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fddd800b6a0 tx=0x7fddd8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.189+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fddcc044e50 con 0x7fdddc062560 2020-11-16T22:27:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.393+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7fdddc0c8ad0 con 0x7fdddc062560 2020-11-16T22:27:39.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.717+0000 7fddd0ff9700 1 -- 172.21.15.181:0/2379201651 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v57) v1 ==== 139+0+0 (secure 0 0 0) 0x7fddcc048470 con 0x7fdddc062560 2020-11-16T22:27:39.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 msgr2=0x7fddb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 0x7fddb4057520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fddd800b6a0 tx=0x7fddd8008040).stop 2020-11-16T22:27:39.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 msgr2=0x7fdddc0ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc0ccbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fddcc00ebd0 tx=0x7fddcc006df0).stop 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 shutdown_connections 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fddb4055060 0x7fddb4057520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdddc063f30 0x7fdddc0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc062180 0x7fdddc0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 --2- 172.21.15.181:0/2379201651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdddc062560 0x7fdddc0ccbe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 >> 172.21.15.181:0/2379201651 conn(0x7fdddc0d2310 msgr2=0x7fdddc15bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 shutdown_connections 2020-11-16T22:27:39.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.721+0000 7fdde1fa6700 1 -- 172.21.15.181:0/2379201651 wait complete. 2020-11-16T22:27:39.722 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:27:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.869+0000 7fd933fff700 1 Processor -- start 2020-11-16T22:27:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.869+0000 7fd933fff700 1 -- start start 2020-11-16T22:27:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd933fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd934160690 0x7fd934066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd933fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93415e2e0 con 0x7fd93415de80 2020-11-16T22:27:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd933fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9340ce460 con 0x7fd9340b2190 2020-11-16T22:27:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd933fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd934066a60 con 0x7fd934160690 2020-11-16T22:27:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd932ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd9340b2190 0x7fd9340cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43658/0 (socket says 172.21.15.181:43658) 2020-11-16T22:27:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd932ffd700 1 -- 172.21.15.181:0/4188943608 learned_addr learned my addr 172.21.15.181:0/4188943608 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 --2- 172.21.15.181:0/4188943608 >> v2:172.21.15.175:3300/0 conn(0x7fd934160690 0x7fd934066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095025133 0 0) 0x7fd9340ce460 con 0x7fd9340b2190 2020-11-16T22:27:39.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd920003240 con 0x7fd9340b2190 2020-11-16T22:27:39.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307138756 0 0) 0x7fd93415e2e0 con 0x7fd93415de80 2020-11-16T22:27:39.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9340ce460 con 0x7fd93415de80 2020-11-16T22:27:39.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3695830334 0 0) 0x7fd920003240 con 0x7fd9340b2190 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 >> v2:172.21.15.175:3300/0 conn(0x7fd934160690 msgr2=0x7fd934066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 --2- 172.21.15.181:0/4188943608 >> v2:172.21.15.175:3300/0 conn(0x7fd934160690 0x7fd934066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 >> v1:172.21.15.175:6789/0 conn(0x7fd93415de80 legacy=0x7fd93415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340ce700 con 0x7fd9340b2190 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7fd9240014e0 con 0x7fd9340b2190 2020-11-16T22:27:39.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 >> v1:172.21.15.181:6789/0 conn(0x7fd9340b2190 legacy=0x7fd9340cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9200038c0 0x7fd920003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9200044e0 0x7fd920004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd92000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd934066a60 con 0x7fd9200044e0 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200053b0 con 0x7fd920004e50 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92000bca0 con 0x7fd9200038c0 2020-11-16T22:27:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd932ffd700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9200038c0 0x7fd920003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd92000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9327fc700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9200044e0 0x7fd920004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 -- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9200038c0 msgr2=0x7fd920003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9200038c0 0x7fd920003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 -- 172.21.15.181:0/4188943608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9200044e0 msgr2=0x7fd920004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9200044e0 0x7fd920004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 -- 172.21.15.181:0/4188943608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920003240 con 0x7fd920004e50 2020-11-16T22:27:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 -- 172.21.15.181:0/4188943608 reap_dead start 2020-11-16T22:27:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd9337fe700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd92000b730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd92c009e90 tx=0x7fd92c0091d0).ready entity=mon.1 client_cookie=bb593e1a52965086 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.873+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00ec40 con 0x7fd920004e50 2020-11-16T22:27:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00eda0 con 0x7fd920004e50 2020-11-16T22:27:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd931ffb700 1 -- 172.21.15.181:0/4188943608 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c012620 con 0x7fd920004e50 2020-11-16T22:27:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 msgr2=0x7fd92000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd92000b730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd92c009e90 tx=0x7fd92c0091d0).stop 2020-11-16T22:27:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- 172.21.15.181:0/4188943608 shutdown_connections 2020-11-16T22:27:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 --2- 172.21.15.181:0/4188943608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd92000b730 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- 172.21.15.181:0/4188943608 >> 172.21.15.181:0/4188943608 conn(0x7fd9340cfe90 msgr2=0x7fd9340d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- 172.21.15.181:0/4188943608 shutdown_connections 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- 172.21.15.181:0/4188943608 wait complete. 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 Processor -- start 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.877+0000 7fd933fff700 1 -- start start 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 0x7fd93416e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93416ee60 0x7fd93416cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93415ac70 con 0x7fd93416ee60 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340d2170 con 0x7fd920004e50 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340d2410 con 0x7fd9340b27f0 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57598/0 (socket says 172.21.15.181:57598) 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd9327fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 0x7fd93416e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 -- 172.21.15.181:0/147004539 learned_addr learned my addr 172.21.15.181:0/147004539 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd9337fe700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93416ee60 0x7fd93416cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 -- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 msgr2=0x7fd93416e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 0x7fd93416e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 -- 172.21.15.181:0/147004539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93416ee60 msgr2=0x7fd93416cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93416ee60 0x7fd93416cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 -- 172.21.15.181:0/147004539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340d26b0 con 0x7fd920004e50 2020-11-16T22:27:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd9327fc700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 0x7fd93416e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd932ffd700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd924004b00 tx=0x7fd93415a510).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400f840 con 0x7fd920004e50 2020-11-16T22:27:39.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92400f9a0 con 0x7fd920004e50 2020-11-16T22:27:39.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9340d2950 con 0x7fd920004e50 2020-11-16T22:27:39.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400fd60 con 0x7fd920004e50 2020-11-16T22:27:39.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.881+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93416c770 con 0x7fd920004e50 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd934044f60 con 0x7fd920004e50 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd92401b030 con 0x7fd920004e50 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd91affd700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 0x7fd904057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd9327fc700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 0x7fd904057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd92406d3b0 con 0x7fd920004e50 2020-11-16T22:27:39.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.885+0000 7fd9327fc700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 0x7fd904057520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd928007ad0 tx=0x7fd928007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:39.893+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd924043610 con 0x7fd920004e50 2020-11-16T22:27:40.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.049+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fd93415a030 con 0x7fd920004e50 2020-11-16T22:27:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd91affd700 1 -- 172.21.15.181:0/147004539 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd924019030 con 0x7fd920004e50 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 msgr2=0x7fd904057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 0x7fd904057520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd928007ad0 tx=0x7fd928007b00).stop 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 msgr2=0x7fd93415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd924004b00 tx=0x7fd93415a510).stop 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 shutdown_connections 2020-11-16T22:27:40.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd904055060 0x7fd904057520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93416ee60 0x7fd93416cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd920004e50 0x7fd93415a780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 --2- 172.21.15.181:0/147004539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9340b27f0 0x7fd93416e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 >> 172.21.15.181:0/147004539 conn(0x7fd9340cfe90 msgr2=0x7fd9341456e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 shutdown_connections 2020-11-16T22:27:40.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.061+0000 7fd933fff700 1 -- 172.21.15.181:0/147004539 wait complete. 2020-11-16T22:27:40.063 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:27:40.064 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:27:40.099 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:27:40.124 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:27:40.248 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:27:40.249 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:27:40.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:27:40.249 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2020-11-16T22:27:40.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:27:40.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:40.332 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:27:40.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-16T22:27:40.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:27:40.359 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2020-11-16T22:27:40.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:27:40.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:27:40.422 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-16T22:27:40.440 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:27:40.440 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:27:40.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 Processor -- start 2020-11-16T22:27:40.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- start start 2020-11-16T22:27:40.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dc415de80 0x7f5dc40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:40.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5dc415ff80 0x7f5dc4160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:40.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc415e2e0 con 0x7f5dc40b2190 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40b2a60 con 0x7f5dc415ff80 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40ce460 con 0x7f5dc415de80 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dc415de80 0x7f5dc40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dca543700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5dc40b2190 0x7f5dc40cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43678/0 (socket says 172.21.15.181:43678) 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dca543700 1 -- 172.21.15.181:0/108635115 learned_addr learned my addr 172.21.15.181:0/108635115 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:40.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcad44700 1 --2- 172.21.15.181:0/108635115 >> v2:172.21.15.175:3300/0 conn(0x7f5dc415ff80 0x7f5dc4160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 -- 172.21.15.181:0/108635115 >> v2:172.21.15.175:3300/0 conn(0x7f5dc415ff80 msgr2=0x7f5dc4160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 --2- 172.21.15.181:0/108635115 >> v2:172.21.15.175:3300/0 conn(0x7f5dc415ff80 0x7f5dc4160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 -- 172.21.15.181:0/108635115 >> v1:172.21.15.181:6789/0 conn(0x7f5dc40b2190 legacy=0x7f5dc40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 -- 172.21.15.181:0/108635115 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40ce700 con 0x7f5dc415de80 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9541700 1 -- 172.21.15.181:0/108635115 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862792643 0 0) 0x7f5dc415e2e0 con 0x7f5dc40b2190 2020-11-16T22:27:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9d42700 1 --2- 172.21.15.181:0/108635115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415de80 0x7f5dc40b2570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f5dc4064d90 tx=0x7f5dc0006df0).ready entity=mon.1 client_cookie=5bd197584b096ddd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9541700 1 -- 172.21.15.181:0/108635115 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00097b0 con 0x7f5dc415de80 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9541700 1 -- 172.21.15.181:0/108635115 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0009910 con 0x7f5dc415de80 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dc9541700 1 -- 172.21.15.181:0/108635115 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0009c90 con 0x7f5dc415de80 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- 172.21.15.181:0/108635115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415de80 msgr2=0x7f5dc40b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 --2- 172.21.15.181:0/108635115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415de80 0x7f5dc40b2570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f5dc4064d90 tx=0x7f5dc0006df0).stop 2020-11-16T22:27:40.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- 172.21.15.181:0/108635115 shutdown_connections 2020-11-16T22:27:40.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 --2- 172.21.15.181:0/108635115 >> v2:172.21.15.175:3300/0 conn(0x7f5dc415ff80 0x7f5dc4160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 --2- 172.21.15.181:0/108635115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415de80 0x7f5dc40b2570 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- 172.21.15.181:0/108635115 >> 172.21.15.181:0/108635115 conn(0x7f5dc40cfe90 msgr2=0x7f5dc40d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:40.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- 172.21.15.181:0/108635115 shutdown_connections 2020-11-16T22:27:40.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.589+0000 7f5dcb545700 1 -- 172.21.15.181:0/108635115 wait complete. 2020-11-16T22:27:40.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.593+0000 7f5dcb545700 1 Processor -- start 2020-11-16T22:27:40.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.593+0000 7f5dcb545700 1 -- start start 2020-11-16T22:27:40.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc40b2190 0x7f5dc4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:40.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc415de80 0x7f5dc4060c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:40.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:40.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4158090 con 0x7f5dc40b2190 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4158330 con 0x7f5dc415ff80 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41585d0 con 0x7f5dc415de80 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dc9d42700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc415de80 0x7f5dc4060c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57614/0 (socket says 172.21.15.181:57614) 2020-11-16T22:27:40.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 -- 172.21.15.181:0/2680919233 learned_addr learned my addr 172.21.15.181:0/2680919233 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:40.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dca543700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc40b2190 0x7f5dc4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:40.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 -- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc415de80 msgr2=0x7f5dc4060c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc415de80 0x7f5dc4060c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 -- 172.21.15.181:0/2680919233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc40b2190 msgr2=0x7f5dc4157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:40.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc40b2190 0x7f5dc4157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:40.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 -- 172.21.15.181:0/2680919233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4158870 con 0x7f5dc415ff80 2020-11-16T22:27:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcad44700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f5dbc00ef20 tx=0x7f5dbc0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:41.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc009690 con 0x7f5dc415ff80 2020-11-16T22:27:41.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00cc50 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4158b10 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0123f0 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc4157d10 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4158fb0 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.597+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5dbc022070 con 0x7f5dc415ff80 2020-11-16T22:27:41.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.601+0000 7f5db67fc700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 0x7f5d9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:41.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.601+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5dbc06e660 con 0x7f5dc415ff80 2020-11-16T22:27:41.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.601+0000 7f5dca543700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 0x7f5d9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:41.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.601+0000 7f5dca543700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 0x7f5d9c057520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f5db8007160 tx=0x7f5db8007960).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:41.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.609+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5dc4158fb0 con 0x7f5dc415ff80 2020-11-16T22:27:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.813+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dc4158fb0 con 0x7f5dc415ff80 2020-11-16T22:27:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.813+0000 7f5db67fc700 1 -- 172.21.15.181:0/2680919233 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f5dc4158fb0 con 0x7f5dc415ff80 2020-11-16T22:27:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 msgr2=0x7f5d9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 0x7f5d9c057520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f5db8007160 tx=0x7f5db8007960).stop 2020-11-16T22:27:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 msgr2=0x7f5dc40610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f5dbc00ef20 tx=0x7f5dbc0091d0).stop 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 shutdown_connections 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5d9c055060 0x7f5d9c057520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc415ff80 0x7f5dc40610f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc40b2190 0x7f5dc4157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 --2- 172.21.15.181:0/2680919233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc415de80 0x7f5dc4060c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 >> 172.21.15.181:0/2680919233 conn(0x7f5dc40cfe90 msgr2=0x7f5dc40618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 shutdown_connections 2020-11-16T22:27:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:40.817+0000 7f5dcb545700 1 -- 172.21.15.181:0/2680919233 wait complete. 2020-11-16T22:27:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 22 2020-11-16T22:27:41.406 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:41.610 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:27:02.107409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2,3,4,5,6,7,8],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299,"mds_3":4296,"mds_4":4276,"mds_5":4266,"mds_6":4256,"mds_7":4253,"mds_8":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":8,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4253":{"gid":4253,"name":"g","rank":7,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4256":{"gid":4256,"name":"a","rank":6,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4266":{"gid":4266,"name":"e","rank":5,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4276":{"gid":4276,"name":"b","rank":4,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4296":{"gid":4296,"name":"h","rank":3,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:27:41.814 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 Processor -- start 2020-11-16T22:27:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 -- start start 2020-11-16T22:27:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa35c063620 0x7fa35c063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c062700 con 0x7fa35c160530 2020-11-16T22:27:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0629a0 con 0x7fa35c063620 2020-11-16T22:27:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c062c40 con 0x7fa35c142660 2020-11-16T22:27:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa35c160530 0x7fa35c061810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43696/0 (socket says 172.21.15.181:43696) 2020-11-16T22:27:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35ad9d700 1 -- 172.21.15.181:0/2782361016 learned_addr learned my addr 172.21.15.181:0/2782361016 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.961+0000 7fa35a59c700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 --2- 172.21.15.181:0/2782361016 >> v2:172.21.15.175:3300/0 conn(0x7fa35c063620 0x7fa35c063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501510450 0 0) 0x7fa35c062700 con 0x7fa35c160530 2020-11-16T22:27:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa340003240 con 0x7fa35c160530 2020-11-16T22:27:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 -- 172.21.15.181:0/2782361016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 msgr2=0x7fa35c142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3910507635 0 0) 0x7fa340003240 con 0x7fa35c160530 2020-11-16T22:27:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 -- 172.21.15.181:0/2782361016 >> v1:172.21.15.181:6789/0 conn(0x7fa35c160530 legacy=0x7fa35c061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 -- 172.21.15.181:0/2782361016 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c062ee0 con 0x7fa35c063620 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35a59c700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa359d9b700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c063ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fa34400b9a0 tx=0x7fa344006df0).ready entity=mon.0 client_cookie=ce3d92369fa18b4b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354001860 con 0x7fa35c063620 2020-11-16T22:27:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344011400 con 0x7fa35c063620 2020-11-16T22:27:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35959a700 1 -- 172.21.15.181:0/2782361016 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344011780 con 0x7fa35c063620 2020-11-16T22:27:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 -- 172.21.15.181:0/2782361016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 msgr2=0x7fa35c063ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c063ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fa34400b9a0 tx=0x7fa344006df0).stop 2020-11-16T22:27:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 -- 172.21.15.181:0/2782361016 shutdown_connections 2020-11-16T22:27:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 --2- 172.21.15.181:0/2782361016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c063ec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 -- 172.21.15.181:0/2782361016 >> 172.21.15.181:0/2782361016 conn(0x7fa35c0d2dc0 msgr2=0x7fa35c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:42.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 -- 172.21.15.181:0/2782361016 shutdown_connections 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.965+0000 7fa35b59e700 1 -- 172.21.15.181:0/2782361016 wait complete. 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.969+0000 7fa35b59e700 1 Processor -- start 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.969+0000 7fa35b59e700 1 -- start start 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:42.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c16f070 con 0x7fa35c063620 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c13ec20 con 0x7fa35c160530 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c13eec0 con 0x7fa35c142660 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa359d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57630/0 (socket says 172.21.15.181:57630) 2020-11-16T22:27:42.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 -- 172.21.15.181:0/3009755852 learned_addr learned my addr 172.21.15.181:0/3009755852 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:42.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35a59c700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 -- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 msgr2=0x7fa35c13e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c13e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 -- 172.21.15.181:0/3009755852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 msgr2=0x7fa35c142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 -- 172.21.15.181:0/3009755852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c13f160 con 0x7fa35c160530 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa359d9b700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35ad9d700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa35400cb30 tx=0x7fa354001570).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354004a90 con 0x7fa35c160530 2020-11-16T22:27:42.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa354004bf0 con 0x7fa35c160530 2020-11-16T22:27:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354007170 con 0x7fa35c160530 2020-11-16T22:27:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c16f2d0 con 0x7fa35c160530 2020-11-16T22:27:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c15f0f0 con 0x7fa35c160530 2020-11-16T22:27:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa354010070 con 0x7fa35c160530 2020-11-16T22:27:42.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c13ea50 con 0x7fa35c160530 2020-11-16T22:27:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.973+0000 7fa3527fc700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 0x7fa32c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.977+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa35406da20 con 0x7fa35c160530 2020-11-16T22:27:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.977+0000 7fa35a59c700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 0x7fa32c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.977+0000 7fa35a59c700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 0x7fa32c057520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fa34c006fd0 tx=0x7fa34c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:41.981+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa354043c20 con 0x7fa35c160530 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35c16f460 con 0x7fa35c160530 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa3527fc700 1 -- 172.21.15.181:0/3009755852 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fa354047240 con 0x7fa35c160530 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 msgr2=0x7fa32c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 0x7fa32c057520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fa34c006fd0 tx=0x7fa34c006df0).stop 2020-11-16T22:27:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 msgr2=0x7fa35c143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa35400cb30 tx=0x7fa354001570).stop 2020-11-16T22:27:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 shutdown_connections 2020-11-16T22:27:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa32c055060 0x7fa32c057520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa35c160530 0x7fa35c143b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa35c063620 0x7fa35c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 --2- 172.21.15.181:0/3009755852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c142660 0x7fa35c13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 >> 172.21.15.181:0/3009755852 conn(0x7fa35c0d2dc0 msgr2=0x7fa35c0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 shutdown_connections 2020-11-16T22:27:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:42.137+0000 7fa35b59e700 1 -- 172.21.15.181:0/3009755852 wait complete. 2020-11-16T22:27:42.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:42.975 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:43.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:26:49.798965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:26:51.012444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:27:44.198 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:27:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.349+0000 7ff14286f700 1 Processor -- start 2020-11-16T22:27:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.349+0000 7ff14286f700 1 -- start start 2020-11-16T22:27:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff13c062790 0x7ff13c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c160e40 con 0x7ff13c15f5b0 2020-11-16T22:27:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c1609a0 con 0x7ff13c062790 2020-11-16T22:27:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c160ba0 con 0x7ff13c0615f0 2020-11-16T22:27:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff13c15f5b0 0x7ff13c0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43706/0 (socket says 172.21.15.181:43706) 2020-11-16T22:27:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13bfff700 1 -- 172.21.15.181:0/3270906470 learned_addr learned my addr 172.21.15.181:0/3270906470 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff140e0c700 1 --2- 172.21.15.181:0/3270906470 >> v2:172.21.15.175:3300/0 conn(0x7ff13c062790 0x7ff13c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13affd700 1 -- 172.21.15.181:0/3270906470 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436326654 0 0) 0x7ff13c160e40 con 0x7ff13c15f5b0 2020-11-16T22:27:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13affd700 1 -- 172.21.15.181:0/3270906470 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c003240 con 0x7ff13c15f5b0 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 -- 172.21.15.181:0/3270906470 >> v2:172.21.15.175:3300/0 conn(0x7ff13c062790 msgr2=0x7ff13c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 --2- 172.21.15.181:0/3270906470 >> v2:172.21.15.175:3300/0 conn(0x7ff13c062790 0x7ff13c080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 -- 172.21.15.181:0/3270906470 >> v1:172.21.15.181:6789/0 conn(0x7ff13c15f5b0 legacy=0x7ff13c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 -- 172.21.15.181:0/3270906470 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c060970 con 0x7ff13c0615f0 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13b7fe700 1 --2- 172.21.15.181:0/3270906470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0622a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff124009c90 tx=0x7ff1240091d0).ready entity=mon.2 client_cookie=45843c56431ac0c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13affd700 1 -- 172.21.15.181:0/3270906470 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400b9c0 con 0x7ff13c0615f0 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13affd700 1 -- 172.21.15.181:0/3270906470 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12400bb20 con 0x7ff13c0615f0 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff13affd700 1 -- 172.21.15.181:0/3270906470 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400bea0 con 0x7ff13c0615f0 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- 172.21.15.181:0/3270906470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 msgr2=0x7ff13c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 --2- 172.21.15.181:0/3270906470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0622a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff124009c90 tx=0x7ff1240091d0).stop 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- 172.21.15.181:0/3270906470 shutdown_connections 2020-11-16T22:27:44.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 --2- 172.21.15.181:0/3270906470 >> v2:172.21.15.175:3300/0 conn(0x7ff13c062790 0x7ff13c080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 --2- 172.21.15.181:0/3270906470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0622a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- 172.21.15.181:0/3270906470 >> 172.21.15.181:0/3270906470 conn(0x7ff13c0d69a0 msgr2=0x7ff13c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:44.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- 172.21.15.181:0/3270906470 shutdown_connections 2020-11-16T22:27:44.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.353+0000 7ff14286f700 1 -- 172.21.15.181:0/3270906470 wait complete. 2020-11-16T22:27:44.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.357+0000 7ff14286f700 1 Processor -- start 2020-11-16T22:27:44.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.357+0000 7ff14286f700 1 -- start start 2020-11-16T22:27:44.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c062790 0x7ff13c0b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 0x7ff13c164550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1728b0 con 0x7ff13c062790 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c173760 con 0x7ff13c15f5b0 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c165230 con 0x7ff13c0615f0 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff13bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff13bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37250/0 (socket says 172.21.15.181:37250) 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff13bfff700 1 -- 172.21.15.181:0/3703166791 learned_addr learned my addr 172.21.15.181:0/3703166791 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:44.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 0x7ff13c164550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff13b7fe700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c062790 0x7ff13c0b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 -- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 msgr2=0x7ff13c0b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 -- 172.21.15.181:0/3703166791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c062790 msgr2=0x7ff13c0b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c062790 0x7ff13c0b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 -- 172.21.15.181:0/3703166791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c172ae0 con 0x7ff13c15f5b0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff140e0c700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 0x7ff13c164550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff13400e950 tx=0x7ff134006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400c6b0 con 0x7ff13c15f5b0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13400c810 con 0x7ff13c15f5b0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c172d80 con 0x7ff13c15f5b0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400cbd0 con 0x7ff13c15f5b0 2020-11-16T22:27:44.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c1732c0 con 0x7ff13c15f5b0 2020-11-16T22:27:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c060b50 con 0x7ff13c15f5b0 2020-11-16T22:27:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.361+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff13401a030 con 0x7ff13c15f5b0 2020-11-16T22:27:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.365+0000 7ff133fff700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 0x7ff118057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.365+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff13406d8e0 con 0x7ff13c15f5b0 2020-11-16T22:27:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.365+0000 7ff13bfff700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 0x7ff118057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.365+0000 7ff13bfff700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 0x7ff118057520 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7ff12c000bc0 tx=0x7ff12c0011e0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.369+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff134043b40 con 0x7ff13c15f5b0 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff13c173450 con 0x7ff13c15f5b0 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff133fff700 1 -- 172.21.15.181:0/3703166791 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff134047160 con 0x7ff13c15f5b0 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 msgr2=0x7ff118057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 0x7ff118057520 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7ff12c000bc0 tx=0x7ff12c0011e0).stop 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 msgr2=0x7ff13c164550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 0x7ff13c164550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff13400e950 tx=0x7ff134006df0).stop 2020-11-16T22:27:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 shutdown_connections 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff118055060 0x7ff118057520 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13c15f5b0 0x7ff13c164550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff13c0615f0 0x7ff13c0b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 --2- 172.21.15.181:0/3703166791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c062790 0x7ff13c0b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.525+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 >> 172.21.15.181:0/3703166791 conn(0x7ff13c0d69a0 msgr2=0x7ff13c0d5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.529+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 shutdown_connections 2020-11-16T22:27:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.529+0000 7ff14286f700 1 -- 172.21.15.181:0/3703166791 wait complete. 2020-11-16T22:27:44.557 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:27:44.557 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:27:44.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 Processor -- start 2020-11-16T22:27:44.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 -- start start 2020-11-16T22:27:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34a40615f0 0x7f34a40622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34a4062790 0x7f34a4080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a4160e40 con 0x7f34a415f5b0 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a41609a0 con 0x7f34a4062790 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34ab570700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a4160ba0 con 0x7f34a40615f0 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34a40615f0 0x7f34a40622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34a40615f0 0x7f34a40622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57650/0 (socket says 172.21.15.181:57650) 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 -- 172.21.15.181:0/1299754116 learned_addr learned my addr 172.21.15.181:0/1299754116 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a9b0d700 1 --2- 172.21.15.181:0/1299754116 >> v2:172.21.15.175:3300/0 conn(0x7f34a4062790 0x7f34a4080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 -- 172.21.15.181:0/1299754116 >> v2:172.21.15.175:3300/0 conn(0x7f34a4062790 msgr2=0x7f34a4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 --2- 172.21.15.181:0/1299754116 >> v2:172.21.15.175:3300/0 conn(0x7f34a4062790 0x7f34a4080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 -- 172.21.15.181:0/1299754116 >> v1:172.21.15.175:6789/0 conn(0x7f34a415f5b0 legacy=0x7f34a40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 -- 172.21.15.181:0/1299754116 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4060970 con 0x7f34a40615f0 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f34a8b0b700 1 --2- 172.21.15.181:0/1299754116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a40622a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f34a40bcb90 tx=0x7f3494006eb0).ready entity=mon.1 client_cookie=5994408841e1e371 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.705+0000 7f3493fff700 1 -- 172.21.15.181:0/1299754116 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494009810 con 0x7f34a40615f0 2020-11-16T22:27:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f3493fff700 1 -- 172.21.15.181:0/1299754116 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494009970 con 0x7f34a40615f0 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f3493fff700 1 -- 172.21.15.181:0/1299754116 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494009cf0 con 0x7f34a40615f0 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- 172.21.15.181:0/1299754116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 msgr2=0x7f34a40622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- 172.21.15.181:0/1299754116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a40622a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f34a40bcb90 tx=0x7f3494006eb0).stop 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- 172.21.15.181:0/1299754116 shutdown_connections 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- 172.21.15.181:0/1299754116 >> v2:172.21.15.175:3300/0 conn(0x7f34a4062790 0x7f34a4080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- 172.21.15.181:0/1299754116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a40622a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- 172.21.15.181:0/1299754116 >> 172.21.15.181:0/1299754116 conn(0x7f34a40d69a0 msgr2=0x7f34a40d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:44.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- 172.21.15.181:0/1299754116 shutdown_connections 2020-11-16T22:27:44.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- 172.21.15.181:0/1299754116 wait complete. 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 Processor -- start 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- start start 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a4081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 0x7f34a40bb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a416f840 con 0x7f34a415f5b0 2020-11-16T22:27:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a416fae0 con 0x7f34a40615f0 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.709+0000 7f34ab570700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a416fd80 con 0x7f34a4062790 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37266/0 (socket says 172.21.15.181:37266) 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 -- 172.21.15.181:0/2706732979 learned_addr learned my addr 172.21.15.181:0/2706732979 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a930c700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a4081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a9b0d700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 0x7f34a40bb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 -- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 msgr2=0x7f34a4081170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a4081170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 -- 172.21.15.181:0/2706732979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 msgr2=0x7f34a40bb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 0x7f34a40bb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 -- 172.21.15.181:0/2706732979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4170020 con 0x7f34a4062790 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a9b0d700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 0x7f34a40bb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34a8b0b700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f349400b1b0 tx=0x7f3494015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494011d50 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349401e460 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a41702c0 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349401e7e0 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a40bb890 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a4069eb0 con 0x7f34a4062790 2020-11-16T22:27:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.713+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3494025030 con 0x7f34a4062790 2020-11-16T22:27:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.717+0000 7f34917fa700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 0x7f347c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.717+0000 7f34a930c700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 0x7f347c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.717+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f349406f610 con 0x7f34a4062790 2020-11-16T22:27:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.717+0000 7f34a930c700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 0x7f347c057520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f34a0003350 tx=0x7f34a0007af0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.721+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3494045780 con 0x7f34a4062790 2020-11-16T22:27:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.865+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34a4082860 con 0x7f34a4062790 2020-11-16T22:27:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.865+0000 7f34917fa700 1 -- 172.21.15.181:0/2706732979 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34940230a0 con 0x7f34a4062790 2020-11-16T22:27:44.851 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:44.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 msgr2=0x7f347c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 0x7f347c057520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f34a0003350 tx=0x7f34a0007af0).stop 2020-11-16T22:27:44.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 msgr2=0x7f34a4081660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f349400b1b0 tx=0x7f3494015000).stop 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 shutdown_connections 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f347c055060 0x7f347c057520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34a415f5b0 0x7f34a40bb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a40615f0 0x7f34a4081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 --2- 172.21.15.181:0/2706732979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a4062790 0x7f34a4081660 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 >> 172.21.15.181:0/2706732979 conn(0x7f34a40d69a0 msgr2=0x7f34a4061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 shutdown_connections 2020-11-16T22:27:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:44.869+0000 7f34ab570700 1 -- 172.21.15.181:0/2706732979 wait complete. 2020-11-16T22:27:44.866 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:27:44.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 Processor -- start 2020-11-16T22:27:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 -- start start 2020-11-16T22:27:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 0x7f12680660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12680cfbf0 con 0x7f12680cf770 2020-11-16T22:27:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1268065190 con 0x7f12680cf390 2020-11-16T22:27:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1268065430 con 0x7f1268060800 2020-11-16T22:27:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f12680cf390 0x7f1268163970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43734/0 (socket says 172.21.15.181:43734) 2020-11-16T22:27:44.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126659c700 1 -- 172.21.15.181:0/2613648710 learned_addr learned my addr 172.21.15.181:0/2613648710 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:44.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f1266d9d700 1 --2- 172.21.15.181:0/2613648710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 0x7f12680660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603267232 0 0) 0x7f1268065190 con 0x7f12680cf390 2020-11-16T22:27:44.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1248003240 con 0x7f12680cf390 2020-11-16T22:27:44.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611867623 0 0) 0x7f12680cfbf0 con 0x7f12680cf770 2020-11-16T22:27:44.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f1266d9d700 1 -- 172.21.15.181:0/2613648710 >> v1:172.21.15.181:6789/0 conn(0x7f12680cf390 legacy=0x7f1268163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f1266d9d700 1 -- 172.21.15.181:0/2613648710 >> v1:172.21.15.175:6789/0 conn(0x7f12680cf770 legacy=0x7f12680600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f1266d9d700 1 -- 172.21.15.181:0/2613648710 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680656d0 con 0x7f1268060800 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f1266d9d700 1 --2- 172.21.15.181:0/2613648710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 0x7f12680660b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f126000b9d0 tx=0x7f1260006df0).ready entity=mon.2 client_cookie=7559b4e3538074e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600097b0 con 0x7f1268060800 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260009910 con 0x7f1268060800 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.005+0000 7f126559a700 1 -- 172.21.15.181:0/2613648710 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260009c90 con 0x7f1268060800 2020-11-16T22:27:44.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- 172.21.15.181:0/2613648710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 msgr2=0x7f12680660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 --2- 172.21.15.181:0/2613648710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 0x7f12680660b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f126000b9d0 tx=0x7f1260006df0).stop 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- 172.21.15.181:0/2613648710 shutdown_connections 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 --2- 172.21.15.181:0/2613648710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1268060800 0x7f12680660b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- 172.21.15.181:0/2613648710 >> 172.21.15.181:0/2613648710 conn(0x7f12680cfe90 msgr2=0x7f12680d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- 172.21.15.181:0/2613648710 shutdown_connections 2020-11-16T22:27:44.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- 172.21.15.181:0/2613648710 wait complete. 2020-11-16T22:27:44.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 Processor -- start 2020-11-16T22:27:44.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.009+0000 7f126759e700 1 -- start start 2020-11-16T22:27:44.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1268060800 0x7f1268062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 0x7f12680631d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268172070 con 0x7f1268060800 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268171890 con 0x7f12680cf390 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1268171b30 con 0x7f12680cf770 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37280/0 (socket says 172.21.15.181:37280) 2020-11-16T22:27:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1265d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 0x7f12680631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:44.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 -- 172.21.15.181:0/622991737 learned_addr learned my addr 172.21.15.181:0/622991737 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:44.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126659c700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1268060800 0x7f1268062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 -- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 msgr2=0x7f12680631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 0x7f12680631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 -- 172.21.15.181:0/622991737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1268060800 msgr2=0x7f1268062ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1268060800 0x7f1268062ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 -- 172.21.15.181:0/622991737 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1268171dd0 con 0x7f12680cf770 2020-11-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1265d9b700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 0x7f12680631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f1266d9d700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f126000b920 tx=0x7f1260015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600097b0 con 0x7f12680cf770 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260009e40 con 0x7f12680cf770 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126001d800 con 0x7f12680cf770 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126815d970 con 0x7f12680cf770 2020-11-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126816fb10 con 0x7f12680cf770 2020-11-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.013+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268044f60 con 0x7f12680cf770 2020-11-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.017+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1260025030 con 0x7f12680cf770 2020-11-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.017+0000 7f125e7fc700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 0x7f1244057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.017+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f126006f670 con 0x7f12680cf770 2020-11-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.017+0000 7f126659c700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 0x7f1244057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.017+0000 7f126659c700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 0x7f1244057520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f1258005920 tx=0x7f1258009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.025+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f12600458d0 con 0x7f12680cf770 2020-11-16T22:27:45.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f126805c4a0 con 0x7f12680cf770 2020-11-16T22:27:45.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f125e7fc700 1 -- 172.21.15.181:0/622991737 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1260048ef0 con 0x7f12680cf770 2020-11-16T22:27:45.162 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:45.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 msgr2=0x7f1244057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 0x7f1244057520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f1258005920 tx=0x7f1258009040).stop 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 msgr2=0x7f126813ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f126000b920 tx=0x7f1260015000).stop 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 shutdown_connections 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1244055060 0x7f1244057520 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12680cf770 0x7f126813ef90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1268060800 0x7f1268062ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 --2- 172.21.15.181:0/622991737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12680cf390 0x7f12680631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 >> 172.21.15.181:0/622991737 conn(0x7f12680cfe90 msgr2=0x7f1268140320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:45.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 shutdown_connections 2020-11-16T22:27:45.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.177+0000 7f126759e700 1 -- 172.21.15.181:0/622991737 wait complete. 2020-11-16T22:27:45.177 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:27:45.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.321+0000 7fecadf22700 1 Processor -- start 2020-11-16T22:27:45.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.321+0000 7fecadf22700 1 -- start start 2020-11-16T22:27:45.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80c8db0 con 0x7feca8161860 2020-11-16T22:27:45.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80c8330 con 0x7feca8164e10 2020-11-16T22:27:45.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca80c85d0 con 0x7feca8161480 2020-11-16T22:27:45.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57676/0 (socket says 172.21.15.181:57676) 2020-11-16T22:27:45.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca7fff700 1 -- 172.21.15.181:0/2433417874 learned_addr learned my addr 172.21.15.181:0/2433417874 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:45.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 --2- 172.21.15.181:0/2433417874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 -- 172.21.15.181:0/2433417874 >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 msgr2=0x7feca8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:45.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 --2- 172.21.15.181:0/2433417874 >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 -- 172.21.15.181:0/2433417874 >> v1:172.21.15.181:6789/0 conn(0x7feca8161860 legacy=0x7feca8062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca77fe700 1 -- 172.21.15.181:0/2433417874 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058268800 0 0) 0x7feca80c8db0 con 0x7feca8161860 2020-11-16T22:27:45.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 -- 172.21.15.181:0/2433417874 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80c8870 con 0x7feca8161480 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca7fff700 1 --2- 172.21.15.181:0/2433417874 >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecacf20700 1 --2- 172.21.15.181:0/2433417874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca8164920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fec9800b9a0 tx=0x7fec98006df0).ready entity=mon.2 client_cookie=72af4adbd9fbf57a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca77fe700 1 -- 172.21.15.181:0/2433417874 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980097b0 con 0x7feca8161480 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca77fe700 1 -- 172.21.15.181:0/2433417874 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec98009910 con 0x7feca8161480 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7feca77fe700 1 -- 172.21.15.181:0/2433417874 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98009c90 con 0x7feca8161480 2020-11-16T22:27:45.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- 172.21.15.181:0/2433417874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 msgr2=0x7feca8164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 --2- 172.21.15.181:0/2433417874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca8164920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fec9800b9a0 tx=0x7fec98006df0).stop 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- 172.21.15.181:0/2433417874 shutdown_connections 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 --2- 172.21.15.181:0/2433417874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca8164920 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 --2- 172.21.15.181:0/2433417874 >> v2:172.21.15.181:3300/0 conn(0x7feca8164e10 0x7feca8062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- 172.21.15.181:0/2433417874 >> 172.21.15.181:0/2433417874 conn(0x7feca80d2dc0 msgr2=0x7feca80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- 172.21.15.181:0/2433417874 shutdown_connections 2020-11-16T22:27:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.325+0000 7fecadf22700 1 -- 172.21.15.181:0/2433417874 wait complete. 2020-11-16T22:27:45.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.329+0000 7fecadf22700 1 Processor -- start 2020-11-16T22:27:45.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.329+0000 7fecadf22700 1 -- start start 2020-11-16T22:27:45.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feca8161860 0x7feca8140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca8145000 0x7feca8145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca8145930 con 0x7feca8145000 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca8177d50 con 0x7feca8161860 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca81772d0 con 0x7feca8161480 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37292/0 (socket says 172.21.15.181:37292) 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 -- 172.21.15.181:0/1011472165 learned_addr learned my addr 172.21.15.181:0/1011472165 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7feca7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feca8161860 0x7feca8140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecad721700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca8145000 0x7feca8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 -- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feca8161860 msgr2=0x7feca8140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feca8161860 0x7feca8140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 -- 172.21.15.181:0/1011472165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca8145000 msgr2=0x7feca8145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca8145000 0x7feca8145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 -- 172.21.15.181:0/1011472165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8177570 con 0x7feca8161480 2020-11-16T22:27:45.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecacf20700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fec98006e30 tx=0x7fec98015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98011dd0 con 0x7feca8161480 2020-11-16T22:27:45.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9801e460 con 0x7feca8161480 2020-11-16T22:27:45.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca8177810 con 0x7feca8161480 2020-11-16T22:27:45.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9801e800 con 0x7feca8161480 2020-11-16T22:27:45.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca8144b60 con 0x7feca8161480 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca80cf7c0 con 0x7feca8161480 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.333+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fec98025030 con 0x7feca8161480 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.337+0000 7feca4ff9700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 0x7fec84057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.337+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec9806f260 con 0x7feca8161480 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.337+0000 7feca7fff700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 0x7fec84057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.337+0000 7feca7fff700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 0x7fec84057520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7feca8147d30 tx=0x7fec90009290).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.341+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7feca80cf7c0 con 0x7feca8161480 2020-11-16T22:27:45.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.493+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feca80cf7c0 con 0x7feca8161480 2020-11-16T22:27:45.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.493+0000 7feca4ff9700 1 -- 172.21.15.181:0/1011472165 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7feca80cf7c0 con 0x7feca8161480 2020-11-16T22:27:45.480 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:27:45.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 msgr2=0x7fec84057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 0x7fec84057520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7feca8147d30 tx=0x7fec90009290).stop 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 msgr2=0x7feca81654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fec98006e30 tx=0x7fec98015000).stop 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 shutdown_connections 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fec84055060 0x7fec84057520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca8145000 0x7feca8145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feca8161480 0x7feca81654c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 --2- 172.21.15.181:0/1011472165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feca8161860 0x7feca8140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 >> 172.21.15.181:0/1011472165 conn(0x7feca80d2dc0 msgr2=0x7feca8147370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:45.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 shutdown_connections 2020-11-16T22:27:45.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.497+0000 7fecadf22700 1 -- 172.21.15.181:0/1011472165 wait complete. 2020-11-16T22:27:45.494 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:27:45.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 Processor -- start 2020-11-16T22:27:45.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 -- start start 2020-11-16T22:27:45.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8a00612a0 0x7fc8a0061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a0061b70 con 0x7fc8a015fa10 2020-11-16T22:27:45.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a015cda0 con 0x7fc8a00ce750 2020-11-16T22:27:45.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc8a6562700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a015d040 con 0x7fc8a00612a0 2020-11-16T22:27:45.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc89f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc8a00ce750 0x7fc8a00cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43762/0 (socket says 172.21.15.181:43762) 2020-11-16T22:27:45.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc89f7fe700 1 -- 172.21.15.181:0/4270892935 learned_addr learned my addr 172.21.15.181:0/4270892935 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:45.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.629+0000 7fc89ffff700 1 --2- 172.21.15.181:0/4270892935 >> v2:172.21.15.175:3300/0 conn(0x7fc8a00612a0 0x7fc8a0061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:45.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822415245 0 0) 0x7fc8a015cda0 con 0x7fc8a00ce750 2020-11-16T22:27:45.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc884003240 con 0x7fc8a00ce750 2020-11-16T22:27:45.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980067218 0 0) 0x7fc8a0061b70 con 0x7fc8a015fa10 2020-11-16T22:27:45.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89ffff700 1 -- 172.21.15.181:0/4270892935 >> v1:172.21.15.181:6789/0 conn(0x7fc8a00ce750 legacy=0x7fc8a00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89ffff700 1 -- 172.21.15.181:0/4270892935 >> v1:172.21.15.175:6789/0 conn(0x7fc8a015fa10 legacy=0x7fc8a015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89ffff700 1 -- 172.21.15.181:0/4270892935 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a015d2e0 con 0x7fc8a00612a0 2020-11-16T22:27:45.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89ffff700 1 --2- 172.21.15.181:0/4270892935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00612a0 0x7fc8a0061680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fc89000b9a0 tx=0x7fc890006df0).ready entity=mon.0 client_cookie=df585c8246e4facf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:45.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900097b0 con 0x7fc8a00612a0 2020-11-16T22:27:45.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890009910 con 0x7fc8a00612a0 2020-11-16T22:27:45.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc89effd700 1 -- 172.21.15.181:0/4270892935 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890009c90 con 0x7fc8a00612a0 2020-11-16T22:27:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 -- 172.21.15.181:0/4270892935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00612a0 msgr2=0x7fc8a0061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4270892935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00612a0 0x7fc8a0061680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fc89000b9a0 tx=0x7fc890006df0).stop 2020-11-16T22:27:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 -- 172.21.15.181:0/4270892935 shutdown_connections 2020-11-16T22:27:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4270892935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00612a0 0x7fc8a0061680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 -- 172.21.15.181:0/4270892935 >> 172.21.15.181:0/4270892935 conn(0x7fc8a00cf890 msgr2=0x7fc8a00cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 -- 172.21.15.181:0/4270892935 shutdown_connections 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.633+0000 7fc8a6562700 1 -- 172.21.15.181:0/4270892935 wait complete. 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 Processor -- start 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 -- start start 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00ce750 0x7fc8a00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a015fa10 0x7fc8a00cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0158690 con 0x7fc8a00ce750 2020-11-16T22:27:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0157eb0 con 0x7fc8a00612a0 2020-11-16T22:27:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a6562700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0158150 con 0x7fc8a015fa10 2020-11-16T22:27:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57698/0 (socket says 172.21.15.181:57698) 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 -- 172.21.15.181:0/4127310772 learned_addr learned my addr 172.21.15.181:0/4127310772 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89f7fe700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00ce750 0x7fc8a00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc8a4aff700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a015fa10 0x7fc8a00cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 -- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a015fa10 msgr2=0x7fc8a00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a015fa10 0x7fc8a00cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 -- 172.21.15.181:0/4127310772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00ce750 msgr2=0x7fc8a00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00ce750 0x7fc8a00cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 -- 172.21.15.181:0/4127310772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a01583f0 con 0x7fc8a00612a0 2020-11-16T22:27:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.637+0000 7fc89ffff700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fc8a0067730 tx=0x7fc890015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:46.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89001e460 con 0x7fc8a00612a0 2020-11-16T22:27:46.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89001e5c0 con 0x7fc8a00612a0 2020-11-16T22:27:46.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a0159bc0 con 0x7fc8a00612a0 2020-11-16T22:27:46.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89001e940 con 0x7fc8a00612a0 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a0157b50 con 0x7fc8a00612a0 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a0066020 con 0x7fc8a00612a0 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc890025030 con 0x7fc8a00612a0 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 0x7fc870057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc89006f640 con 0x7fc8a00612a0 2020-11-16T22:27:46.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc89f7fe700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 0x7fc870057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.641+0000 7fc89f7fe700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 0x7fc870057520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc8a0156d60 tx=0x7fc888009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:46.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.649+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc8a0066020 con 0x7fc8a00612a0 2020-11-16T22:27:46.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8a00ce490 con 0x7fc8a00612a0 2020-11-16T22:27:46.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc897fff700 1 -- 172.21.15.181:0/4127310772 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fc8a00ce490 con 0x7fc8a00612a0 2020-11-16T22:27:46.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 msgr2=0x7fc870057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:46.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 0x7fc870057520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc8a0156d60 tx=0x7fc888009040).stop 2020-11-16T22:27:46.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 msgr2=0x7fc8a0157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:46.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.849+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fc8a0067730 tx=0x7fc890015000).stop 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 shutdown_connections 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc870055060 0x7fc870057520 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a015fa10 0x7fc8a00cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a00612a0 0x7fc8a0157660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 --2- 172.21.15.181:0/4127310772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8a00ce750 0x7fc8a00cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 >> 172.21.15.181:0/4127310772 conn(0x7fc8a00cf890 msgr2=0x7fc8a00b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 shutdown_connections 2020-11-16T22:27:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:45.853+0000 7fc8a6562700 1 -- 172.21.15.181:0/4127310772 wait complete. 2020-11-16T22:27:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 22 2020-11-16T22:27:46.444 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:46.444 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:26:51.998542+0000","modified":"2020-11-16T22:27:02.107409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":9,"in":[0,1,2,3,4,5,6,7,8],"up":{"mds_0":4305,"mds_1":4315,"mds_2":4299,"mds_3":4296,"mds_4":4276,"mds_5":4266,"mds_6":4256,"mds_7":4253,"mds_8":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":8,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6837/3313292089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3313292089},{"type":"v1","addr":"172.21.15.181:6837","nonce":3313292089}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4253":{"gid":4253,"name":"g","rank":7,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6835/228556183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":228556183},{"type":"v1","addr":"172.21.15.175:6835","nonce":228556183}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4256":{"gid":4256,"name":"a","rank":6,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6840/2424566790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2424566790},{"type":"v1","addr":"172.21.15.181:6840","nonce":2424566790}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4266":{"gid":4266,"name":"e","rank":5,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.175:6833/3836362112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3836362112},{"type":"v1","addr":"172.21.15.175:6833","nonce":3836362112}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4276":{"gid":4276,"name":"b","rank":4,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6835/3777546427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3777546427},{"type":"v1","addr":"172.21.15.181:6835","nonce":3777546427}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4296":{"gid":4296,"name":"h","rank":3,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6839/390679745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":390679745},{"type":"v1","addr":"172.21.15.175:6839","nonce":390679745}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4299":{"gid":4299,"name":"f","rank":2,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6838/2983366267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2983366267},{"type":"v1","addr":"172.21.15.175:6838","nonce":2983366267}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4305":{"gid":4305,"name":"i","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.175:6841/1137266867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":1137266867},{"type":"v1","addr":"172.21.15.175:6841","nonce":1137266867}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4315":{"gid":4315,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3996054507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3996054507},{"type":"v1","addr":"172.21.15.181:6841","nonce":3996054507}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:27:46.850 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:27:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 Processor -- start 2020-11-16T22:27:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 -- start start 2020-11-16T22:27:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffb3c160530 0x7ffb3c061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c063b10 con 0x7ffb3c142660 2020-11-16T22:27:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c062700 con 0x7ffb3c160530 2020-11-16T22:27:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb4326a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0629a0 con 0x7ffb3c142a40 2020-11-16T22:27:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37318/0 (socket says 172.21.15.181:37318) 2020-11-16T22:27:46.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 -- 172.21.15.181:0/3061947228 learned_addr learned my addr 172.21.15.181:0/3061947228 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:46.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb42a69700 1 --2- 172.21.15.181:0/3061947228 >> v2:172.21.15.175:3300/0 conn(0x7ffb3c160530 0x7ffb3c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:46.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 -- 172.21.15.181:0/3061947228 >> v2:172.21.15.175:3300/0 conn(0x7ffb3c160530 msgr2=0x7ffb3c061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:46.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 --2- 172.21.15.181:0/3061947228 >> v2:172.21.15.175:3300/0 conn(0x7ffb3c160530 0x7ffb3c061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:46.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 -- 172.21.15.181:0/3061947228 >> v1:172.21.15.175:6789/0 conn(0x7ffb3c142660 legacy=0x7ffb3c063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:46.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:46.997+0000 7ffb41a67700 1 -- 172.21.15.181:0/3061947228 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c062c40 con 0x7ffb3c142a40 2020-11-16T22:27:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb41a67700 1 --2- 172.21.15.181:0/3061947228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ffb24006fd0 tx=0x7ffb24006eb0).ready entity=mon.2 client_cookie=e9bc1beed2aee054 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb41266700 1 -- 172.21.15.181:0/3061947228 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24009810 con 0x7ffb3c142a40 2020-11-16T22:27:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb41266700 1 -- 172.21.15.181:0/3061947228 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24009970 con 0x7ffb3c142a40 2020-11-16T22:27:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb41266700 1 -- 172.21.15.181:0/3061947228 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24009cf0 con 0x7ffb3c142a40 2020-11-16T22:27:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- 172.21.15.181:0/3061947228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 msgr2=0x7ffb3c063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 --2- 172.21.15.181:0/3061947228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ffb24006fd0 tx=0x7ffb24006eb0).stop 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- 172.21.15.181:0/3061947228 shutdown_connections 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 --2- 172.21.15.181:0/3061947228 >> v2:172.21.15.175:3300/0 conn(0x7ffb3c160530 0x7ffb3c061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 --2- 172.21.15.181:0/3061947228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c142a40 0x7ffb3c063620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- 172.21.15.181:0/3061947228 >> 172.21.15.181:0/3061947228 conn(0x7ffb3c0d2dc0 msgr2=0x7ffb3c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- 172.21.15.181:0/3061947228 shutdown_connections 2020-11-16T22:27:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- 172.21.15.181:0/3061947228 wait complete. 2020-11-16T22:27:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 Processor -- start 2020-11-16T22:27:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.001+0000 7ffb4326a700 1 -- start start 2020-11-16T22:27:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3c142660 0x7ffb3c144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb3c142a40 0x7ffb3c140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:47.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:47.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c15d4e0 con 0x7ffb3c142660 2020-11-16T22:27:47.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c1456e0 con 0x7ffb3c142a40 2020-11-16T22:27:47.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c145980 con 0x7ffb3c160530 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb41a67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb3c142a40 0x7ffb3c140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37324/0 (socket says 172.21.15.181:37324) 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 -- 172.21.15.181:0/1052938947 learned_addr learned my addr 172.21.15.181:0/1052938947 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42268700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3c142660 0x7ffb3c144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 -- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb3c142a40 msgr2=0x7ffb3c140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb3c142a40 0x7ffb3c140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 -- 172.21.15.181:0/1052938947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3c142660 msgr2=0x7ffb3c144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3c142660 0x7ffb3c144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 -- 172.21.15.181:0/1052938947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c145c20 con 0x7ffb3c160530 2020-11-16T22:27:47.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb42a69700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ffb3400ef20 tx=0x7ffb340091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:47.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400cb90 con 0x7ffb3c160530 2020-11-16T22:27:47.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3400ccf0 con 0x7ffb3c160530 2020-11-16T22:27:47.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c145ec0 con 0x7ffb3c160530 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34012600 con 0x7ffb3c160530 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.005+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c145240 con 0x7ffb3c160530 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c15fb10 con 0x7ffb3c160530 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ffb34022070 con 0x7ffb3c160530 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb3a7fc700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 0x7ffb18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:47.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffb3406e5d0 con 0x7ffb3c160530 2020-11-16T22:27:47.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb42268700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 0x7ffb18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:47.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.009+0000 7ffb42268700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 0x7ffb18057520 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ffb2c003300 tx=0x7ffb2c000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:47.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.017+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ffb34044720 con 0x7ffb3c160530 2020-11-16T22:27:47.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.169+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb3c0b1bb0 con 0x7ffb3c160530 2020-11-16T22:27:47.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.169+0000 7ffb3a7fc700 1 -- 172.21.15.181:0/1052938947 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7ffb3c0b1bb0 con 0x7ffb3c160530 2020-11-16T22:27:47.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 msgr2=0x7ffb18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:47.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 0x7ffb18057520 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ffb2c003300 tx=0x7ffb2c000b80).stop 2020-11-16T22:27:47.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 msgr2=0x7ffb3c140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ffb3400ef20 tx=0x7ffb340091d0).stop 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 shutdown_connections 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb18055060 0x7ffb18057520 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb3c160530 0x7ffb3c140540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb3c142660 0x7ffb3c144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 --2- 172.21.15.181:0/1052938947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb3c142a40 0x7ffb3c140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:47.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 >> 172.21.15.181:0/1052938947 conn(0x7ffb3c0d2dc0 msgr2=0x7ffb3c0d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:47.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 shutdown_connections 2020-11-16T22:27:47.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:47.173+0000 7ffb4326a700 1 -- 172.21.15.181:0/1052938947 wait complete. 2020-11-16T22:27:47.808 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:47.808 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":23,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:26:53.002253+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:26:49.798965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:26:51.012444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:27:48.826 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:27:48.827 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:27:48.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 Processor -- start 2020-11-16T22:27:48.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 -- start start 2020-11-16T22:27:48.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6fbc161480 0x7f6fbc164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 0x7f6fbc062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6fbc161860 0x7f6fbc062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c8db0 con 0x7f6fbc161480 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c8330 con 0x7f6fbc161860 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c85d0 con 0x7f6fbc164e10 2020-11-16T22:27:48.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6fbc161860 0x7f6fbc062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:48.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc0850700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 0x7f6fbc062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:48.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6fbc161860 0x7f6fbc062740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57722/0 (socket says 172.21.15.181:57722) 2020-11-16T22:27:48.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 -- 172.21.15.181:0/1885115914 learned_addr learned my addr 172.21.15.181:0/1885115914 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:48.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1051700 1 --2- 172.21.15.181:0/1885115914 >> v2:172.21.15.175:3300/0 conn(0x7f6fbc161480 0x7f6fbc164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:48.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 -- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 msgr2=0x7f6fbc062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:49.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 0x7f6fbc062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 -- 172.21.15.181:0/1885115914 >> v2:172.21.15.175:3300/0 conn(0x7f6fbc161480 msgr2=0x7f6fbc164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:49.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1885115914 >> v2:172.21.15.175:3300/0 conn(0x7f6fbc161480 0x7f6fbc164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 -- 172.21.15.181:0/1885115914 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0c8870 con 0x7f6fbc161860 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc0850700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 0x7f6fbc062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc062740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6fbc0b8580 tx=0x7f6fb4009290).ready entity=mon.1 client_cookie=ea6210486eb19103 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6faffff700 1 -- 172.21.15.181:0/1885115914 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400bb10 con 0x7f6fbc161860 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6faffff700 1 -- 172.21.15.181:0/1885115914 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb400bc70 con 0x7f6fbc161860 2020-11-16T22:27:49.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6faffff700 1 -- 172.21.15.181:0/1885115914 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4011400 con 0x7f6fbc161860 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 -- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 msgr2=0x7f6fbc062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.981+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc062740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6fbc0b8580 tx=0x7f6fb4009290).stop 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- 172.21.15.181:0/1885115914 shutdown_connections 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc062740 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1885115914 >> v2:172.21.15.175:3300/0 conn(0x7f6fbc161480 0x7f6fbc164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1885115914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc164e10 0x7f6fbc062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- 172.21.15.181:0/1885115914 >> 172.21.15.181:0/1885115914 conn(0x7f6fbc0d2dc0 msgr2=0x7f6fbc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:49.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- 172.21.15.181:0/1885115914 shutdown_connections 2020-11-16T22:27:49.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- 172.21.15.181:0/1885115914 wait complete. 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 Processor -- start 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- start start 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fbc161480 0x7f6fbc0b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc174990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:49.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc177430 con 0x7f6fbc161480 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b5780 con 0x7f6fbc161860 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc2053700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b5a20 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc0850700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc174990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.985+0000 7f6fc1852700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc0850700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc174990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57726/0 (socket says 172.21.15.181:57726) 2020-11-16T22:27:49.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37338/0 (socket says 172.21.15.181:37338) 2020-11-16T22:27:49.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc0850700 1 -- 172.21.15.181:0/1187164152 learned_addr learned my addr 172.21.15.181:0/1187164152 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1051700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fbc161480 0x7f6fbc0b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 -- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 msgr2=0x7f6fbc174990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc174990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 -- 172.21.15.181:0/1187164152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fbc161480 msgr2=0x7f6fbc0b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fbc161480 0x7f6fbc0b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 -- 172.21.15.181:0/1187164152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0b5cc0 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc1852700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6fbc0b8580 tx=0x7f6fb4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb401b5e0 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb401b740 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb401bac0 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc0b5f60 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0b64a0 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc0b6630 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.989+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6fb4017030 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.993+0000 7f6fadffb700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 0x7f6f90057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:49.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.993+0000 7f6fc1051700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 0x7f6f90057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:49.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.993+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fb406ece0 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.993+0000 7f6fc1051700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 0x7f6f90057520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6fb8006fd0 tx=0x7f6fb8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:49.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:48.997+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6fb4044f40 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.201+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6fbc044f60 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.789 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:27:49.752+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:27:49.789 INFO:tasks.ceph.mds.i.smithi175.stderr: -3004> 2020-11-16T22:26:59.280+0000 7fa5ccdc1700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2020-11-16T22:27:49.790 INFO:tasks.ceph.mds.i.smithi175.stderr: -3002> 2020-11-16T22:26:59.280+0000 7fa5ccdc1700 -1 mds.pinger is_rank_lagging: rank=5 was never sent ping request. 2020-11-16T22:27:49.790 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:27:49.790 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:27:49.790 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:27:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.769+0000 7f6fadffb700 1 -- 172.21.15.181:0/1187164152 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6fb4021070 con 0x7f6fbc0c98f0 2020-11-16T22:27:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 msgr2=0x7f6f90057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 0x7f6f90057520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f6fb8006fd0 tx=0x7f6fb8006df0).stop 2020-11-16T22:27:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 msgr2=0x7f6fbc1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:49.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6fbc0b8580 tx=0x7f6fb4016000).stop 2020-11-16T22:27:49.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 shutdown_connections 2020-11-16T22:27:49.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6f90055060 0x7f6f90057520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c98f0 0x7f6fbc1606a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fbc161480 0x7f6fbc0b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 --2- 172.21.15.181:0/1187164152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc161860 0x7f6fbc174990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.777+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 >> 172.21.15.181:0/1187164152 conn(0x7f6fbc0d2dc0 msgr2=0x7f6fbc064fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.781+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 shutdown_connections 2020-11-16T22:27:49.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:49.781+0000 7f6fc2053700 1 -- 172.21.15.181:0/1187164152 wait complete. 2020-11-16T22:27:49.996 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:27:49.996 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:27:49.996 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:27:49.996 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:27:49.997 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:27:49.998 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:27:49.998 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:27:49.999 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:27:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.149+0000 7f3e4f1f2700 1 Processor -- start 2020-11-16T22:27:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.149+0000 7f3e4f1f2700 1 -- start start 2020-11-16T22:27:50.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3e48160430 0x7f3e48161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e48160810 0x7f3e480600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e48160ca0 con 0x7f3e480c85e0 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480c8a40 con 0x7f3e48160430 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480cf030 con 0x7f3e48160810 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e48160810 0x7f3e480600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e48160810 0x7f3e480600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57780/0 (socket says 172.21.15.181:57780) 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 -- 172.21.15.181:0/2030288571 learned_addr learned my addr 172.21.15.181:0/2030288571 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4cf8e700 1 --2- 172.21.15.181:0/2030288571 >> v2:172.21.15.175:3300/0 conn(0x7f3e48160430 0x7f3e48161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 -- 172.21.15.181:0/2030288571 >> v2:172.21.15.175:3300/0 conn(0x7f3e48160430 msgr2=0x7f3e48161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:50.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/2030288571 >> v2:172.21.15.175:3300/0 conn(0x7f3e48160430 0x7f3e48161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:50.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 -- 172.21.15.181:0/2030288571 >> v1:172.21.15.175:6789/0 conn(0x7f3e480c85e0 legacy=0x7f3e48162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:50.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 -- 172.21.15.181:0/2030288571 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48161a10 con 0x7f3e48160810 2020-11-16T22:27:50.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/2030288571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e480600f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3e4815db10 tx=0x7f3e40009290).ready entity=mon.1 client_cookie=7c4dfee1d7fb1b14 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:50.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e477fe700 1 -- 172.21.15.181:0/2030288571 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000bbc0 con 0x7f3e48160810 2020-11-16T22:27:50.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e477fe700 1 -- 172.21.15.181:0/2030288571 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000bd20 con 0x7f3e48160810 2020-11-16T22:27:50.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e477fe700 1 -- 172.21.15.181:0/2030288571 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40011610 con 0x7f3e48160810 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/2030288571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 msgr2=0x7f3e480600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/2030288571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e480600f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3e4815db10 tx=0x7f3e40009290).stop 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/2030288571 shutdown_connections 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/2030288571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e480600f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/2030288571 >> v2:172.21.15.175:3300/0 conn(0x7f3e48160430 0x7f3e48161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/2030288571 >> 172.21.15.181:0/2030288571 conn(0x7f3e480d3680 msgr2=0x7f3e480d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/2030288571 shutdown_connections 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.153+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/2030288571 wait complete. 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 Processor -- start 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- start start 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e48160430 0x7f3e481671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e4815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48170660 con 0x7f3e48160430 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48170900 con 0x7f3e48160810 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48170ba0 con 0x7f3e480c85e0 2020-11-16T22:27:50.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4cf8e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4cf8e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37392/0 (socket says 172.21.15.181:37392) 2020-11-16T22:27:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4cf8e700 1 -- 172.21.15.181:0/1181355105 learned_addr learned my addr 172.21.15.181:0/1181355105 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e4815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e47fff700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e48160430 0x7f3e481671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 -- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 msgr2=0x7f3e48142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 -- 172.21.15.181:0/1181355105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e48160430 msgr2=0x7f3e481671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e48160430 0x7f3e481671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 -- 172.21.15.181:0/1181355105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48170e40 con 0x7f3e48160810 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4cf8e700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4d78f700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e4815f1b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f3e4815db10 tx=0x7f3e40016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000bbc0 con 0x7f3e48160810 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e481710e0 con 0x7f3e48160810 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000be90 con 0x7f3e48160810 2020-11-16T22:27:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40010cf0 con 0x7f3e48160810 2020-11-16T22:27:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.157+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48142660 con 0x7f3e48160810 2020-11-16T22:27:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e48069eb0 con 0x7f3e48160810 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3e40017030 con 0x7f3e48160810 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e44ff9700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 0x7f3e20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f3e4006f3a0 con 0x7f3e48160810 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e4cf8e700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 0x7f3e20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.161+0000 7f3e4cf8e700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 0x7f3e20057520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3e38006a90 tx=0x7f3e38006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.169+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3e400452a0 con 0x7f3e48160810 2020-11-16T22:27:50.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.369+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3e48160e80 con 0x7f3e48160810 2020-11-16T22:27:50.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e44ff9700 1 -- 172.21.15.181:0/1181355105 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3e400210a0 con 0x7f3e48160810 2020-11-16T22:27:50.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 msgr2=0x7f3e20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:50.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 0x7f3e20057520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3e38006a90 tx=0x7f3e38006df0).stop 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 msgr2=0x7f3e4815f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e4815f1b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f3e4815db10 tx=0x7f3e40016000).stop 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 shutdown_connections 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3e20055060 0x7f3e20057520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e48160810 0x7f3e4815f1b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e480c85e0 0x7f3e48142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 --2- 172.21.15.181:0/1181355105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e48160430 0x7f3e481671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 >> 172.21.15.181:0/1181355105 conn(0x7f3e480d3680 msgr2=0x7f3e480cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 shutdown_connections 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:50.769+0000 7f3e4f1f2700 1 -- 172.21.15.181:0/1181355105 wait complete. 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:27:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.317+0000 7fceb51a5700 1 Processor -- start 2020-11-16T22:27:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.317+0000 7fceb51a5700 1 -- start start 2020-11-16T22:27:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fceb0081550 0x7fceb00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb00d1800 con 0x7fceb0081170 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb00d1aa0 con 0x7fceb0142ad0 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb00d1d40 con 0x7fceb0081550 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb49a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fceb0081550 0x7fceb00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fceb0142ad0 0x7fceb0161630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43862/0 (socket says 172.21.15.181:43862) 2020-11-16T22:27:51.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaf7fe700 1 -- 172.21.15.181:0/1283119810 learned_addr learned my addr 172.21.15.181:0/1283119810 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaeffd700 1 -- 172.21.15.181:0/1283119810 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575378138 0 0) 0x7fceb00d1aa0 con 0x7fceb0142ad0 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaeffd700 1 -- 172.21.15.181:0/1283119810 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce90003240 con 0x7fceb0142ad0 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb49a4700 1 -- 172.21.15.181:0/1283119810 >> v1:172.21.15.181:6789/0 conn(0x7fceb0142ad0 legacy=0x7fceb0161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb49a4700 1 -- 172.21.15.181:0/1283119810 >> v1:172.21.15.175:6789/0 conn(0x7fceb0081170 legacy=0x7fceb01423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb49a4700 1 -- 172.21.15.181:0/1283119810 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00d1fe0 con 0x7fceb0081550 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb49a4700 1 --2- 172.21.15.181:0/1283119810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0081550 0x7fceb00d1060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcea8009cc0 tx=0x7fcea80091d0).ready entity=mon.1 client_cookie=a949a801a047581 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaeffd700 1 -- 172.21.15.181:0/1283119810 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea800ba90 con 0x7fceb0081550 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaeffd700 1 -- 172.21.15.181:0/1283119810 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea800bbf0 con 0x7fceb0081550 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceaeffd700 1 -- 172.21.15.181:0/1283119810 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8011400 con 0x7fceb0081550 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- 172.21.15.181:0/1283119810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0081550 msgr2=0x7fceb00d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 --2- 172.21.15.181:0/1283119810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0081550 0x7fceb00d1060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcea8009cc0 tx=0x7fcea80091d0).stop 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- 172.21.15.181:0/1283119810 shutdown_connections 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 --2- 172.21.15.181:0/1283119810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0081550 0x7fceb00d1060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- 172.21.15.181:0/1283119810 >> 172.21.15.181:0/1283119810 conn(0x7fceb00682f0 msgr2=0x7fceb00d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- 172.21.15.181:0/1283119810 shutdown_connections 2020-11-16T22:27:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.321+0000 7fceb51a5700 1 -- 172.21.15.181:0/1283119810 wait complete. 2020-11-16T22:27:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 Processor -- start 2020-11-16T22:27:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 -- start start 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0081170 0x7fceb0139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 0x7fceb00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00ce3f0 con 0x7fceb0081170 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb0170570 con 0x7fceb0142ad0 2020-11-16T22:27:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb51a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00ce5e0 con 0x7fceb0081550 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37408/0 (socket says 172.21.15.181:37408) 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb49a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 0x7fceb00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 -- 172.21.15.181:0/3638426779 learned_addr learned my addr 172.21.15.181:0/3638426779 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaffff700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0081170 0x7fceb0139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 -- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 msgr2=0x7fceb00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 0x7fceb00cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 -- 172.21.15.181:0/3638426779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0081170 msgr2=0x7fceb0139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0081170 0x7fceb0139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 -- 172.21.15.181:0/3638426779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00ce880 con 0x7fceb0081550 2020-11-16T22:27:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceb49a4700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 0x7fceb00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:51.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.325+0000 7fceaf7fe700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fce9800c710 tx=0x7fce98007ad0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980072e0 con 0x7fceb0081550 2020-11-16T22:27:51.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98007440 con 0x7fceb0081550 2020-11-16T22:27:51.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98005800 con 0x7fceb0081550 2020-11-16T22:27:51.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb00ceb20 con 0x7fceb0081550 2020-11-16T22:27:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb00cf060 con 0x7fceb0081550 2020-11-16T22:27:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb0044f60 con 0x7fceb0081550 2020-11-16T22:27:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce98010070 con 0x7fceb0081550 2020-11-16T22:27:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 0x7fce8c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fce9806d840 con 0x7fceb0081550 2020-11-16T22:27:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.329+0000 7fceaffff700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 0x7fce8c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.333+0000 7fceaffff700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 0x7fce8c057550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcea0003300 tx=0x7fcea0007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.337+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fce98043690 con 0x7fceb0081550 2020-11-16T22:27:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.485+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fceb0082560 con 0x7fceb0081550 2020-11-16T22:27:51.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.837+0000 7fcea7fff700 1 -- 172.21.15.181:0/3638426779 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce98046cb0 con 0x7fceb0081550 2020-11-16T22:27:51.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 msgr2=0x7fce8c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 0x7fce8c057550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcea0003300 tx=0x7fcea0007b90).stop 2020-11-16T22:27:51.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 msgr2=0x7fceb00cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fce9800c710 tx=0x7fce98007ad0).stop 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 shutdown_connections 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce8c055090 0x7fce8c057550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb0142ad0 0x7fceb00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb0081170 0x7fceb0139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 --2- 172.21.15.181:0/3638426779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb0081550 0x7fceb00cda10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 >> 172.21.15.181:0/3638426779 conn(0x7fceb00682f0 msgr2=0x7fceb013baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:51.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 shutdown_connections 2020-11-16T22:27:51.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.841+0000 7fceb51a5700 1 -- 172.21.15.181:0/3638426779 wait complete. 2020-11-16T22:27:51.829 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:27:51.845 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:27:51.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.989+0000 7f58f4501700 1 Processor -- start 2020-11-16T22:27:51.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.989+0000 7f58f4501700 1 -- start start 2020-11-16T22:27:51.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58ec0615b0 0x7f58ec080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f58ec061a30 0x7f58ec0814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec0cb950 con 0x7f58ec061a30 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec0cbbf0 con 0x7f58ec0615b0 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0cbe90 con 0x7f58ec145a60 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58ec0615b0 0x7f58ec080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f229d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58ec0615b0 0x7f58ec080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57804/0 (socket says 172.21.15.181:57804) 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 -- 172.21.15.181:0/3736393640 learned_addr learned my addr 172.21.15.181:0/3736393640 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f2a9e700 1 --2- 172.21.15.181:0/3736393640 >> v2:172.21.15.175:3300/0 conn(0x7f58ec061a30 0x7f58ec0814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 -- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 msgr2=0x7f58ec145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 -- 172.21.15.181:0/3736393640 >> v2:172.21.15.175:3300/0 conn(0x7f58ec061a30 msgr2=0x7f58ec0814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3736393640 >> v2:172.21.15.175:3300/0 conn(0x7f58ec061a30 0x7f58ec0814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 -- 172.21.15.181:0/3736393640 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0cc130 con 0x7f58ec0615b0 2020-11-16T22:27:51.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec080fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f58dc006fd0 tx=0x7f58dc006eb0).ready entity=mon.1 client_cookie=b04716d6553f1f83 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f129b700 1 -- 172.21.15.181:0/3736393640 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0096e0 con 0x7f58ec0615b0 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f129b700 1 -- 172.21.15.181:0/3736393640 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc009840 con 0x7f58ec0615b0 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f129b700 1 -- 172.21.15.181:0/3736393640 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc009bc0 con 0x7f58ec0615b0 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 msgr2=0x7f58ec080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec080fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f58dc006fd0 tx=0x7f58dc006eb0).stop 2020-11-16T22:27:51.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- 172.21.15.181:0/3736393640 shutdown_connections 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- 172.21.15.181:0/3736393640 >> v2:172.21.15.175:3300/0 conn(0x7f58ec061a30 0x7f58ec0814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 --2- 172.21.15.181:0/3736393640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec080fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- 172.21.15.181:0/3736393640 >> 172.21.15.181:0/3736393640 conn(0x7f58ec0d68e0 msgr2=0x7f58ec0d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- 172.21.15.181:0/3736393640 shutdown_connections 2020-11-16T22:27:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.993+0000 7f58f4501700 1 -- 172.21.15.181:0/3736393640 wait complete. 2020-11-16T22:27:51.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.997+0000 7f58f4501700 1 Processor -- start 2020-11-16T22:27:51.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:51.997+0000 7f58f4501700 1 -- start start 2020-11-16T22:27:51.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58ec061a30 0x7f58ec1706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec170bc0 con 0x7f58ec061a30 2020-11-16T22:27:51.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0ce050 con 0x7f58ec0615b0 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0ce2f0 con 0x7f58ec145a60 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57810/0 (socket says 172.21.15.181:57810) 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 -- 172.21.15.181:0/3450489814 learned_addr learned my addr 172.21.15.181:0/3450489814 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f2a9e700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58ec061a30 0x7f58ec1706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 -- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 msgr2=0x7f58ec0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 -- 172.21.15.181:0/3450489814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58ec061a30 msgr2=0x7f58ec1706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58ec061a30 0x7f58ec1706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 -- 172.21.15.181:0/3450489814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0ce590 con 0x7f58ec0615b0 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f229d700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f58e800eba0 tx=0x7f58e8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800c8e0 con 0x7f58ec0615b0 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e800ca40 con 0x7f58ec0615b0 2020-11-16T22:27:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800cdc0 con 0x7f58ec0615b0 2020-11-16T22:27:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec0ce830 con 0x7f58ec0615b0 2020-11-16T22:27:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec0ced70 con 0x7f58ec0615b0 2020-11-16T22:27:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.001+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec0cef70 con 0x7f58ec0615b0 2020-11-16T22:27:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.005+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f58e8012600 con 0x7f58ec0615b0 2020-11-16T22:27:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.005+0000 7f58da7fc700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 0x7f58c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.005+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 0x7f58c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.005+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f58e806e260 con 0x7f58ec0615b0 2020-11-16T22:27:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.005+0000 7f58f1a9c700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 0x7f58c4057520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f58dc0093e0 tx=0x7f58dc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:51.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.009+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f58ec0cef70 con 0x7f58ec0615b0 2020-11-16T22:27:52.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.169+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58ec0cef70 con 0x7f58ec0615b0 2020-11-16T22:27:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.849+0000 7f58da7fc700 1 -- 172.21.15.181:0/3450489814 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58ec0cef70 con 0x7f58ec0615b0 2020-11-16T22:27:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 msgr2=0x7f58c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 0x7f58c4057520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f58dc0093e0 tx=0x7f58dc015000).stop 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 msgr2=0x7f58ec1634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f58e800eba0 tx=0x7f58e8006df0).stop 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 shutdown_connections 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f58c4055060 0x7f58c4057520 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58ec145a60 0x7f58ec0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:52.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58ec0615b0 0x7f58ec1634b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 --2- 172.21.15.181:0/3450489814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58ec061a30 0x7f58ec1706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 >> 172.21.15.181:0/3450489814 conn(0x7f58ec0d68e0 msgr2=0x7f58ec142630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 shutdown_connections 2020-11-16T22:27:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:52.853+0000 7f58f4501700 1 -- 172.21.15.181:0/3450489814 wait complete. 2020-11-16T22:27:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:27:52.853 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:27:52.853 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:27:52.853 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:27:52.854 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.854 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:27:52.854 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:27:52.854 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:27:52.855 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.855 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:27:52.855 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:27:52.855 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:27:52.855 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.856 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:27:52.856 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:27:52.856 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:27:52.856 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.856 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:27:52.857 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:27:52.857 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:27:52.857 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.857 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:27:52.858 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:27:52.858 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:27:52.858 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.858 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:27:52.858 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:27:52.859 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:27:52.859 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.859 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:27:52.859 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:27:52.860 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:27:52.860 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.860 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:27:52.860 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:27:52.860 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:27:52.861 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:27:52.865 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:27:52.865 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:27:52.866 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:27:52.867 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:27:52.868 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:27:52.868 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:27:52.868 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:27:52.869 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.870 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:27:52.870 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:27:52.871 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:27:52.871 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.872 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:27:52.872 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:27:52.874 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:27:52.874 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:27:52.875 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:27:52.875 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:27:52.876 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:27:52.877 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:27:52.877 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:27:52.877 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:27:52.878 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:27:52.878 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.917 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:27:52.917 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.918 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:27:52.918 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.924 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:27:52.924 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:27:52.924 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.925 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.925 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:27:52.926 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.926 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:27:52.927 DEBUG:teuthology.parallel:result is None 2020-11-16T22:27:52.927 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:27:52.930 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:27:52.944 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:27:52.944 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:27:52.983 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:27:52.988 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:27:52.990 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:27:53.002 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:27:53.005 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:27:53.005 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:27:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 Processor -- start 2020-11-16T22:27:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 -- start start 2020-11-16T22:27:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f73c0145dc0 0x7f73c00c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 0x7f73c00cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c00c8420 con 0x7f73c0145dc0 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c00c86c0 con 0x7f73c00615f0 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c52ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00c8960 con 0x7f73c01461a0 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37476/0 (socket says 172.21.15.181:37476) 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 -- 172.21.15.181:0/1050175364 learned_addr learned my addr 172.21.15.181:0/1050175364 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.085+0000 7f73c4ae9700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 0x7f73c00cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bffff700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.175:3300/0 conn(0x7f73c0145dc0 0x7f73c00c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 -- 172.21.15.181:0/1050175364 >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 msgr2=0x7f73c00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 0x7f73c00cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 -- 172.21.15.181:0/1050175364 >> v2:172.21.15.175:3300/0 conn(0x7f73c0145dc0 msgr2=0x7f73c00c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.175:3300/0 conn(0x7f73c0145dc0 0x7f73c00c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 -- 172.21.15.181:0/1050175364 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c00c8c00 con 0x7f73c01461a0 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c4ae9700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 0x7f73c00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1050175364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f73c005d8c0 tx=0x7f73a8009290).ready entity=mon.2 client_cookie=bab2d029f83876d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73beffd700 1 -- 172.21.15.181:0/1050175364 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800bb10 con 0x7f73c01461a0 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73beffd700 1 -- 172.21.15.181:0/1050175364 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a800bc70 con 0x7f73c01461a0 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73beffd700 1 -- 172.21.15.181:0/1050175364 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8011400 con 0x7f73c01461a0 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 -- 172.21.15.181:0/1050175364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 msgr2=0x7f73c0061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1050175364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f73c005d8c0 tx=0x7f73a8009290).stop 2020-11-16T22:27:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 -- 172.21.15.181:0/1050175364 shutdown_connections 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.181:3300/0 conn(0x7f73c00615f0 0x7f73c00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1050175364 >> v2:172.21.15.175:3300/0 conn(0x7f73c0145dc0 0x7f73c00c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1050175364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c0061100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 -- 172.21.15.181:0/1050175364 >> 172.21.15.181:0/1050175364 conn(0x7f73c00cf630 msgr2=0x7f73c00cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 -- 172.21.15.181:0/1050175364 shutdown_connections 2020-11-16T22:27:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.089+0000 7f73c52ea700 1 -- 172.21.15.181:0/1050175364 wait complete. 2020-11-16T22:27:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 Processor -- start 2020-11-16T22:27:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 -- start start 2020-11-16T22:27:53.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g is failed for ~0s 2020-11-16T22:27:53.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i is failed for ~0s 2020-11-16T22:27:53.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0148130 con 0x7f73c00615f0 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0149500 con 0x7f73c0145dc0 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c52ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0148320 con 0x7f73c01461a0 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c4ae9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73bffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73c4ae9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c00cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37484/0 (socket says 172.21.15.181:37484) 2020-11-16T22:27:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73bffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:43642/0 (socket says 172.21.15.181:43642) 2020-11-16T22:27:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73bffff700 1 -- 172.21.15.181:0/1643459904 learned_addr learned my addr 172.21.15.181:0/1643459904 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.093+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57874/0 (socket says 172.21.15.181:57874) 2020-11-16T22:27:53.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 -- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 msgr2=0x7f73c00cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:53.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c00cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 -- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 msgr2=0x7f73c0149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 -- 172.21.15.181:0/1643459904 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c01485c0 con 0x7f73c00615f0 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73bffff700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f73b800ebd0 tx=0x7f73b8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:53.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800c8e0 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b800ca40 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c0148860 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800cdc0 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c00cb6b0 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c0044f60 con 0x7f73c00615f0 2020-11-16T22:27:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f73b8012600 con 0x7f73c00615f0 2020-11-16T22:27:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.097+0000 7f73b7fff700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 0x7f739c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.101+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f73b806e0f0 con 0x7f73c00615f0 2020-11-16T22:27:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.101+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 0x7f739c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.101+0000 7f73bf7fe700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 0x7f739c057520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f73a80090f0 tx=0x7f73a8016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:53.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.109+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f73b8044380 con 0x7f73c00615f0 2020-11-16T22:27:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:53.277+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f73c00cb8b0 con 0x7f73c00615f0 2020-11-16T22:27:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.057+0000 7f73b7fff700 1 -- 172.21.15.181:0/1643459904 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f73c00cb8b0 con 0x7f73c00615f0 2020-11-16T22:27:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 msgr2=0x7f739c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 0x7f739c057520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f73a80090f0 tx=0x7f73a8016000).stop 2020-11-16T22:27:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 msgr2=0x7f73c01727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f73b800ebd0 tx=0x7f73b8006df0).stop 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 shutdown_connections 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f739c055060 0x7f739c057520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73c01461a0 0x7f73c00cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c00615f0 0x7f73c01727c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 --2- 172.21.15.181:0/1643459904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73c0145dc0 0x7f73c0149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 >> 172.21.15.181:0/1643459904 conn(0x7f73c00cf630 msgr2=0x7f73c00caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 shutdown_connections 2020-11-16T22:27:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.061+0000 7f73c52ea700 1 -- 172.21.15.181:0/1643459904 wait complete. 2020-11-16T22:27:54.048 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:27:54.061 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:27:54.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.249+0000 7f2b3cec2700 1 Processor -- start 2020-11-16T22:27:54.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.249+0000 7f2b3cec2700 1 -- start start 2020-11-16T22:27:54.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.249+0000 7f2b3cec2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b38142a40 0x7f2b38063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.249+0000 7f2b3cec2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b38063b10 con 0x7f2b38142660 2020-11-16T22:27:54.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b38062700 con 0x7f2b38142a40 2020-11-16T22:27:54.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380629a0 con 0x7f2b38160530 2020-11-16T22:27:54.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37500/0 (socket says 172.21.15.181:37500) 2020-11-16T22:27:54.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b36ffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b38142a40 0x7f2b38063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 -- 172.21.15.181:0/2925964991 learned_addr learned my addr 172.21.15.181:0/2925964991 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:54.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 -- 172.21.15.181:0/2925964991 >> v2:172.21.15.175:3300/0 conn(0x7f2b38142a40 msgr2=0x7f2b38063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:54.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 --2- 172.21.15.181:0/2925964991 >> v2:172.21.15.175:3300/0 conn(0x7f2b38142a40 0x7f2b38063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 -- 172.21.15.181:0/2925964991 >> v1:172.21.15.175:6789/0 conn(0x7f2b38142660 legacy=0x7f2b38063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 -- 172.21.15.181:0/2925964991 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38062c40 con 0x7f2b38160530 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b37fff700 1 --2- 172.21.15.181:0/2925964991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f2b30009d50 tx=0x7f2b30009290).ready entity=mon.2 client_cookie=6e5e379d9db3657b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b367fc700 1 -- 172.21.15.181:0/2925964991 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000bbc0 con 0x7f2b38160530 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b367fc700 1 -- 172.21.15.181:0/2925964991 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3000bd20 con 0x7f2b38160530 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b367fc700 1 -- 172.21.15.181:0/2925964991 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30011610 con 0x7f2b38160530 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- 172.21.15.181:0/2925964991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 msgr2=0x7f2b38061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/2925964991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f2b30009d50 tx=0x7f2b30009290).stop 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- 172.21.15.181:0/2925964991 shutdown_connections 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/2925964991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38160530 0x7f2b38061810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/2925964991 >> v2:172.21.15.175:3300/0 conn(0x7f2b38142a40 0x7f2b38063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- 172.21.15.181:0/2925964991 >> 172.21.15.181:0/2925964991 conn(0x7f2b380d2dc0 msgr2=0x7f2b380ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- 172.21.15.181:0/2925964991 shutdown_connections 2020-11-16T22:27:54.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.253+0000 7f2b3cec2700 1 -- 172.21.15.181:0/2925964991 wait complete. 2020-11-16T22:27:54.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 Processor -- start 2020-11-16T22:27:54.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- start start 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b38142a40 0x7f2b380cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 0x7f2b38141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380c9eb0 con 0x7f2b38142a40 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380ca150 con 0x7f2b38160530 2020-11-16T22:27:54.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380ca3f0 con 0x7f2b38142660 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37502/0 (socket says 172.21.15.181:37502) 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 -- 172.21.15.181:0/3633800471 learned_addr learned my addr 172.21.15.181:0/3633800471 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b37fff700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 0x7f2b38141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b36ffd700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b38142a40 0x7f2b380cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 -- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 msgr2=0x7f2b38141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 0x7f2b38141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 -- 172.21.15.181:0/3633800471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b38142a40 msgr2=0x7f2b380cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b38142a40 0x7f2b380cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 -- 172.21.15.181:0/3633800471 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380ccba0 con 0x7f2b38142660 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b37fff700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 0x7f2b38141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:54.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b377fe700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2b2800c710 tx=0x7f2b28007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:54.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800e490 con 0x7f2b38142660 2020-11-16T22:27:54.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2800e5f0 con 0x7f2b38142660 2020-11-16T22:27:54.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b380cce40 con 0x7f2b38142660 2020-11-16T22:27:54.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800e990 con 0x7f2b38142660 2020-11-16T22:27:54.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.257+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b380cdc90 con 0x7f2b38142660 2020-11-16T22:27:54.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b380c8e00 con 0x7f2b38142660 2020-11-16T22:27:54.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2b28019030 con 0x7f2b38142660 2020-11-16T22:27:54.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b177fe700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 0x7f2b10057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:54.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2b2806ccf0 con 0x7f2b38142660 2020-11-16T22:27:54.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b36ffd700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 0x7f2b10057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:54.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.261+0000 7f2b36ffd700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 0x7f2b10057520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f2b3815cad0 tx=0x7f2b2c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:54.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.269+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2b380c8e00 con 0x7f2b38142660 2020-11-16T22:27:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.469+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b38063b10 con 0x7f2b38142660 2020-11-16T22:27:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b177fe700 1 -- 172.21.15.181:0/3633800471 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+7325 (secure 0 0 0) 0x7f2b38063b10 con 0x7f2b38142660 2020-11-16T22:27:54.457 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:27:54.457 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:27:54.457 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:27:54.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.457 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.459 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:27:54.460 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:27:54.463 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.464 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.466 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:27:54.467 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.469 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.470 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:27:54.471 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.472 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.474 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.475 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:27:54.476 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.477 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.479 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:27:54.482 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:27:54.482 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.482 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:27:54.483 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:27:54.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.486 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.487 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mds", 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJfM5SMMhAAS4qc6raz3c659MbdQOTNtQ==", 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mds" 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:54.488 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mgr", 2020-11-16T22:27:54.489 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJffKyMMhAAKj4uQLZGQdiMHoIUt/Wpbw==", 2020-11-16T22:27:54.489 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:54.489 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mgr" 2020-11-16T22:27:54.489 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-osd", 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJfIMKMMhAAl9NvMKAOl589XZM8ju5ZWQ==", 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-osd" 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd", 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJfgNeMMhAA112ncqYfJWi84URA0SXUfQ==", 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd" 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJfSu6MMhAABFrvE4reVdWFRv5wUXwbmg==", 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rgw", 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCa/LJfjgSNMhAAPt3pwKIsbQsW7SSfPSwVtA==", 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rgw" 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:27:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:27:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:27:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:27:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:27:58.645 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:27:58.647 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 msgr2=0x7f2b10057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 0x7f2b10057520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f2b3815cad0 tx=0x7f2b2c006df0).stop 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 msgr2=0x7f2b3815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2b2800c710 tx=0x7f2b28007c10).stop 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 shutdown_connections 2020-11-16T22:27:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2b10055060 0x7f2b10057520 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38160530 0x7f2b38141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38142660 0x7f2b3815e240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 --2- 172.21.15.181:0/3633800471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b38142a40 0x7f2b380cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 >> 172.21.15.181:0/3633800471 conn(0x7f2b380d2dc0 msgr2=0x7f2b3815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 shutdown_connections 2020-11-16T22:27:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:54.473+0000 7f2b3cec2700 1 -- 172.21.15.181:0/3633800471 wait complete. 2020-11-16T22:27:58.650 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2020-11-16T22:27:58.656 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:58.658 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:27:58.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.801+0000 7fed27fff700 1 Processor -- start 2020-11-16T22:27:58.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.801+0000 7fed27fff700 1 -- start start 2020-11-16T22:27:58.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed2815b810 0x7fed2815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 0x7fed2815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280cfbb0 con 0x7fed2815cd00 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed280659d0 con 0x7fed2815b810 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2815d770 con 0x7fed2815fbe0 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed277fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fed2815cd00 0x7fed280cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43980/0 (socket says 172.21.15.181:43980) 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed277fe700 1 -- 172.21.15.181:0/2334087396 learned_addr learned my addr 172.21.15.181:0/2334087396 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 --2- 172.21.15.181:0/2334087396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 0x7fed2815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed26ffd700 1 --2- 172.21.15.181:0/2334087396 >> v2:172.21.15.175:3300/0 conn(0x7fed2815b810 0x7fed2815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed25ffb700 1 -- 172.21.15.181:0/2334087396 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458049816 0 0) 0x7fed280cfbb0 con 0x7fed2815cd00 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed25ffb700 1 -- 172.21.15.181:0/2334087396 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed08003240 con 0x7fed2815cd00 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 -- 172.21.15.181:0/2334087396 >> v2:172.21.15.175:3300/0 conn(0x7fed2815b810 msgr2=0x7fed2815f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 --2- 172.21.15.181:0/2334087396 >> v2:172.21.15.175:3300/0 conn(0x7fed2815b810 0x7fed2815f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 -- 172.21.15.181:0/2334087396 >> v1:172.21.15.181:6789/0 conn(0x7fed2815cd00 legacy=0x7fed280cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 -- 172.21.15.181:0/2334087396 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2815da10 con 0x7fed2815fbe0 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed267fc700 1 --2- 172.21.15.181:0/2334087396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 0x7fed2815e3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fed1000b9a0 tx=0x7fed10006df0).ready entity=mon.2 client_cookie=9250992e89b17491 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed25ffb700 1 -- 172.21.15.181:0/2334087396 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10009680 con 0x7fed2815fbe0 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed25ffb700 1 -- 172.21.15.181:0/2334087396 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed100097e0 con 0x7fed2815fbe0 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed25ffb700 1 -- 172.21.15.181:0/2334087396 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10009b60 con 0x7fed2815fbe0 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- 172.21.15.181:0/2334087396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 msgr2=0x7fed2815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 --2- 172.21.15.181:0/2334087396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 0x7fed2815e3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fed1000b9a0 tx=0x7fed10006df0).stop 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- 172.21.15.181:0/2334087396 shutdown_connections 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 --2- 172.21.15.181:0/2334087396 >> v2:172.21.15.175:3300/0 conn(0x7fed2815b810 0x7fed2815f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 --2- 172.21.15.181:0/2334087396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815fbe0 0x7fed2815e3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- 172.21.15.181:0/2334087396 >> 172.21.15.181:0/2334087396 conn(0x7fed280d2dc0 msgr2=0x7fed280ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- 172.21.15.181:0/2334087396 shutdown_connections 2020-11-16T22:27:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- 172.21.15.181:0/2334087396 wait complete. 2020-11-16T22:27:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 Processor -- start 2020-11-16T22:27:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.805+0000 7fed27fff700 1 -- start start 2020-11-16T22:27:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 0x7fed28164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 0x7fed28165250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280b5320 con 0x7fed2815fbe0 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280b55c0 con 0x7fed2815b810 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280b5860 con 0x7fed2815cd00 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed267fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 0x7fed28164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57912/0 (socket says 172.21.15.181:57912) 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 -- 172.21.15.181:0/3477055519 learned_addr learned my addr 172.21.15.181:0/3477055519 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed277fe700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 0x7fed28165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 -- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 msgr2=0x7fed28164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 0x7fed28164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 -- 172.21.15.181:0/3477055519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 msgr2=0x7fed28165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 0x7fed28165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 -- 172.21.15.181:0/3477055519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed280b5b00 con 0x7fed2815b810 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed277fe700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 0x7fed28165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed267fc700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 0x7fed28164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed26ffd700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fed1800eba0 tx=0x7fed18006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800c9b0 con 0x7fed2815b810 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1800cb10 con 0x7fed2815b810 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed280b5da0 con 0x7fed2815b810 2020-11-16T22:27:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800ce90 con 0x7fed2815b810 2020-11-16T22:27:58.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed281595e0 con 0x7fed2815b810 2020-11-16T22:27:58.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2815de90 con 0x7fed2815b810 2020-11-16T22:27:58.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.809+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fed18012600 con 0x7fed2815b810 2020-11-16T22:27:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.813+0000 7fed1effd700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 0x7fed04057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.813+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fed1806de20 con 0x7fed2815b810 2020-11-16T22:27:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.813+0000 7fed267fc700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 0x7fed04057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.813+0000 7fed267fc700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 0x7fed04057520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fed1000b150 tx=0x7fed10015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:58.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:58.817+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fed18044350 con 0x7fed2815b810 2020-11-16T22:27:59.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.029+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fed28044f60 con 0x7fed2815b810 2020-11-16T22:27:59.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed1effd700 1 -- 172.21.15.181:0/3477055519 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fed18047970 con 0x7fed2815b810 2020-11-16T22:27:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 msgr2=0x7fed04057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 0x7fed04057520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fed1000b150 tx=0x7fed10015000).stop 2020-11-16T22:27:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 msgr2=0x7fed28159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fed1800eba0 tx=0x7fed18006df0).stop 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 shutdown_connections 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fed04055060 0x7fed04057520 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed2815fbe0 0x7fed28165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed2815b810 0x7fed28159070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 --2- 172.21.15.181:0/3477055519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed2815cd00 0x7fed28164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 >> 172.21.15.181:0/3477055519 conn(0x7fed280d2dc0 msgr2=0x7fed280b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 shutdown_connections 2020-11-16T22:27:59.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.041+0000 7fed27fff700 1 -- 172.21.15.181:0/3477055519 wait complete. 2020-11-16T22:27:59.028 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:27:59.040 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2020-11-16T22:27:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.177+0000 7f9cd759e700 1 Processor -- start 2020-11-16T22:27:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.177+0000 7f9cd759e700 1 -- start start 2020-11-16T22:27:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cd815d140 0x7f9cd815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd80c85e0 con 0x7f9cd80cf4a0 2020-11-16T22:27:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd815ff70 con 0x7f9cd815d140 2020-11-16T22:27:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd80c8840 con 0x7f9cd80cf920 2020-11-16T22:27:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37534/0 (socket says 172.21.15.181:37534) 2020-11-16T22:27:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/3036789400 learned_addr learned my addr 172.21.15.181:0/3036789400 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd659c700 1 --2- 172.21.15.181:0/3036789400 >> v2:172.21.15.175:3300/0 conn(0x7f9cd815d140 0x7f9cd815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/3036789400 >> v2:172.21.15.175:3300/0 conn(0x7f9cd815d140 msgr2=0x7f9cd815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:59.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 --2- 172.21.15.181:0/3036789400 >> v2:172.21.15.175:3300/0 conn(0x7f9cd815d140 0x7f9cd815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/3036789400 >> v1:172.21.15.175:6789/0 conn(0x7f9cd80cf4a0 legacy=0x7f9cd8064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/3036789400 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80c8ae0 con 0x7f9cd80cf920 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd6d9d700 1 --2- 172.21.15.181:0/3036789400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9cd0009d50 tx=0x7f9cd0009290).ready entity=mon.2 client_cookie=2341bd50d4e1923b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd559a700 1 -- 172.21.15.181:0/3036789400 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000bbc0 con 0x7f9cd80cf920 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd559a700 1 -- 172.21.15.181:0/3036789400 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd000bd20 con 0x7f9cd80cf920 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd559a700 1 -- 172.21.15.181:0/3036789400 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0011610 con 0x7f9cd80cf920 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- 172.21.15.181:0/3036789400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 msgr2=0x7f9cd80656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 --2- 172.21.15.181:0/3036789400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9cd0009d50 tx=0x7f9cd0009290).stop 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- 172.21.15.181:0/3036789400 shutdown_connections 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 --2- 172.21.15.181:0/3036789400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd80cf920 0x7f9cd80656e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 --2- 172.21.15.181:0/3036789400 >> v2:172.21.15.175:3300/0 conn(0x7f9cd815d140 0x7f9cd815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- 172.21.15.181:0/3036789400 >> 172.21.15.181:0/3036789400 conn(0x7f9cd80d2dc0 msgr2=0x7f9cd80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- 172.21.15.181:0/3036789400 shutdown_connections 2020-11-16T22:27:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.181+0000 7f9cd759e700 1 -- 172.21.15.181:0/3036789400 wait complete. 2020-11-16T22:27:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 Processor -- start 2020-11-16T22:27:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 -- start start 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cd80cf4a0 0x7f9cd81600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 0x7f9cd813f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8171060 con 0x7f9cd80cf4a0 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8170dc0 con 0x7f9cd80cf920 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd816cc90 con 0x7f9cd815d140 2020-11-16T22:27:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37540/0 (socket says 172.21.15.181:37540) 2020-11-16T22:27:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/1629179185 learned_addr learned my addr 172.21.15.181:0/1629179185 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd5d9b700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 0x7f9cd813f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd659c700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cd80cf4a0 0x7f9cd81600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 msgr2=0x7f9cd813f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 0x7f9cd813f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/1629179185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cd80cf4a0 msgr2=0x7f9cd81600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cd80cf4a0 0x7f9cd81600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd6d9d700 1 -- 172.21.15.181:0/1629179185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd816cf30 con 0x7f9cd815d140 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.185+0000 7f9cd5d9b700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 0x7f9cd813f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd6d9d700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9cd00092d0 tx=0x7f9cd0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000bbc0 con 0x7f9cd815d140 2020-11-16T22:27:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd000be90 con 0x7f9cd815d140 2020-11-16T22:27:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd816d1d0 con 0x7f9cd815d140 2020-11-16T22:27:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0010cb0 con 0x7f9cd815d140 2020-11-16T22:27:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd816d910 con 0x7f9cd815d140 2020-11-16T22:27:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd8044f60 con 0x7f9cd815d140 2020-11-16T22:27:59.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9cd0023030 con 0x7f9cd815d140 2020-11-16T22:27:59.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 0x7f9cac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9cd006efa0 con 0x7f9cd815d140 2020-11-16T22:27:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd659c700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 0x7f9cac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.189+0000 7f9cd659c700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 0x7f9cac057520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9cc8006fd0 tx=0x7f9cc8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.197+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9cd0045660 con 0x7f9cd815d140 2020-11-16T22:27:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.397+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9cd805d550 con 0x7f9cd815d140 2020-11-16T22:27:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cc67fc700 1 -- 172.21.15.181:0/1629179185 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9cd0021070 con 0x7f9cd815d140 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 msgr2=0x7f9cac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 0x7f9cac057520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9cc8006fd0 tx=0x7f9cc8006df0).stop 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 msgr2=0x7f9cd80d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9cd00092d0 tx=0x7f9cd0016000).stop 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 shutdown_connections 2020-11-16T22:27:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cac055060 0x7f9cac057520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cd815d140 0x7f9cd80d10f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cd80cf4a0 0x7f9cd81600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 --2- 172.21.15.181:0/1629179185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd80cf920 0x7f9cd813f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 >> 172.21.15.181:0/1629179185 conn(0x7f9cd80d2dc0 msgr2=0x7f9cd815f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 shutdown_connections 2020-11-16T22:27:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.409+0000 7f9cd759e700 1 -- 172.21.15.181:0/1629179185 wait complete. 2020-11-16T22:27:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:27:59.410 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2020-11-16T22:27:59.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 Processor -- start 2020-11-16T22:27:59.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- start start 2020-11-16T22:27:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a4081f20 con 0x7f08a4145770 2020-11-16T22:27:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a4060470 con 0x7f08a40d2810 2020-11-16T22:27:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4145fb0 con 0x7f08a4145b50 2020-11-16T22:27:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a91cb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f08a40d2810 0x7f08a41612b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44010/0 (socket says 172.21.15.181:44010) 2020-11-16T22:27:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37546/0 (socket says 172.21.15.181:37546) 2020-11-16T22:27:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a91cb700 1 -- 172.21.15.181:0/3480278905 learned_addr learned my addr 172.21.15.181:0/3480278905 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:59.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 -- 172.21.15.181:0/3480278905 >> v1:172.21.15.181:6789/0 conn(0x7f08a40d2810 legacy=0x7f08a41612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 -- 172.21.15.181:0/3480278905 >> v1:172.21.15.175:6789/0 conn(0x7f08a4145770 legacy=0x7f08a4160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 -- 172.21.15.181:0/3480278905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4060710 con 0x7f08a4145b50 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a3fff700 1 --2- 172.21.15.181:0/3480278905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f089c00b9d0 tx=0x7f089c006df0).ready entity=mon.2 client_cookie=164b5cae82366e76 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a37fe700 1 -- 172.21.15.181:0/3480278905 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c009870 con 0x7f08a4145b50 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a37fe700 1 -- 172.21.15.181:0/3480278905 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c0099d0 con 0x7f08a4145b50 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a37fe700 1 -- 172.21.15.181:0/3480278905 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c009d50 con 0x7f08a4145b50 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- 172.21.15.181:0/3480278905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 msgr2=0x7f08a40d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 --2- 172.21.15.181:0/3480278905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f089c00b9d0 tx=0x7f089c006df0).stop 2020-11-16T22:27:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- 172.21.15.181:0/3480278905 shutdown_connections 2020-11-16T22:27:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 --2- 172.21.15.181:0/3480278905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a4145b50 0x7f08a40d2320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- 172.21.15.181:0/3480278905 >> 172.21.15.181:0/3480278905 conn(0x7f08a40d6760 msgr2=0x7f08a40d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.549+0000 7f08a99cc700 1 -- 172.21.15.181:0/3480278905 shutdown_connections 2020-11-16T22:27:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 -- 172.21.15.181:0/3480278905 wait complete. 2020-11-16T22:27:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 Processor -- start 2020-11-16T22:27:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 -- start start 2020-11-16T22:27:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 0x7f08a4139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08a4145770 0x7f08a40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41700d0 con 0x7f08a4145770 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a416f650 con 0x7f08a4145b50 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a99cc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a416f8f0 con 0x7f08a40d2810 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a91cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a89ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 0x7f08a4139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a91cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57944/0 (socket says 172.21.15.181:57944) 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a91cb700 1 -- 172.21.15.181:0/63230286 learned_addr learned my addr 172.21.15.181:0/63230286 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:27:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.553+0000 7f08a3fff700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08a4145770 0x7f08a40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 -- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 msgr2=0x7f08a4139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 0x7f08a4139ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 -- 172.21.15.181:0/63230286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08a4145770 msgr2=0x7f08a40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08a4145770 0x7f08a40cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 -- 172.21.15.181:0/63230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a416fb90 con 0x7f08a4145b50 2020-11-16T22:27:59.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a89ca700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 0x7f08a4139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:27:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a91cb700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f088c00c710 tx=0x7f088c007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c0072e0 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a416fe30 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c007440 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c005800 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a40cc6d0 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a415ea50 con 0x7f08a4145b50 2020-11-16T22:27:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f088c010070 con 0x7f08a4145b50 2020-11-16T22:27:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 0x7f0874057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f088c06d070 con 0x7f08a4145b50 2020-11-16T22:27:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a89ca700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 0x7f0874057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.557+0000 7f08a89ca700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 0x7f0874057520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0894001db0 tx=0x7f0894009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.569+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f088c043600 con 0x7f08a4145b50 2020-11-16T22:27:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.773+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f08a4044f60 con 0x7f08a4145b50 2020-11-16T22:27:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a0ff9700 1 -- 172.21.15.181:0/63230286 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f088c046c20 con 0x7f08a4145b50 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 msgr2=0x7f0874057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 0x7f0874057520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0894001db0 tx=0x7f0894009040).stop 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 msgr2=0x7f08a40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f088c00c710 tx=0x7f088c007ba0).stop 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 shutdown_connections 2020-11-16T22:27:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0874055060 0x7f0874057520 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4145b50 0x7f08a40cc160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40d2810 0x7f08a4139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 --2- 172.21.15.181:0/63230286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08a4145770 0x7f08a40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 >> 172.21.15.181:0/63230286 conn(0x7f08a40d6760 msgr2=0x7f08a4163d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 shutdown_connections 2020-11-16T22:27:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.785+0000 7f08a99cc700 1 -- 172.21.15.181:0/63230286 wait complete. 2020-11-16T22:27:59.774 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:27:59.789 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2020-11-16T22:27:59.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 Processor -- start 2020-11-16T22:27:59.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 -- start start 2020-11-16T22:27:59.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8e30142ab0 0x7f8e30061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e300615f0 0x7f8e30065a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e30142f60 con 0x7f8e301426d0 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e30064af0 con 0x7f8e30142ab0 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e3790a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e30064d90 con 0x7f8e300615f0 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e35ea7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e300615f0 0x7f8e30065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e356a6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8e301426d0 0x7f8e30160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44020/0 (socket says 172.21.15.181:44020) 2020-11-16T22:27:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e35ea7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e300615f0 0x7f8e30065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57952/0 (socket says 172.21.15.181:57952) 2020-11-16T22:27:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e356a6700 1 -- 172.21.15.181:0/2277752506 learned_addr learned my addr 172.21.15.181:0/2277752506 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:27:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e34ea5700 1 --2- 172.21.15.181:0/2277752506 >> v2:172.21.15.175:3300/0 conn(0x7f8e30142ab0 0x7f8e30061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:27:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e35ea7700 1 -- 172.21.15.181:0/2277752506 >> v2:172.21.15.175:3300/0 conn(0x7f8e30142ab0 msgr2=0x7f8e30061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:27:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e35ea7700 1 --2- 172.21.15.181:0/2277752506 >> v2:172.21.15.175:3300/0 conn(0x7f8e30142ab0 0x7f8e30061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e35ea7700 1 -- 172.21.15.181:0/2277752506 >> v1:172.21.15.181:6789/0 conn(0x7f8e301426d0 legacy=0x7f8e30160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.933+0000 7f8e1ffff700 1 -- 172.21.15.181:0/2277752506 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204843012 0 0) 0x7f8e30142f60 con 0x7f8e301426d0 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e35ea7700 1 -- 172.21.15.181:0/2277752506 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30065030 con 0x7f8e300615f0 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e35ea7700 1 --2- 172.21.15.181:0/2277752506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e300615f0 0x7f8e30065a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8e30069200 tx=0x7f8e280091d0).ready entity=mon.1 client_cookie=7116e5fa684f6628 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e1ffff700 1 -- 172.21.15.181:0/2277752506 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800ba70 con 0x7f8e300615f0 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e1ffff700 1 -- 172.21.15.181:0/2277752506 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800bbd0 con 0x7f8e300615f0 2020-11-16T22:27:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e1ffff700 1 -- 172.21.15.181:0/2277752506 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011400 con 0x7f8e300615f0 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- 172.21.15.181:0/2277752506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e300615f0 msgr2=0x7f8e30065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 --2- 172.21.15.181:0/2277752506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e300615f0 0x7f8e30065a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8e30069200 tx=0x7f8e280091d0).stop 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- 172.21.15.181:0/2277752506 shutdown_connections 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 --2- 172.21.15.181:0/2277752506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e300615f0 0x7f8e30065a10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 --2- 172.21.15.181:0/2277752506 >> v2:172.21.15.175:3300/0 conn(0x7f8e30142ab0 0x7f8e30061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- 172.21.15.181:0/2277752506 >> 172.21.15.181:0/2277752506 conn(0x7f8e300cf890 msgr2=0x7f8e300cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- 172.21.15.181:0/2277752506 shutdown_connections 2020-11-16T22:27:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- 172.21.15.181:0/2277752506 wait complete. 2020-11-16T22:27:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 Processor -- start 2020-11-16T22:27:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.937+0000 7f8e3790a700 1 -- start start 2020-11-16T22:27:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:27:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e301426d0 0x7f8e3015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 0x7f8e3015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300625b0 con 0x7f8e301426d0 2020-11-16T22:28:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30062850 con 0x7f8e30142ab0 2020-11-16T22:28:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30062af0 con 0x7f8e300615f0 2020-11-16T22:28:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37564/0 (socket says 172.21.15.181:37564) 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e35ea7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 0x7f8e3015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 -- 172.21.15.181:0/4005090961 learned_addr learned my addr 172.21.15.181:0/4005090961 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e34ea5700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e301426d0 0x7f8e3015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 -- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 msgr2=0x7f8e3015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 0x7f8e3015f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 -- 172.21.15.181:0/4005090961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e301426d0 msgr2=0x7f8e3015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e301426d0 0x7f8e3015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 -- 172.21.15.181:0/4005090961 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30062d90 con 0x7f8e300615f0 2020-11-16T22:28:00.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e35ea7700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 0x7f8e3015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:00.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e356a6700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f8e2c00c710 tx=0x7f8e2c007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:00.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0072e0 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c007440 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c005800 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30063030 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3015fd90 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.941+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e300634d0 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.945+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8e2c010070 con 0x7f8e300615f0 2020-11-16T22:28:00.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.945+0000 7f8e1d7fa700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 0x7f8e08057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.945+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8e2c06d0c0 con 0x7f8e300615f0 2020-11-16T22:28:00.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.945+0000 7f8e34ea5700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 0x7f8e08057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:00.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.945+0000 7f8e34ea5700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 0x7f8e08057520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8e3014ede0 tx=0x7f8e20006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:00.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:27:59.949+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8e300634d0 con 0x7f8e300615f0 2020-11-16T22:28:00.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.153+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8e30069eb0 con 0x7f8e300615f0 2020-11-16T22:28:00.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.165+0000 7f8e1d7fa700 1 -- 172.21.15.181:0/4005090961 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8e2c043650 con 0x7f8e300615f0 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 msgr2=0x7f8e08057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 0x7f8e08057520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8e3014ede0 tx=0x7f8e20006df0).stop 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 msgr2=0x7f8e301418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f8e2c00c710 tx=0x7f8e2c007a20).stop 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 shutdown_connections 2020-11-16T22:28:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8e08055060 0x7f8e08057520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e30142ab0 0x7f8e3015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e300615f0 0x7f8e301418f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 --2- 172.21.15.181:0/4005090961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e301426d0 0x7f8e3015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 >> 172.21.15.181:0/4005090961 conn(0x7f8e300cf890 msgr2=0x7f8e300a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 shutdown_connections 2020-11-16T22:28:00.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.169+0000 7f8e3790a700 1 -- 172.21.15.181:0/4005090961 wait complete. 2020-11-16T22:28:00.746 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:28:00.747 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2020-11-16T22:28:00.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 Processor -- start 2020-11-16T22:28:00.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- start start 2020-11-16T22:28:00.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac061b50 con 0x7f64ac0816e0 2020-11-16T22:28:00.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0618b0 con 0x7f64ac145a30 2020-11-16T22:28:00.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0cb930 con 0x7f64ac145e10 2020-11-16T22:28:00.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:00.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37574/0 (socket says 172.21.15.181:37574) 2020-11-16T22:28:00.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 -- 172.21.15.181:0/3164870999 learned_addr learned my addr 172.21.15.181:0/3164870999 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:00.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aa59c700 1 --1- 172.21.15.181:0/3164870999 >> v1:172.21.15.181:6789/0 conn(0x7f64ac145a30 0x7f64ac080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44034/0 (socket says 172.21.15.181:44034) 2020-11-16T22:28:00.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 -- 172.21.15.181:0/3164870999 >> v1:172.21.15.181:6789/0 conn(0x7f64ac145a30 legacy=0x7f64ac080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 -- 172.21.15.181:0/3164870999 >> v1:172.21.15.175:6789/0 conn(0x7f64ac0816e0 legacy=0x7f64ac0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 -- 172.21.15.181:0/3164870999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0cbbd0 con 0x7f64ac145e10 2020-11-16T22:28:00.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64aad9d700 1 --2- 172.21.15.181:0/3164870999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f64a400ba20 tx=0x7f64a4006eb0).ready entity=mon.2 client_cookie=32a81f34ac9199ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:00.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64a959a700 1 -- 172.21.15.181:0/3164870999 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40098d0 con 0x7f64ac145e10 2020-11-16T22:28:00.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64a959a700 1 -- 172.21.15.181:0/3164870999 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4009a30 con 0x7f64ac145e10 2020-11-16T22:28:00.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64a959a700 1 -- 172.21.15.181:0/3164870999 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4009db0 con 0x7f64ac145e10 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- 172.21.15.181:0/3164870999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 msgr2=0x7f64ac1492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 --2- 172.21.15.181:0/3164870999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f64a400ba20 tx=0x7f64a4006eb0).stop 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- 172.21.15.181:0/3164870999 shutdown_connections 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 --2- 172.21.15.181:0/3164870999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145e10 0x7f64ac1492c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- 172.21.15.181:0/3164870999 >> 172.21.15.181:0/3164870999 conn(0x7f64ac0d68c0 msgr2=0x7f64ac0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- 172.21.15.181:0/3164870999 shutdown_connections 2020-11-16T22:28:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.897+0000 7f64ab59e700 1 -- 172.21.15.181:0/3164870999 wait complete. 2020-11-16T22:28:00.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 Processor -- start 2020-11-16T22:28:00.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- start start 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 0x7f64ac1627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac145e10 0x7f64ac13e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac163090 con 0x7f64ac0816e0 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac163330 con 0x7f64ac145e10 2020-11-16T22:28:00.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1635d0 con 0x7f64ac145a30 2020-11-16T22:28:01.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37578/0 (socket says 172.21.15.181:37578) 2020-11-16T22:28:01.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 -- 172.21.15.181:0/2802080620 learned_addr learned my addr 172.21.15.181:0/2802080620 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64aad9d700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac145e10 0x7f64ac13e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64aa59c700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 0x7f64ac1627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 -- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac145e10 msgr2=0x7f64ac13e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac145e10 0x7f64ac13e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 -- 172.21.15.181:0/2802080620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 msgr2=0x7f64ac1627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 0x7f64ac1627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 -- 172.21.15.181:0/2802080620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac163870 con 0x7f64ac145a30 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64aa59c700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 0x7f64ac1627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a9d9b700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f649400c6e0 tx=0x7f6494007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494007020 con 0x7f64ac145a30 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6494007180 con 0x7f64ac145a30 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400e560 con 0x7f64ac145a30 2020-11-16T22:28:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac163b10 con 0x7f64ac145a30 2020-11-16T22:28:01.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.901+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac164250 con 0x7f64ac145a30 2020-11-16T22:28:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac044f60 con 0x7f64ac145a30 2020-11-16T22:28:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6494010030 con 0x7f64ac145a30 2020-11-16T22:28:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64a27fc700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 0x7f6488057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f649406d240 con 0x7f64ac145a30 2020-11-16T22:28:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64aa59c700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 0x7f6488057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.905+0000 7f64aa59c700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 0x7f6488057520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f649c0017c0 tx=0x7f649c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:00.913+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f64940437d0 con 0x7f64ac145a30 2020-11-16T22:28:01.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.121+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f64ac068e30 con 0x7f64ac145a30 2020-11-16T22:28:01.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64a27fc700 1 -- 172.21.15.181:0/2802080620 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6494046df0 con 0x7f64ac145a30 2020-11-16T22:28:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 msgr2=0x7f6488057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 0x7f6488057520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f649c0017c0 tx=0x7f649c009040).stop 2020-11-16T22:28:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 msgr2=0x7f64ac13dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f649400c6e0 tx=0x7f6494007c50).stop 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 shutdown_connections 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6488055060 0x7f6488057520 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac145e10 0x7f64ac13e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0816e0 0x7f64ac1627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 --2- 172.21.15.181:0/2802080620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ac145a30 0x7f64ac13dba0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 >> 172.21.15.181:0/2802080620 conn(0x7f64ac0d68c0 msgr2=0x7f64ac13eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 shutdown_connections 2020-11-16T22:28:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.133+0000 7f64ab59e700 1 -- 172.21.15.181:0/2802080620 wait complete. 2020-11-16T22:28:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:28:01.321 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2020-11-16T22:28:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.477+0000 7fa0410e5700 1 Processor -- start 2020-11-16T22:28:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.477+0000 7fa0410e5700 1 -- start start 2020-11-16T22:28:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa03c13e870 0x7fa03c13ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa03c061810 0x7fa03c160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c15d180 con 0x7fa03c15fdc0 2020-11-16T22:28:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c160730 con 0x7fa03c13e870 2020-11-16T22:28:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c13f140 con 0x7fa03c061810 2020-11-16T22:28:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa03c061810 0x7fa03c160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa03c15fdc0 0x7fa03c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44048/0 (socket says 172.21.15.181:44048) 2020-11-16T22:28:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03b7fe700 1 -- 172.21.15.181:0/1324577177 learned_addr learned my addr 172.21.15.181:0/1324577177 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03bfff700 1 --2- 172.21.15.181:0/1324577177 >> v2:172.21.15.175:3300/0 conn(0x7fa03c13e870 0x7fa03c13ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 -- 172.21.15.181:0/1324577177 >> v2:172.21.15.175:3300/0 conn(0x7fa03c13e870 msgr2=0x7fa03c13ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 --2- 172.21.15.181:0/1324577177 >> v2:172.21.15.175:3300/0 conn(0x7fa03c13e870 0x7fa03c13ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 -- 172.21.15.181:0/1324577177 >> v1:172.21.15.181:6789/0 conn(0x7fa03c15fdc0 legacy=0x7fa03c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 -- 172.21.15.181:0/1324577177 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c15d810 con 0x7fa03c061810 2020-11-16T22:28:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0408e4700 1 --2- 172.21.15.181:0/1324577177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fa03c160870 tx=0x7fa034006df0).ready entity=mon.1 client_cookie=189d0c54fab56a78 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03affd700 1 -- 172.21.15.181:0/1324577177 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340097b0 con 0x7fa03c061810 2020-11-16T22:28:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03affd700 1 -- 172.21.15.181:0/1324577177 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa034009910 con 0x7fa03c061810 2020-11-16T22:28:01.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa03affd700 1 -- 172.21.15.181:0/1324577177 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034009c90 con 0x7fa03c061810 2020-11-16T22:28:01.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- 172.21.15.181:0/1324577177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 msgr2=0x7fa03c160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 --2- 172.21.15.181:0/1324577177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fa03c160870 tx=0x7fa034006df0).stop 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- 172.21.15.181:0/1324577177 shutdown_connections 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 --2- 172.21.15.181:0/1324577177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160240 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 --2- 172.21.15.181:0/1324577177 >> v2:172.21.15.175:3300/0 conn(0x7fa03c13e870 0x7fa03c13ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- 172.21.15.181:0/1324577177 >> 172.21.15.181:0/1324577177 conn(0x7fa03c0cf630 msgr2=0x7fa03c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- 172.21.15.181:0/1324577177 shutdown_connections 2020-11-16T22:28:01.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.481+0000 7fa0410e5700 1 -- 172.21.15.181:0/1324577177 wait complete. 2020-11-16T22:28:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 Processor -- start 2020-11-16T22:28:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 -- start start 2020-11-16T22:28:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c13e870 0x7fa03c1364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 0x7fa03c136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c137040 con 0x7fa03c13e870 2020-11-16T22:28:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1372a0 con 0x7fa03c061810 2020-11-16T22:28:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0410e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c137540 con 0x7fa03c15fdc0 2020-11-16T22:28:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa03bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa0408e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 0x7fa03c136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa03bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57982/0 (socket says 172.21.15.181:57982) 2020-11-16T22:28:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa03bfff700 1 -- 172.21.15.181:0/140962165 learned_addr learned my addr 172.21.15.181:0/140962165 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.485+0000 7fa03b7fe700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c13e870 0x7fa03c1364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 -- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 msgr2=0x7fa03c160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 -- 172.21.15.181:0/140962165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c13e870 msgr2=0x7fa03c1364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c13e870 0x7fa03c1364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 -- 172.21.15.181:0/140962165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c1377e0 con 0x7fa03c15fdc0 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa03bfff700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0408e4700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 0x7fa03c136990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa03c0c9ed0 tx=0x7fa034015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034006b20 con 0x7fa03c15fdc0 2020-11-16T22:28:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03401e560 con 0x7fa03c15fdc0 2020-11-16T22:28:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c137a80 con 0x7fa03c15fdc0 2020-11-16T22:28:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03401e6d0 con 0x7fa03c15fdc0 2020-11-16T22:28:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c1614c0 con 0x7fa03c15fdc0 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c1616b0 con 0x7fa03c15fdc0 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa03401a040 con 0x7fa03c15fdc0 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 0x7fa018057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa034071520 con 0x7fa03c15fdc0 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.489+0000 7fa03bfff700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 0x7fa018057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.493+0000 7fa03bfff700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 0x7fa018057520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa03c163b10 tx=0x7fa02c0091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.497+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa034047b30 con 0x7fa03c15fdc0 2020-11-16T22:28:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.701+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa03c044f60 con 0x7fa03c15fdc0 2020-11-16T22:28:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.709+0000 7fa033fff700 1 -- 172.21.15.181:0/140962165 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa03404b150 con 0x7fa03c15fdc0 2020-11-16T22:28:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 msgr2=0x7fa018057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 0x7fa018057520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fa03c163b10 tx=0x7fa02c0091d0).stop 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 msgr2=0x7fa03c136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 0x7fa03c136990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fa03c0c9ed0 tx=0x7fa034015000).stop 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 shutdown_connections 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa018055060 0x7fa018057520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03c15fdc0 0x7fa03c136990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03c061810 0x7fa03c160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 --2- 172.21.15.181:0/140962165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c13e870 0x7fa03c1364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 >> 172.21.15.181:0/140962165 conn(0x7fa03c0cf630 msgr2=0x7fa03c0d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 shutdown_connections 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:01.713+0000 7fa0410e5700 1 -- 172.21.15.181:0/140962165 wait complete. 2020-11-16T22:28:02.087 INFO:teuthology.orchestra.run.smithi181.stderr:updated 2020-11-16T22:28:02.088 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:28:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 Processor -- start 2020-11-16T22:28:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- start start 2020-11-16T22:28:02.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f400600f0 0x7f3f400b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 0x7f3f400604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400b3310 con 0x7f3f400b44d0 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400b3120 con 0x7f3f40165880 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400cf030 con 0x7f3f400600f0 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f400600f0 0x7f3f400b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f400600f0 0x7f3f400b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57990/0 (socket says 172.21.15.181:57990) 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 -- 172.21.15.181:0/3703053500 learned_addr learned my addr 172.21.15.181:0/3703053500 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:02.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g has been restored 2020-11-16T22:28:02.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i has been restored 2020-11-16T22:28:02.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f451a6700 1 --2- 172.21.15.181:0/3703053500 >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 0x7f3f400604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 -- 172.21.15.181:0/3703053500 >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 msgr2=0x7f3f400604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:02.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 --2- 172.21.15.181:0/3703053500 >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 0x7f3f400604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 -- 172.21.15.181:0/3703053500 >> v1:172.21.15.181:6789/0 conn(0x7f3f400b44d0 legacy=0x7f3f40167980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 -- 172.21.15.181:0/3703053500 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400b35b0 con 0x7f3f400600f0 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f451a6700 1 --2- 172.21.15.181:0/3703053500 >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 0x7f3f400604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f459a7700 1 --2- 172.21.15.181:0/3703053500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400600f0 0x7f3f400b6b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f3f30009d80 tx=0x7f3f30009290).ready entity=mon.1 client_cookie=612ab5987146f1d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f449a5700 1 -- 172.21.15.181:0/3703053500 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000bc00 con 0x7f3f400600f0 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f449a5700 1 -- 172.21.15.181:0/3703053500 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3000bd60 con 0x7f3f400600f0 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f449a5700 1 -- 172.21.15.181:0/3703053500 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30011610 con 0x7f3f400600f0 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- 172.21.15.181:0/3703053500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400600f0 msgr2=0x7f3f400b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/3703053500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400600f0 0x7f3f400b6b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f3f30009d80 tx=0x7f3f30009290).stop 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- 172.21.15.181:0/3703053500 shutdown_connections 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/3703053500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400600f0 0x7f3f400b6b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/3703053500 >> v2:172.21.15.175:3300/0 conn(0x7f3f40165880 0x7f3f400604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- 172.21.15.181:0/3703053500 >> 172.21.15.181:0/3703053500 conn(0x7f3f400d3680 msgr2=0x7f3f400d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- 172.21.15.181:0/3703053500 shutdown_connections 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.233+0000 7f3f47c0b700 1 -- 172.21.15.181:0/3703053500 wait complete. 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.237+0000 7f3f47c0b700 1 Processor -- start 2020-11-16T22:28:02.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.237+0000 7f3f47c0b700 1 -- start start 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 0x7f3f400c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f40165880 0x7f3f400c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400c8fc0 con 0x7f3f40165880 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40174790 con 0x7f3f400b44d0 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400cfb40 con 0x7f3f400600f0 2020-11-16T22:28:02.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f459a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 0x7f3f400c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57998/0 (socket says 172.21.15.181:57998) 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f459a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 0x7f3f400c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37606/0 (socket says 172.21.15.181:37606) 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 -- 172.21.15.181:0/991567999 learned_addr learned my addr 172.21.15.181:0/991567999 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f461a8700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f40165880 0x7f3f400c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 -- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 msgr2=0x7f3f400c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 0x7f3f400c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 -- 172.21.15.181:0/991567999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f40165880 msgr2=0x7f3f400c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f40165880 0x7f3f400c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 -- 172.21.15.181:0/991567999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400cfde0 con 0x7f3f400b44d0 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f451a6700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f3f3400cb80 tx=0x7f3f34008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34007710 con 0x7f3f400b44d0 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f34007870 con 0x7f3f400b44d0 2020-11-16T22:28:02.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34007bf0 con 0x7f3f400b44d0 2020-11-16T22:28:02.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f400d0080 con 0x7f3f400b44d0 2020-11-16T22:28:02.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.241+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f400d0840 con 0x7f3f400b44d0 2020-11-16T22:28:02.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f40069eb0 con 0x7f3f400b44d0 2020-11-16T22:28:02.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3f3401b460 con 0x7f3f400b44d0 2020-11-16T22:28:02.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f3dffb700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 0x7f3f18057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3f3406dbd0 con 0x7f3f400b44d0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f459a7700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 0x7f3f18057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.245+0000 7f3f459a7700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 0x7f3f18057550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3f300090f0 tx=0x7f3f30016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.253+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3f34044100 con 0x7f3f400b44d0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f400d44b0 con 0x7f3f400b44d0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f3dffb700 1 -- 172.21.15.181:0/991567999 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f3f34047720 con 0x7f3f400b44d0 2020-11-16T22:28:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 msgr2=0x7f3f18057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 0x7f3f18057550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3f300090f0 tx=0x7f3f30016000).stop 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 msgr2=0x7f3f400c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.401+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f3f3400cb80 tx=0x7f3f34008040).stop 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 shutdown_connections 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3f18055090 0x7f3f18057550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f40165880 0x7f3f400c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f400600f0 0x7f3f400c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 --2- 172.21.15.181:0/991567999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f400b44d0 0x7f3f400c8ad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 >> 172.21.15.181:0/991567999 conn(0x7f3f400d3680 msgr2=0x7f3f40163050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 shutdown_connections 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.405+0000 7f3f47c0b700 1 -- 172.21.15.181:0/991567999 wait complete. 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:02.436 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":31,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:27:57.034415+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:28:02.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:28:02.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.793+0000 7f309359e700 1 Processor -- start 2020-11-16T22:28:02.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.793+0000 7f309359e700 1 -- start start 2020-11-16T22:28:02.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30941461a0 0x7f3094061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30940c8420 con 0x7f30940615f0 2020-11-16T22:28:02.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30940c86c0 con 0x7f30941461a0 2020-11-16T22:28:02.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30940c8960 con 0x7f3094145dc0 2020-11-16T22:28:02.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37614/0 (socket says 172.21.15.181:37614) 2020-11-16T22:28:02.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 -- 172.21.15.181:0/2007431008 learned_addr learned my addr 172.21.15.181:0/2007431008 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:02.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f3091d9b700 1 --2- 172.21.15.181:0/2007431008 >> v2:172.21.15.181:3300/0 conn(0x7f30941461a0 0x7f3094061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 -- 172.21.15.181:0/2007431008 >> v2:172.21.15.181:3300/0 conn(0x7f30941461a0 msgr2=0x7f3094061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:02.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 --2- 172.21.15.181:0/2007431008 >> v2:172.21.15.181:3300/0 conn(0x7f30941461a0 0x7f3094061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 -- 172.21.15.181:0/2007431008 >> v1:172.21.15.181:6789/0 conn(0x7f30940615f0 legacy=0x7f30940cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 -- 172.21.15.181:0/2007431008 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940c8c00 con 0x7f3094145dc0 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309259c700 1 --2- 172.21.15.181:0/2007431008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f308400ba20 tx=0x7f3084006eb0).ready entity=mon.2 client_cookie=fb845730ece31b4b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309159a700 1 -- 172.21.15.181:0/2007431008 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30840097d0 con 0x7f3094145dc0 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309159a700 1 -- 172.21.15.181:0/2007431008 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084009930 con 0x7f3094145dc0 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309159a700 1 -- 172.21.15.181:0/2007431008 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084009cb0 con 0x7f3094145dc0 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- 172.21.15.181:0/2007431008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 msgr2=0x7f30940c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 --2- 172.21.15.181:0/2007431008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f308400ba20 tx=0x7f3084006eb0).stop 2020-11-16T22:28:02.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- 172.21.15.181:0/2007431008 shutdown_connections 2020-11-16T22:28:02.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 --2- 172.21.15.181:0/2007431008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940c7af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 --2- 172.21.15.181:0/2007431008 >> v2:172.21.15.181:3300/0 conn(0x7f30941461a0 0x7f3094061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.797+0000 7f309359e700 1 -- 172.21.15.181:0/2007431008 >> 172.21.15.181:0/2007431008 conn(0x7f30940cf630 msgr2=0x7f30940cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:02.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- 172.21.15.181:0/2007431008 shutdown_connections 2020-11-16T22:28:02.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- 172.21.15.181:0/2007431008 wait complete. 2020-11-16T22:28:02.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 Processor -- start 2020-11-16T22:28:02.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- start start 2020-11-16T22:28:02.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 0x7f30940b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30941461a0 0x7f3094060f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3094061470 con 0x7f30940615f0 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30940b3760 con 0x7f30941461a0 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30941727c0 con 0x7f3094145dc0 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3092d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30941461a0 0x7f3094060f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37622/0 (socket says 172.21.15.181:37622) 2020-11-16T22:28:02.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 -- 172.21.15.181:0/4069307428 learned_addr learned my addr 172.21.15.181:0/4069307428 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309259c700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 0x7f30940b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 -- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30941461a0 msgr2=0x7f3094060f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30941461a0 0x7f3094060f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 -- 172.21.15.181:0/4069307428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 msgr2=0x7f30940b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 0x7f30940b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 -- 172.21.15.181:0/4069307428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094172a60 con 0x7f3094145dc0 2020-11-16T22:28:02.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309259c700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 0x7f30940b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f3091d9b700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f308800ebd0 tx=0x7f3088006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308800c8e0 con 0x7f3094145dc0 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308800ca40 con 0x7f3094145dc0 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308800cdc0 con 0x7f3094145dc0 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.801+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3094172d00 con 0x7f3094145dc0 2020-11-16T22:28:02.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3094173240 con 0x7f3094145dc0 2020-11-16T22:28:02.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3094173400 con 0x7f3094145dc0 2020-11-16T22:28:02.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f308801a030 con 0x7f3094145dc0 2020-11-16T22:28:02.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f30827fc700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 0x7f3068057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:02.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f308806d560 con 0x7f3094145dc0 2020-11-16T22:28:02.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f309259c700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 0x7f3068057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:02.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.805+0000 7f309259c700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 0x7f3068057520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f308400c010 tx=0x7f3084015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:02.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.809+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3088043ba0 con 0x7f3094145dc0 2020-11-16T22:28:02.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.965+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f30941735e0 con 0x7f3094145dc0 2020-11-16T22:28:02.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.965+0000 7f30827fc700 1 -- 172.21.15.181:0/4069307428 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v31) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3088018030 con 0x7f3094145dc0 2020-11-16T22:28:02.950 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:28:02.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 msgr2=0x7f3068057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 0x7f3068057520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f308400c010 tx=0x7f3084015000).stop 2020-11-16T22:28:02.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 msgr2=0x7f30940b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f308800ebd0 tx=0x7f3088006df0).stop 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 shutdown_connections 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3068055060 0x7f3068057520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30941461a0 0x7f3094060f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30940615f0 0x7f30940b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 --2- 172.21.15.181:0/4069307428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3094145dc0 0x7f30940b52f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:02.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 >> 172.21.15.181:0/4069307428 conn(0x7f30940cf630 msgr2=0x7f30940ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:02.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 shutdown_connections 2020-11-16T22:28:02.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:02.969+0000 7f309359e700 1 -- 172.21.15.181:0/4069307428 wait complete. 2020-11-16T22:28:02.971 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:28:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 Processor -- start 2020-11-16T22:28:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- start start 2020-11-16T22:28:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff08066230 0x7eff08066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff08066b00 con 0x7eff080632c0 2020-11-16T22:28:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff08063d00 con 0x7eff08066230 2020-11-16T22:28:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff08063fa0 con 0x7eff08063750 2020-11-16T22:28:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37630/0 (socket says 172.21.15.181:37630) 2020-11-16T22:28:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0d810700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff08066230 0x7eff08066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 -- 172.21.15.181:0/2897797854 learned_addr learned my addr 172.21.15.181:0/2897797854 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 -- 172.21.15.181:0/2897797854 >> v2:172.21.15.181:3300/0 conn(0x7eff08066230 msgr2=0x7eff08066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 --2- 172.21.15.181:0/2897797854 >> v2:172.21.15.181:3300/0 conn(0x7eff08066230 0x7eff08066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 -- 172.21.15.181:0/2897797854 >> v1:172.21.15.175:6789/0 conn(0x7eff080632c0 legacy=0x7eff080c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 -- 172.21.15.181:0/2897797854 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08064240 con 0x7eff08063750 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e011700 1 --2- 172.21.15.181:0/2897797854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7eff00009d50 tx=0x7eff00009290).ready entity=mon.2 client_cookie=673cc1f0eed014ce server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0c80e700 1 -- 172.21.15.181:0/2897797854 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000bc50 con 0x7eff08063750 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0c80e700 1 -- 172.21.15.181:0/2897797854 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000bdb0 con 0x7eff08063750 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0c80e700 1 -- 172.21.15.181:0/2897797854 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00011610 con 0x7eff08063750 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- 172.21.15.181:0/2897797854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 msgr2=0x7eff080c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 --2- 172.21.15.181:0/2897797854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7eff00009d50 tx=0x7eff00009290).stop 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- 172.21.15.181:0/2897797854 shutdown_connections 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 --2- 172.21.15.181:0/2897797854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff08063750 0x7eff080c8b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 --2- 172.21.15.181:0/2897797854 >> v2:172.21.15.181:3300/0 conn(0x7eff08066230 0x7eff08066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- 172.21.15.181:0/2897797854 >> 172.21.15.181:0/2897797854 conn(0x7eff080cfe90 msgr2=0x7eff080d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- 172.21.15.181:0/2897797854 shutdown_connections 2020-11-16T22:28:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.117+0000 7eff0e812700 1 -- 172.21.15.181:0/2897797854 wait complete. 2020-11-16T22:28:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 Processor -- start 2020-11-16T22:28:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 -- start start 2020-11-16T22:28:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff08063750 0x7eff0814b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 0x7eff080ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08173590 con 0x7eff08063750 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08173780 con 0x7eff08066230 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e812700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff08173a20 con 0x7eff080632c0 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37634/0 (socket says 172.21.15.181:37634) 2020-11-16T22:28:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 -- 172.21.15.181:0/2712237317 learned_addr learned my addr 172.21.15.181:0/2712237317 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d00f700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff08063750 0x7eff0814b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e011700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 0x7eff080ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 -- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 msgr2=0x7eff080ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 0x7eff080ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 -- 172.21.15.181:0/2712237317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff08063750 msgr2=0x7eff0814b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff08063750 0x7eff0814b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0d810700 1 -- 172.21.15.181:0/2712237317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08173cc0 con 0x7eff080632c0 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.121+0000 7eff0e011700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 0x7eff080ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0d810700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7eff0400ebd0 tx=0x7eff04006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400c8e0 con 0x7eff080632c0 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff08173f60 con 0x7eff080632c0 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0400ca40 con 0x7eff080632c0 2020-11-16T22:28:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400cdc0 con 0x7eff080632c0 2020-11-16T22:28:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff080cebc0 con 0x7eff080632c0 2020-11-16T22:28:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff08044f60 con 0x7eff080632c0 2020-11-16T22:28:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7eff04012600 con 0x7eff080632c0 2020-11-16T22:28:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 0x7efee0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7eff0406de00 con 0x7eff080632c0 2020-11-16T22:28:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0d00f700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 0x7efee0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.125+0000 7eff0d00f700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 0x7efee0057520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efefc003300 tx=0x7efefc007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.129+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7eff04044390 con 0x7eff080632c0 2020-11-16T22:28:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.281+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7eff080d0d10 con 0x7eff080632c0 2020-11-16T22:28:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.281+0000 7efef9ffb700 1 -- 172.21.15.181:0/2712237317 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7eff040479b0 con 0x7eff080632c0 2020-11-16T22:28:03.267 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 msgr2=0x7efee0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 0x7efee0057520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efefc003300 tx=0x7efefc007b90).stop 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 msgr2=0x7eff0814ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7eff0400ebd0 tx=0x7eff04006df0).stop 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 shutdown_connections 2020-11-16T22:28:03.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efee0055060 0x7efee0057520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff08066230 0x7eff080ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff080632c0 0x7eff0814ab50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 --2- 172.21.15.181:0/2712237317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff08063750 0x7eff0814b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 >> 172.21.15.181:0/2712237317 conn(0x7eff080cfe90 msgr2=0x7eff080cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 shutdown_connections 2020-11-16T22:28:03.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.285+0000 7eff0e812700 1 -- 172.21.15.181:0/2712237317 wait complete. 2020-11-16T22:28:03.287 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:28:03.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 Processor -- start 2020-11-16T22:28:03.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 -- start start 2020-11-16T22:28:03.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a0167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a00b3220 0x7f25a00b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25a0166bb0 0x7f25a0167060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00b5f00 con 0x7f25a0166bb0 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00b3b90 con 0x7f25a00b3220 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a869e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0168390 con 0x7f25a0167ac0 2020-11-16T22:28:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a00b3220 0x7f25a00b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a00b3220 0x7f25a00b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58032/0 (socket says 172.21.15.181:58032) 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a643a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a0167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 -- 172.21.15.181:0/1411540604 learned_addr learned my addr 172.21.15.181:0/1411540604 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a6c3b700 1 --2- 172.21.15.181:0/1411540604 >> v2:172.21.15.175:3300/0 conn(0x7f25a0166bb0 0x7f25a0167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 -- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 msgr2=0x7f25a0167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 --2- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a0167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 -- 172.21.15.181:0/1411540604 >> v2:172.21.15.175:3300/0 conn(0x7f25a0166bb0 msgr2=0x7f25a0167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 --2- 172.21.15.181:0/1411540604 >> v2:172.21.15.175:3300/0 conn(0x7f25a0166bb0 0x7f25a0167060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 -- 172.21.15.181:0/1411540604 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00b2060 con 0x7f25a00b3220 2020-11-16T22:28:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5c39700 1 --2- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a00b3220 0x7f25a00b36a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f25a005ce10 tx=0x7f259c009290).ready entity=mon.1 client_cookie=9ab4e9a3cbdcdfa4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5438700 1 -- 172.21.15.181:0/1411540604 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00bbc0 con 0x7f25a00b3220 2020-11-16T22:28:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5438700 1 -- 172.21.15.181:0/1411540604 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c00bd20 con 0x7f25a00b3220 2020-11-16T22:28:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.421+0000 7f25a5438700 1 -- 172.21.15.181:0/1411540604 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c011610 con 0x7f25a00b3220 2020-11-16T22:28:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 -- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a00b3220 msgr2=0x7f25a00b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 --2- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a00b3220 0x7f25a00b36a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f25a005ce10 tx=0x7f259c009290).stop 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 -- 172.21.15.181:0/1411540604 shutdown_connections 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 --2- 172.21.15.181:0/1411540604 >> v2:172.21.15.175:3300/0 conn(0x7f25a0166bb0 0x7f25a0167060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 --2- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a0167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 --2- 172.21.15.181:0/1411540604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a00b3220 0x7f25a00b36a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 -- 172.21.15.181:0/1411540604 >> 172.21.15.181:0/1411540604 conn(0x7f25a0060bc0 msgr2=0x7f25a00d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 -- 172.21.15.181:0/1411540604 shutdown_connections 2020-11-16T22:28:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 -- 172.21.15.181:0/1411540604 wait complete. 2020-11-16T22:28:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.425+0000 7f25a869e700 1 Processor -- start 2020-11-16T22:28:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- start start 2020-11-16T22:28:03.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a00b3220 0x7f25a00ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 0x7f25a00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0175a20 con 0x7f25a00b3220 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0175cc0 con 0x7f25a0166bb0 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0175f60 con 0x7f25a0167ac0 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a5c39700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 0x7f25a00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37652/0 (socket says 172.21.15.181:37652) 2020-11-16T22:28:03.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 -- 172.21.15.181:0/1914656299 learned_addr learned my addr 172.21.15.181:0/1914656299 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a643a700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a00b3220 0x7f25a00ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 -- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 msgr2=0x7f25a00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 0x7f25a00cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 -- 172.21.15.181:0/1914656299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a00b3220 msgr2=0x7f25a00ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a00b3220 0x7f25a00ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 -- 172.21.15.181:0/1914656299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0176200 con 0x7f25a0167ac0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a5c39700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 0x7f25a00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a6c3b700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f25980067d0 tx=0x7f2598008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800f5e0 con 0x7f25a0167ac0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a01764a0 con 0x7f25a0167ac0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800f740 con 0x7f25a0167ac0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800fae0 con 0x7f25a0167ac0 2020-11-16T22:28:03.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.429+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a00b7950 con 0x7f25a0167ac0 2020-11-16T22:28:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a0069eb0 con 0x7f25a0167ac0 2020-11-16T22:28:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f259801c460 con 0x7f25a0167ac0 2020-11-16T22:28:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25967fc700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 0x7f2578057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25a643a700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 0x7f2578057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f259806da90 con 0x7f25a0167ac0 2020-11-16T22:28:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.433+0000 7f25a643a700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 0x7f2578057520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2590006fd0 tx=0x7f2590006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.441+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2598043fc0 con 0x7f25a0167ac0 2020-11-16T22:28:03.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.597+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25a0061100 con 0x7f25a0167ac0 2020-11-16T22:28:03.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.597+0000 7f25967fc700 1 -- 172.21.15.181:0/1914656299 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7f25980475e0 con 0x7f25a0167ac0 2020-11-16T22:28:03.582 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:03.582 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:28:03.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:28:03.583 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:28:03.584 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:28:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 msgr2=0x7f2578057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 0x7f2578057520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2590006fd0 tx=0x7f2590006df0).stop 2020-11-16T22:28:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 msgr2=0x7f25a00b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f25980067d0 tx=0x7f2598008040).stop 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 shutdown_connections 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055060 0x7f2578057520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a0167ac0 0x7f25a00b7460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a00b3220 0x7f25a00ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 --2- 172.21.15.181:0/1914656299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a0166bb0 0x7f25a00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 >> 172.21.15.181:0/1914656299 conn(0x7f25a0060bc0 msgr2=0x7f25a00ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:03.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 shutdown_connections 2020-11-16T22:28:03.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.601+0000 7f25a869e700 1 -- 172.21.15.181:0/1914656299 wait complete. 2020-11-16T22:28:03.601 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:28:03.601 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:28:03.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 Processor -- start 2020-11-16T22:28:03.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 -- start start 2020-11-16T22:28:03.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f271c13e870 0x7f271c13ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 0x7f271c160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 0x7f271c061570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c160730 con 0x7f271c13e870 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c15d180 con 0x7f271c15fdc0 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c13f140 con 0x7f271c061100 2020-11-16T22:28:03.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 0x7f271c160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 0x7f271c160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58048/0 (socket says 172.21.15.181:58048) 2020-11-16T22:28:03.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 0x7f271c061570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271a7fc700 1 -- 172.21.15.181:0/2650974010 learned_addr learned my addr 172.21.15.181:0/2650974010 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.745+0000 7f271affd700 1 --2- 172.21.15.181:0/2650974010 >> v2:172.21.15.175:3300/0 conn(0x7f271c13e870 0x7f271c13ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 -- 172.21.15.181:0/2650974010 >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 msgr2=0x7f271c160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:03.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 --2- 172.21.15.181:0/2650974010 >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 0x7f271c160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 -- 172.21.15.181:0/2650974010 >> v2:172.21.15.175:3300/0 conn(0x7f271c13e870 msgr2=0x7f271c13ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 --2- 172.21.15.181:0/2650974010 >> v2:172.21.15.175:3300/0 conn(0x7f271c13e870 0x7f271c13ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 -- 172.21.15.181:0/2650974010 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c15d810 con 0x7f271c061100 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271b7fe700 1 --2- 172.21.15.181:0/2650974010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 0x7f271c061570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f271c0c9ed0 tx=0x7f27140091d0).ready entity=mon.2 client_cookie=4b730dc9af516cbc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f2719ffb700 1 -- 172.21.15.181:0/2650974010 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400ba70 con 0x7f271c061100 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f2719ffb700 1 -- 172.21.15.181:0/2650974010 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271400bbd0 con 0x7f271c061100 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f2719ffb700 1 -- 172.21.15.181:0/2650974010 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714011400 con 0x7f271c061100 2020-11-16T22:28:03.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 -- 172.21.15.181:0/2650974010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 msgr2=0x7f271c061570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 --2- 172.21.15.181:0/2650974010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 0x7f271c061570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f271c0c9ed0 tx=0x7f27140091d0).stop 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 -- 172.21.15.181:0/2650974010 shutdown_connections 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 --2- 172.21.15.181:0/2650974010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c061100 0x7f271c061570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 --2- 172.21.15.181:0/2650974010 >> v2:172.21.15.175:3300/0 conn(0x7f271c13e870 0x7f271c13ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 --2- 172.21.15.181:0/2650974010 >> v2:172.21.15.181:3300/0 conn(0x7f271c15fdc0 0x7f271c160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 -- 172.21.15.181:0/2650974010 >> 172.21.15.181:0/2650974010 conn(0x7f271c0cf630 msgr2=0x7f271c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:03.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 -- 172.21.15.181:0/2650974010 shutdown_connections 2020-11-16T22:28:03.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.749+0000 7f271bfff700 1 -- 172.21.15.181:0/2650974010 wait complete. 2020-11-16T22:28:03.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 Processor -- start 2020-11-16T22:28:03.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- start start 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271c061100 0x7f271c0b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f271c15fdc0 0x7f271c0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0cb380 con 0x7f271c15fdc0 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0cb620 con 0x7f271c061100 2020-11-16T22:28:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0cb8c0 con 0x7f271c13e870 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37662/0 (socket says 172.21.15.181:37662) 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 -- 172.21.15.181:0/63972687 learned_addr learned my addr 172.21.15.181:0/63972687 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271affd700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271c061100 0x7f271c0b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 -- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271c061100 msgr2=0x7f271c0b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271c061100 0x7f271c0b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:03.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 -- 172.21.15.181:0/63972687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f271c15fdc0 msgr2=0x7f271c0ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f271c15fdc0 0x7f271c0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 -- 172.21.15.181:0/63972687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0cbb60 con 0x7f271c13e870 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271a7fc700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f271000f830 tx=0x7f2710008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271000d6c0 con 0x7f271c13e870 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271000d820 con 0x7f271c13e870 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c0cbe00 con 0x7f271c13e870 2020-11-16T22:28:03.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271000dba0 con 0x7f271c13e870 2020-11-16T22:28:03.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.753+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c0b5b20 con 0x7f271c13e870 2020-11-16T22:28:03.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c044f60 con 0x7f271c13e870 2020-11-16T22:28:03.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2710024070 con 0x7f271c13e870 2020-11-16T22:28:03.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f2702ffd700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 0x7f26f8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:03.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f271006e020 con 0x7f271c13e870 2020-11-16T22:28:03.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f271affd700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 0x7f26f8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:03.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.757+0000 7f271affd700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 0x7f26f8057520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f270c00b500 tx=0x7f270c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:03.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.765+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f27100445b0 con 0x7f271c13e870 2020-11-16T22:28:03.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:03.921+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f271c13e130 con 0x7f271c13e870 2020-11-16T22:28:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f2702ffd700 1 -- 172.21.15.181:0/63972687 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2710018030 con 0x7f271c13e870 2020-11-16T22:28:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 msgr2=0x7f26f8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 0x7f26f8057520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f270c00b500 tx=0x7f270c008040).stop 2020-11-16T22:28:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 msgr2=0x7f271c0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f271000f830 tx=0x7f2710008040).stop 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 shutdown_connections 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f26f8055060 0x7f26f8057520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f271c15fdc0 0x7f271c0ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271c061100 0x7f271c0b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 --2- 172.21.15.181:0/63972687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f271c13e870 0x7f271c0ca280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 >> 172.21.15.181:0/63972687 conn(0x7f271c0cf630 msgr2=0x7f271c0cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 shutdown_connections 2020-11-16T22:28:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:04.925+0000 7f271bfff700 1 -- 172.21.15.181:0/63972687 wait complete. 2020-11-16T22:28:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:28:04.926 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:28:05.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.061+0000 7fa2a294b700 1 Processor -- start 2020-11-16T22:28:05.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.061+0000 7fa2a294b700 1 -- start start 2020-11-16T22:28:05.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa29c0629b0 0x7fa29c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29c160e40 con 0x7fa29c0615f0 2020-11-16T22:28:05.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29c160ba0 con 0x7fa29c15f5b0 2020-11-16T22:28:05.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29c1609a0 con 0x7fa29c0629b0 2020-11-16T22:28:05.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa29c15f5b0 0x7fa29c0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44134/0 (socket says 172.21.15.181:44134) 2020-11-16T22:28:05.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29bfff700 1 -- 172.21.15.181:0/203127577 learned_addr learned my addr 172.21.15.181:0/203127577 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:05.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/203127577 >> v2:172.21.15.175:3300/0 conn(0x7fa29c0629b0 0x7fa29c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204347520 0 0) 0x7fa29c160ba0 con 0x7fa29c15f5b0 2020-11-16T22:28:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa27c003240 con 0x7fa29c15f5b0 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2970177092 0 0) 0x7fa27c003240 con 0x7fa29c15f5b0 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 >> v2:172.21.15.175:3300/0 conn(0x7fa29c0629b0 msgr2=0x7fa29c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 --2- 172.21.15.181:0/203127577 >> v2:172.21.15.175:3300/0 conn(0x7fa29c0629b0 0x7fa29c080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 >> v1:172.21.15.175:6789/0 conn(0x7fa29c0615f0 legacy=0x7fa29c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c060970 con 0x7fa29c15f5b0 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7fa28c001660 con 0x7fa29c15f5b0 2020-11-16T22:28:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 >> v1:172.21.15.181:6789/0 conn(0x7fa29c15f5b0 legacy=0x7fa29c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c003730 0x7fa27c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa27c0044e0 0x7fa27c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa27c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c1609a0 con 0x7fa27c0044e0 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c004350 con 0x7fa27c003730 2020-11-16T22:28:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c00bc50 con 0x7fa27c004e50 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa27c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29bfff700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c003730 0x7fa27c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29b7fe700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa27c0044e0 0x7fa27c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 -- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c003730 msgr2=0x7fa27c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa27c003730 0x7fa27c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 -- 172.21.15.181:0/203127577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa27c0044e0 msgr2=0x7fa27c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa27c0044e0 0x7fa27c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 -- 172.21.15.181:0/203127577 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c003240 con 0x7fa27c004e50 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 -- 172.21.15.181:0/203127577 reap_dead start 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa27c00b730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa29400bbf0 tx=0x7fa294006df0).ready entity=mon.2 client_cookie=913bc59091626dc9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294016070 con 0x7fa27c004e50 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400b300 con 0x7fa27c004e50 2020-11-16T22:28:05.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa29affd700 1 -- 172.21.15.181:0/203127577 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400ecc0 con 0x7fa27c004e50 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 msgr2=0x7fa27c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa27c00b730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa29400bbf0 tx=0x7fa294006df0).stop 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- 172.21.15.181:0/203127577 shutdown_connections 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 --2- 172.21.15.181:0/203127577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa27c00b730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- 172.21.15.181:0/203127577 >> 172.21.15.181:0/203127577 conn(0x7fa29c0d69a0 msgr2=0x7fa29c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- 172.21.15.181:0/203127577 shutdown_connections 2020-11-16T22:28:05.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.065+0000 7fa2a294b700 1 -- 172.21.15.181:0/203127577 wait complete. 2020-11-16T22:28:05.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 Processor -- start 2020-11-16T22:28:05.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- start start 2020-11-16T22:28:05.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c16fa10 0x7fa29c16fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa29c0d2ef0 0x7fa29c0d3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0cbdf0 con 0x7fa29c0d2ef0 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c15b660 con 0x7fa29c16fa10 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c15b900 con 0x7fa27c004e50 2020-11-16T22:28:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c16fa10 0x7fa29c16fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37682/0 (socket says 172.21.15.181:37682) 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 -- 172.21.15.181:0/1095594831 learned_addr learned my addr 172.21.15.181:0/1095594831 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a0ee8700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa29c0d2ef0 0x7fa29c0d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 -- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c16fa10 msgr2=0x7fa29c16fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c16fa10 0x7fa29c16fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 -- 172.21.15.181:0/1095594831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa29c0d2ef0 msgr2=0x7fa29c0d3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa29c0d2ef0 0x7fa29c0d3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 -- 172.21.15.181:0/1095594831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c15bba0 con 0x7fa27c004e50 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa29bfff700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa28c0015f0 tx=0x7fa29c0b8580).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c006e70 con 0x7fa27c004e50 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c001d70 con 0x7fa27c004e50 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00f8c0 con 0x7fa27c004e50 2020-11-16T22:28:05.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c15be40 con 0x7fa27c004e50 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.069+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c15c380 con 0x7fa27c004e50 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa28c01a030 con 0x7fa27c004e50 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c069eb0 con 0x7fa27c004e50 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa293fff700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 0x7fa278057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa28c06d170 con 0x7fa27c004e50 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa29b7fe700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 0x7fa278057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.073+0000 7fa29b7fe700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 0x7fa278057550 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa2840012a0 tx=0x7fa28400c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.081+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa28c043650 con 0x7fa27c004e50 2020-11-16T22:28:05.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.233+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa29c1642f0 con 0x7fa27c004e50 2020-11-16T22:28:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.933+0000 7fa293fff700 1 -- 172.21.15.181:0/1095594831 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v33) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa28c046c70 con 0x7fa27c004e50 2020-11-16T22:28:05.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 msgr2=0x7fa278057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 0x7fa278057550 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa2840012a0 tx=0x7fa28400c040).stop 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 msgr2=0x7fa29c0cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa28c0015f0 tx=0x7fa29c0b8580).stop 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 shutdown_connections 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa278055090 0x7fa278057550 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa29c0d2ef0 0x7fa29c0d3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27c004e50 0x7fa29c0cb900 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0x7fa28c0015f0 tx=0x7fa29c0b8580).stop 2020-11-16T22:28:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 --2- 172.21.15.181:0/1095594831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c16fa10 0x7fa29c16fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 >> 172.21.15.181:0/1095594831 conn(0x7fa29c0d69a0 msgr2=0x7fa29c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 shutdown_connections 2020-11-16T22:28:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:05.937+0000 7fa2a294b700 1 -- 172.21.15.181:0/1095594831 wait complete. 2020-11-16T22:28:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:28:05.938 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:28:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.077+0000 7f8c65440700 1 Processor -- start 2020-11-16T22:28:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.077+0000 7f8c65440700 1 -- start start 2020-11-16T22:28:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8c6013f710 0x7f8c60066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c60065460 0x7f8c60062980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c600659d0 con 0x7f8c6013f330 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c600b61c0 con 0x7f8c6013f710 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c60065c70 con 0x7f8c60065460 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c60065460 0x7f8c60062980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c60065460 0x7f8c60062980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58084/0 (socket says 172.21.15.181:58084) 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2206515588 learned_addr learned my addr 172.21.15.181:0/2206515588 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5e7fc700 1 --2- 172.21.15.181:0/2206515588 >> v2:172.21.15.175:3300/0 conn(0x7f8c6013f710 0x7f8c60066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2206515588 >> v2:172.21.15.175:3300/0 conn(0x7f8c6013f710 msgr2=0x7f8c60066b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 --2- 172.21.15.181:0/2206515588 >> v2:172.21.15.175:3300/0 conn(0x7f8c6013f710 0x7f8c60066b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2206515588 >> v1:172.21.15.175:6789/0 conn(0x7f8c6013f330 legacy=0x7f8c6015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2206515588 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60065f10 con 0x7f8c60065460 2020-11-16T22:28:06.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5f7fe700 1 --2- 172.21.15.181:0/2206515588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c60062980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8c600626a0 tx=0x7f8c58006ea0).ready entity=mon.1 client_cookie=2b03e62fedebc94c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5dffb700 1 -- 172.21.15.181:0/2206515588 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58006030 con 0x7f8c60065460 2020-11-16T22:28:06.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5dffb700 1 -- 172.21.15.181:0/2206515588 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58006190 con 0x7f8c60065460 2020-11-16T22:28:06.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c5dffb700 1 -- 172.21.15.181:0/2206515588 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58006510 con 0x7f8c60065460 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- 172.21.15.181:0/2206515588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 msgr2=0x7f8c60062980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 --2- 172.21.15.181:0/2206515588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c60062980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8c600626a0 tx=0x7f8c58006ea0).stop 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- 172.21.15.181:0/2206515588 shutdown_connections 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 --2- 172.21.15.181:0/2206515588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c60062980 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 --2- 172.21.15.181:0/2206515588 >> v2:172.21.15.175:3300/0 conn(0x7f8c6013f710 0x7f8c60066b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- 172.21.15.181:0/2206515588 >> 172.21.15.181:0/2206515588 conn(0x7f8c600d35d0 msgr2=0x7f8c600d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:06.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.081+0000 7f8c65440700 1 -- 172.21.15.181:0/2206515588 shutdown_connections 2020-11-16T22:28:06.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.085+0000 7f8c65440700 1 -- 172.21.15.181:0/2206515588 wait complete. 2020-11-16T22:28:06.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.085+0000 7f8c65440700 1 Processor -- start 2020-11-16T22:28:06.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.085+0000 7f8c65440700 1 -- start start 2020-11-16T22:28:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c6013f330 0x7f8c60160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 0x7f8c6013b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600c9910 con 0x7f8c6013f330 2020-11-16T22:28:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600c9bb0 con 0x7f8c60065460 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600c9e50 con 0x7f8c6013f710 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 0x7f8c6013b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58086/0 (socket says 172.21.15.181:58086) 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5effd700 1 -- 172.21.15.181:0/2717578841 learned_addr learned my addr 172.21.15.181:0/2717578841 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5e7fc700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c6013f330 0x7f8c60160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 msgr2=0x7f8c600d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2717578841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c6013f330 msgr2=0x7f8c60160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c6013f330 0x7f8c60160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 -- 172.21.15.181:0/2717578841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600ca0f0 con 0x7f8c6013f710 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5effd700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c5f7fe700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 0x7f8c6013b3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8c600626a0 tx=0x7f8c58015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58005a80 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58005be0 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5801d450 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6013ac70 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.089+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c601375f0 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c60069eb0 con 0x7f8c6013f710 2020-11-16T22:28:06.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8c5800f070 con 0x7f8c6013f710 2020-11-16T22:28:06.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c56ffd700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 0x7f8c3c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f8c5806eec0 con 0x7f8c6013f710 2020-11-16T22:28:06.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c5effd700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 0x7f8c3c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.093+0000 7f8c5effd700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 0x7f8c3c057520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f8c601570e0 tx=0x7f8c50002420).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.101+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8c580452c0 con 0x7f8c6013f710 2020-11-16T22:28:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.305+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8c60064fe0 con 0x7f8c6013f710 2020-11-16T22:28:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.317+0000 7f8c56ffd700 1 -- 172.21.15.181:0/2717578841 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v27) v1 ==== 187+0+0 (secure 0 0 0) 0x7f8c58022070 con 0x7f8c6013f710 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 msgr2=0x7f8c3c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 0x7f8c3c057520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f8c601570e0 tx=0x7f8c50002420).stop 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 msgr2=0x7f8c6013b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 0x7f8c6013b3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8c600626a0 tx=0x7f8c58015000).stop 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 shutdown_connections 2020-11-16T22:28:06.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c3c055060 0x7f8c3c057520 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c6013f710 0x7f8c6013b3b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c60065460 0x7f8c600d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 --2- 172.21.15.181:0/2717578841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c6013f330 0x7f8c60160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.321+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 >> 172.21.15.181:0/2717578841 conn(0x7f8c600d35d0 msgr2=0x7f8c6013c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:06.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.325+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 shutdown_connections 2020-11-16T22:28:06.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.325+0000 7f8c65440700 1 -- 172.21.15.181:0/2717578841 wait complete. 2020-11-16T22:28:06.310 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 5 and data pool 6 2020-11-16T22:28:06.325 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:28:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 Processor -- start 2020-11-16T22:28:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- start start 2020-11-16T22:28:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf8061b50 con 0x7f8bf81492c0 2020-11-16T22:28:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80618b0 con 0x7f8bf8145a30 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80cb930 con 0x7f8bf80816e0 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37708/0 (socket says 172.21.15.181:37708) 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58096/0 (socket says 172.21.15.181:58096) 2020-11-16T22:28:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 -- 172.21.15.181:0/2339097473 learned_addr learned my addr 172.21.15.181:0/2339097473 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:06.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bef7fe700 1 --1- 172.21.15.181:0/2339097473 >> v1:172.21.15.181:6789/0 conn(0x7f8bf81492c0 0x7f8bf8080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44166/0 (socket says 172.21.15.181:44166) 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 -- 172.21.15.181:0/2339097473 >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 msgr2=0x7f8bf8145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 --2- 172.21.15.181:0/2339097473 >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 -- 172.21.15.181:0/2339097473 >> v1:172.21.15.181:6789/0 conn(0x7f8bf81492c0 legacy=0x7f8bf8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 -- 172.21.15.181:0/2339097473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80cbbd0 con 0x7f8bf80816e0 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf7fff700 1 --2- 172.21.15.181:0/2339097473 >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfcc86700 1 --2- 172.21.15.181:0/2339097473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8bf0009d50 tx=0x7f8bf0009290).ready entity=mon.2 client_cookie=6aae707d19fe30ce server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf77fe700 1 -- 172.21.15.181:0/2339097473 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000bbc0 con 0x7f8bf80816e0 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf77fe700 1 -- 172.21.15.181:0/2339097473 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000bd20 con 0x7f8bf80816e0 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bf77fe700 1 -- 172.21.15.181:0/2339097473 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0011610 con 0x7f8bf80816e0 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- 172.21.15.181:0/2339097473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 msgr2=0x7f8bf80cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 --2- 172.21.15.181:0/2339097473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8bf0009d50 tx=0x7f8bf0009290).stop 2020-11-16T22:28:06.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- 172.21.15.181:0/2339097473 shutdown_connections 2020-11-16T22:28:06.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 --2- 172.21.15.181:0/2339097473 >> v2:172.21.15.181:3300/0 conn(0x7f8bf8145a30 0x7f8bf8145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 --2- 172.21.15.181:0/2339097473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80cc5b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- 172.21.15.181:0/2339097473 >> 172.21.15.181:0/2339097473 conn(0x7f8bf80d68c0 msgr2=0x7f8bf80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:06.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- 172.21.15.181:0/2339097473 shutdown_connections 2020-11-16T22:28:06.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.449+0000 7f8bfd487700 1 -- 172.21.15.181:0/2339097473 wait complete. 2020-11-16T22:28:06.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 Processor -- start 2020-11-16T22:28:06.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- start start 2020-11-16T22:28:06.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 0x7f8bf80cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d2cb0 con 0x7f8bf81492c0 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d2f50 con 0x7f8bf8145a30 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d31f0 con 0x7f8bf80816e0 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58100/0 (socket says 172.21.15.181:58100) 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 -- 172.21.15.181:0/853996708 learned_addr learned my addr 172.21.15.181:0/853996708 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfcc86700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 0x7f8bf80cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 -- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 msgr2=0x7f8bf80ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 -- 172.21.15.181:0/853996708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 msgr2=0x7f8bf80cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 0x7f8bf80cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 -- 172.21.15.181:0/853996708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80d3490 con 0x7f8bf8145a30 2020-11-16T22:28:06.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bf7fff700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfcc86700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 0x7f8bf80cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bef7fe700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f8be000c710 tx=0x7f8be0007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00072e0 con 0x7f8bf8145a30 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0005530 con 0x7f8bf8145a30 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf80d3730 con 0x7f8bf8145a30 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00058b0 con 0x7f8bf8145a30 2020-11-16T22:28:06.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.453+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf813e280 con 0x7f8bf8145a30 2020-11-16T22:28:06.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf80cc2f0 con 0x7f8bf8145a30 2020-11-16T22:28:06.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8be001a060 con 0x7f8bf8145a30 2020-11-16T22:28:06.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bf4ff9700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 0x7f8bd4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:06.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8be006d4e0 con 0x7f8bf8145a30 2020-11-16T22:28:06.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bf7fff700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 0x7f8bd4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:06.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.457+0000 7f8bf7fff700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 0x7f8bd4057550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8be8006fd0 tx=0x7f8be8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:06.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.461+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8be0043740 con 0x7f8bf8145a30 2020-11-16T22:28:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.613+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf8044f60 con 0x7f8bf8145a30 2020-11-16T22:28:06.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.613+0000 7f8bf4ff9700 1 -- 172.21.15.181:0/853996708 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f8be0018020 con 0x7f8bf8145a30 2020-11-16T22:28:06.600 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:06.600 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":34,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:06.309707+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:28:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 msgr2=0x7f8bd4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 0x7f8bd4057550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8be8006fd0 tx=0x7f8be8006df0).stop 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 msgr2=0x7f8bf80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f8be000c710 tx=0x7f8be0007c40).stop 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 shutdown_connections 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bd4055090 0x7f8bd4057550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bf81492c0 0x7f8bf80cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80816e0 0x7f8bf80ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 --2- 172.21.15.181:0/853996708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf8145a30 0x7f8bf80cfa70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 >> 172.21.15.181:0/853996708 conn(0x7f8bf80d68c0 msgr2=0x7f8bf81480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 shutdown_connections 2020-11-16T22:28:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:06.617+0000 7f8bfd487700 1 -- 172.21.15.181:0/853996708 wait complete. 2020-11-16T22:28:07.418 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:28:07.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.561+0000 7facab17f700 1 Processor -- start 2020-11-16T22:28:07.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.561+0000 7facab17f700 1 -- start start 2020-11-16T22:28:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 0x7faca40b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 0x7faca4080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca4061840 con 0x7faca414a280 2020-11-16T22:28:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca4061ae0 con 0x7faca40ccc50 2020-11-16T22:28:07.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40cb8c0 con 0x7faca414a660 2020-11-16T22:28:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 0x7faca40b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facaa17d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faca414a280 0x7faca40cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44180/0 (socket says 172.21.15.181:44180) 2020-11-16T22:28:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facaa17d700 1 -- 172.21.15.181:0/3997982587 learned_addr learned my addr 172.21.15.181:0/3997982587 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facaa97e700 1 --2- 172.21.15.181:0/3997982587 >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 0x7faca4080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 -- 172.21.15.181:0/3997982587 >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 msgr2=0x7faca4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 --2- 172.21.15.181:0/3997982587 >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 0x7faca4080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 -- 172.21.15.181:0/3997982587 >> v1:172.21.15.181:6789/0 conn(0x7faca414a280 legacy=0x7faca40cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca917b700 1 -- 172.21.15.181:0/3997982587 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889185046 0 0) 0x7faca4061840 con 0x7faca414a280 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 -- 172.21.15.181:0/3997982587 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40cbb60 con 0x7faca414a660 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facaa97e700 1 --2- 172.21.15.181:0/3997982587 >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 0x7faca4080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca997c700 1 --2- 172.21.15.181:0/3997982587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 0x7faca40b6720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7faca000b9a0 tx=0x7faca0006df0).ready entity=mon.2 client_cookie=ab283df8115a8e71 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca917b700 1 -- 172.21.15.181:0/3997982587 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00097b0 con 0x7faca414a660 2020-11-16T22:28:07.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca917b700 1 -- 172.21.15.181:0/3997982587 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca0009910 con 0x7faca414a660 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7faca917b700 1 -- 172.21.15.181:0/3997982587 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0009c90 con 0x7faca414a660 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- 172.21.15.181:0/3997982587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 msgr2=0x7faca40b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 --2- 172.21.15.181:0/3997982587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 0x7faca40b6720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7faca000b9a0 tx=0x7faca0006df0).stop 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- 172.21.15.181:0/3997982587 shutdown_connections 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 --2- 172.21.15.181:0/3997982587 >> v2:172.21.15.175:3300/0 conn(0x7faca40ccc50 0x7faca4080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 --2- 172.21.15.181:0/3997982587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca414a660 0x7faca40b6720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- 172.21.15.181:0/3997982587 >> 172.21.15.181:0/3997982587 conn(0x7faca40d68c0 msgr2=0x7faca40d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- 172.21.15.181:0/3997982587 shutdown_connections 2020-11-16T22:28:07.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.565+0000 7facab17f700 1 -- 172.21.15.181:0/3997982587 wait complete. 2020-11-16T22:28:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 Processor -- start 2020-11-16T22:28:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- start start 2020-11-16T22:28:07.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 0x7faca41463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca414a280 0x7faca41468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 0x7faca41643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4173f20 con 0x7faca414a280 2020-11-16T22:28:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca41741c0 con 0x7faca414a660 2020-11-16T22:28:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4174460 con 0x7faca40ccc50 2020-11-16T22:28:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa97e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 0x7faca41643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 0x7faca41463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa97e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 0x7faca41643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58116/0 (socket says 172.21.15.181:58116) 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa97e700 1 -- 172.21.15.181:0/1054479345 learned_addr learned my addr 172.21.15.181:0/1054479345 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7faca997c700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca414a280 0x7faca41468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 -- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 msgr2=0x7faca41643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 0x7faca41643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 -- 172.21.15.181:0/1054479345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca414a280 msgr2=0x7faca41468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca414a280 0x7faca41468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 -- 172.21.15.181:0/1054479345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4174700 con 0x7faca40ccc50 2020-11-16T22:28:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facaa17d700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 0x7faca41463b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fac9c007fd0 tx=0x7fac9c007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0072e0 con 0x7faca40ccc50 2020-11-16T22:28:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c007440 con 0x7faca40ccc50 2020-11-16T22:28:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c005840 con 0x7faca40ccc50 2020-11-16T22:28:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4165770 con 0x7faca40ccc50 2020-11-16T22:28:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.569+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca41749a0 con 0x7faca40ccc50 2020-11-16T22:28:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4044f60 con 0x7faca40ccc50 2020-11-16T22:28:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fac9c010070 con 0x7faca40ccc50 2020-11-16T22:28:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7fac9a7fc700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 0x7fac7c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7faca997c700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 0x7fac7c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac9c06d3c0 con 0x7faca40ccc50 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.573+0000 7faca997c700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 0x7fac7c057520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7faca000b150 tx=0x7faca0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.577+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fac9c043620 con 0x7faca40ccc50 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.729+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faca41652c0 con 0x7faca40ccc50 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7fac9a7fc700 1 -- 172.21.15.181:0/1054479345 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fac9c046c40 con 0x7faca40ccc50 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 msgr2=0x7fac7c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 0x7fac7c057520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7faca000b150 tx=0x7faca0015000).stop 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 msgr2=0x7faca41463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 0x7faca41463b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fac9c007fd0 tx=0x7fac9c007a00).stop 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 shutdown_connections 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac7c055060 0x7fac7c057520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faca414a660 0x7faca41643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faca40ccc50 0x7faca41463b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 --2- 172.21.15.181:0/1054479345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faca414a280 0x7faca41468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 >> 172.21.15.181:0/1054479345 conn(0x7faca40d68c0 msgr2=0x7faca40d5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 shutdown_connections 2020-11-16T22:28:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:07.733+0000 7facab17f700 1 -- 172.21.15.181:0/1054479345 wait complete. 2020-11-16T22:28:07.968 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:07.968 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:07.314280+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:28:08.578 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:28:08.852 INFO:tasks.ceph.mds.e.smithi175.stderr:2020-11-16T22:28:08.868+0000 7f9f35d4c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:28:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 Processor -- start 2020-11-16T22:28:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 -- start start 2020-11-16T22:28:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b00615f0 0x7f53b00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b0062790 0x7f53b0080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b0160e40 con 0x7f53b015f5b0 2020-11-16T22:28:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b01609a0 con 0x7f53b0062790 2020-11-16T22:28:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b8798700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0160ba0 con 0x7f53b00615f0 2020-11-16T22:28:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b0062790 0x7f53b0080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b5d33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b00615f0 0x7f53b00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b0062790 0x7f53b0080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58124/0 (socket says 172.21.15.181:58124) 2020-11-16T22:28:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 -- 172.21.15.181:0/1060567647 learned_addr learned my addr 172.21.15.181:0/1060567647 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:08.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 -- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b00615f0 msgr2=0x7f53b00622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:08.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 --2- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b00615f0 0x7f53b00622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:08.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 -- 172.21.15.181:0/1060567647 >> v1:172.21.15.175:6789/0 conn(0x7f53b015f5b0 legacy=0x7f53b00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 -- 172.21.15.181:0/1060567647 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0060970 con 0x7f53b0062790 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b6d35700 1 --2- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b0062790 0x7f53b0080fd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f53a8009d50 tx=0x7f53a8009290).ready entity=mon.1 client_cookie=41af97b55764c34b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b5532700 1 -- 172.21.15.181:0/1060567647 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800bbc0 con 0x7f53b0062790 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b5532700 1 -- 172.21.15.181:0/1060567647 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800bd20 con 0x7f53b0062790 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.937+0000 7f53b5532700 1 -- 172.21.15.181:0/1060567647 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8011610 con 0x7f53b0062790 2020-11-16T22:28:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b0062790 msgr2=0x7f53b0080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 --2- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b0062790 0x7f53b0080fd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f53a8009d50 tx=0x7f53a8009290).stop 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- 172.21.15.181:0/1060567647 shutdown_connections 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 --2- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b0062790 0x7f53b0080fd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 --2- 172.21.15.181:0/1060567647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b00615f0 0x7f53b00622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- 172.21.15.181:0/1060567647 >> 172.21.15.181:0/1060567647 conn(0x7f53b00d69a0 msgr2=0x7f53b00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- 172.21.15.181:0/1060567647 shutdown_connections 2020-11-16T22:28:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- 172.21.15.181:0/1060567647 wait complete. 2020-11-16T22:28:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 Processor -- start 2020-11-16T22:28:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.941+0000 7f53b8798700 1 -- start start 2020-11-16T22:28:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b00615f0 0x7f53b0081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53b015f5b0 0x7f53b0172cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0173850 con 0x7f53b015f5b0 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0173af0 con 0x7f53b00615f0 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0170bf0 con 0x7f53b0062790 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b6534700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b00615f0 0x7f53b0081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37740/0 (socket says 172.21.15.181:37740) 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 -- 172.21.15.181:0/836243280 learned_addr learned my addr 172.21.15.181:0/836243280 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b6d35700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53b015f5b0 0x7f53b0172cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 -- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b00615f0 msgr2=0x7f53b0081170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b00615f0 0x7f53b0081170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 -- 172.21.15.181:0/836243280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53b015f5b0 msgr2=0x7f53b0172cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53b015f5b0 0x7f53b0172cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 -- 172.21.15.181:0/836243280 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0170e90 con 0x7f53b0062790 2020-11-16T22:28:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b5d33700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f53a000ebd0 tx=0x7f53a0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000c8e0 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a000ca40 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b0171130 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000cdc0 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b01731e0 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f53a0012600 con 0x7f53b0062790 2020-11-16T22:28:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.945+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b0069eb0 con 0x7f53b0062790 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.949+0000 7f539e7fc700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 0x7f5388057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.949+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f53a006e020 con 0x7f53b0062790 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.949+0000 7f53b6534700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 0x7f5388057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.949+0000 7f53b6534700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 0x7f5388057520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f53ac000e70 tx=0x7f53ac007ad0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:08.949+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f53a0044280 con 0x7f53b0062790 2020-11-16T22:28:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.149+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53b01642f0 con 0x7f53b0062790 2020-11-16T22:28:09.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.329+0000 7f539e7fc700 1 -- 172.21.15.181:0/836243280 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f53a00478a0 con 0x7f53b0062790 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 msgr2=0x7f5388057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 0x7f5388057520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f53ac000e70 tx=0x7f53ac007ad0).stop 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 msgr2=0x7f53b0081660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f53a000ebd0 tx=0x7f53a0006df0).stop 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 shutdown_connections 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5388055060 0x7f5388057520 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53b015f5b0 0x7f53b0172cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b00615f0 0x7f53b0081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 --2- 172.21.15.181:0/836243280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b0062790 0x7f53b0081660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 >> 172.21.15.181:0/836243280 conn(0x7f53b00d69a0 msgr2=0x7f53b0061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:09.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 shutdown_connections 2020-11-16T22:28:09.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.333+0000 7f53b8798700 1 -- 172.21.15.181:0/836243280 wait complete. 2020-11-16T22:28:09.342 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:09.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 Processor -- start 2020-11-16T22:28:09.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- start start 2020-11-16T22:28:09.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53f4164e10 0x7f53f4062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f40c8db0 con 0x7f53f4161860 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f40c8330 con 0x7f53f4164e10 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40c85d0 con 0x7f53f4161480 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37746/0 (socket says 172.21.15.181:37746) 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 -- 172.21.15.181:0/2196332133 learned_addr learned my addr 172.21.15.181:0/2196332133 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53f99ef700 1 --2- 172.21.15.181:0/2196332133 >> v2:172.21.15.181:3300/0 conn(0x7f53f4164e10 0x7f53f4062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 -- 172.21.15.181:0/2196332133 >> v2:172.21.15.181:3300/0 conn(0x7f53f4164e10 msgr2=0x7f53f4062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 --2- 172.21.15.181:0/2196332133 >> v2:172.21.15.181:3300/0 conn(0x7f53f4164e10 0x7f53f4062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 -- 172.21.15.181:0/2196332133 >> v1:172.21.15.181:6789/0 conn(0x7f53f4161860 legacy=0x7f53f4062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 -- 172.21.15.181:0/2196332133 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40c8870 con 0x7f53f4161480 2020-11-16T22:28:09.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fa1f0700 1 --2- 172.21.15.181:0/2196332133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f53dc006fd0 tx=0x7f53dc006eb0).ready entity=mon.2 client_cookie=92d7719ab16b4853 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:09.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53f91ee700 1 -- 172.21.15.181:0/2196332133 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009810 con 0x7f53f4161480 2020-11-16T22:28:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53f91ee700 1 -- 172.21.15.181:0/2196332133 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc009970 con 0x7f53f4161480 2020-11-16T22:28:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53f91ee700 1 -- 172.21.15.181:0/2196332133 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009cf0 con 0x7f53f4161480 2020-11-16T22:28:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- 172.21.15.181:0/2196332133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 msgr2=0x7f53f4164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/2196332133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f53dc006fd0 tx=0x7f53dc006eb0).stop 2020-11-16T22:28:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- 172.21.15.181:0/2196332133 shutdown_connections 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/2196332133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f4164920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/2196332133 >> v2:172.21.15.181:3300/0 conn(0x7f53f4164e10 0x7f53f4062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- 172.21.15.181:0/2196332133 >> 172.21.15.181:0/2196332133 conn(0x7f53f40d2dc0 msgr2=0x7f53f40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- 172.21.15.181:0/2196332133 shutdown_connections 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.489+0000 7f53fb1f2700 1 -- 172.21.15.181:0/2196332133 wait complete. 2020-11-16T22:28:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 Processor -- start 2020-11-16T22:28:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- start start 2020-11-16T22:28:09.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 0x7f53f40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 0x7f53f41603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40cc500 con 0x7f53f40cadf0 2020-11-16T22:28:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40cbd20 con 0x7f53f4161860 2020-11-16T22:28:09.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40cbfc0 con 0x7f53f4161480 2020-11-16T22:28:09.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53f99ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 0x7f53f40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37752/0 (socket says 172.21.15.181:37752) 2020-11-16T22:28:09.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 -- 172.21.15.181:0/166531814 learned_addr learned my addr 172.21.15.181:0/166531814 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:09.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa9f1700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 0x7f53f41603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 -- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 msgr2=0x7f53f40ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 0x7f53f40ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 -- 172.21.15.181:0/166531814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 msgr2=0x7f53f41603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 0x7f53f41603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 -- 172.21.15.181:0/166531814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40cc260 con 0x7f53f4161480 2020-11-16T22:28:09.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53f99ef700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 0x7f53f40ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:09.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa9f1700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 0x7f53f41603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fa1f0700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f53dc009e10 tx=0x7f53dc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc011dd0 con 0x7f53f4161480 2020-11-16T22:28:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc01e460 con 0x7f53f4161480 2020-11-16T22:28:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f4165600 con 0x7f53f4161480 2020-11-16T22:28:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc01e820 con 0x7f53f4161480 2020-11-16T22:28:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.493+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f40cb930 con 0x7f53f4161480 2020-11-16T22:28:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f4164630 con 0x7f53f4161480 2020-11-16T22:28:09.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f53dc025030 con 0x7f53f4161480 2020-11-16T22:28:09.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53f27fc700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 0x7f53d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f53dc06f650 con 0x7f53f4161480 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53f99ef700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 0x7f53d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.497+0000 7f53f99ef700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 0x7f53d0057520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f53f40d2040 tx=0x7f53e40091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.505+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f53f4164630 con 0x7f53f4161480 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53f4164630 con 0x7f53f4161480 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53f27fc700 1 -- 172.21.15.181:0/166531814 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4626 (secure 0 0 0) 0x7f53f4164630 con 0x7f53f4161480 2020-11-16T22:28:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 msgr2=0x7f53d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 0x7f53d0057520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f53f40d2040 tx=0x7f53e40091d0).stop 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 msgr2=0x7f53f40cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f53dc009e10 tx=0x7f53dc015000).stop 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 shutdown_connections 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f53d0055060 0x7f53d0057520 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40cadf0 0x7f53f41603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53f4161480 0x7f53f40cb440 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 --2- 172.21.15.181:0/166531814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f4161860 0x7f53f40ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 >> 172.21.15.181:0/166531814 conn(0x7f53f40d2dc0 msgr2=0x7f53f40cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 shutdown_connections 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:09.701+0000 7f53fb1f2700 1 -- 172.21.15.181:0/166531814 wait complete. 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 30 2020-11-16T22:28:09.896 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:09.897 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":30,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":5007,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:09.322519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:10.302 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:28:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.445+0000 7f2a4359e700 1 Processor -- start 2020-11-16T22:28:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.445+0000 7f2a4359e700 1 -- start start 2020-11-16T22:28:10.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:10.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a44061b50 con 0x7f2a440816e0 2020-11-16T22:28:10.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a440618b0 con 0x7f2a44145a30 2020-11-16T22:28:10.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440cb930 con 0x7f2a44145e10 2020-11-16T22:28:10.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:10.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37764/0 (socket says 172.21.15.181:37764) 2020-11-16T22:28:10.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 -- 172.21.15.181:0/130868751 learned_addr learned my addr 172.21.15.181:0/130868751 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:10.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2a44145a30 0x7f2a44080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44224/0 (socket says 172.21.15.181:44224) 2020-11-16T22:28:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 -- 172.21.15.181:0/130868751 >> v1:172.21.15.181:6789/0 conn(0x7f2a44145a30 legacy=0x7f2a44080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 -- 172.21.15.181:0/130868751 >> v1:172.21.15.175:6789/0 conn(0x7f2a440816e0 legacy=0x7f2a440cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 -- 172.21.15.181:0/130868751 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440cbbd0 con 0x7f2a44145e10 2020-11-16T22:28:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a42d9d700 1 --2- 172.21.15.181:0/130868751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2a3400ba20 tx=0x7f2a34006eb0).ready entity=mon.2 client_cookie=5f1892a21024f29e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4159a700 1 -- 172.21.15.181:0/130868751 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340098d0 con 0x7f2a44145e10 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4159a700 1 -- 172.21.15.181:0/130868751 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a34009a30 con 0x7f2a44145e10 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4159a700 1 -- 172.21.15.181:0/130868751 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34009db0 con 0x7f2a44145e10 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- 172.21.15.181:0/130868751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 msgr2=0x7f2a441492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 --2- 172.21.15.181:0/130868751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2a3400ba20 tx=0x7f2a34006eb0).stop 2020-11-16T22:28:10.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- 172.21.15.181:0/130868751 shutdown_connections 2020-11-16T22:28:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 --2- 172.21.15.181:0/130868751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145e10 0x7f2a441492c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- 172.21.15.181:0/130868751 >> 172.21.15.181:0/130868751 conn(0x7f2a440d68c0 msgr2=0x7f2a440d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- 172.21.15.181:0/130868751 shutdown_connections 2020-11-16T22:28:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.449+0000 7f2a4359e700 1 -- 172.21.15.181:0/130868751 wait complete. 2020-11-16T22:28:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 Processor -- start 2020-11-16T22:28:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- start start 2020-11-16T22:28:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145a30 0x7f2a4413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a44145e10 0x7f2a4413e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:10.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44163090 con 0x7f2a44145e10 2020-11-16T22:28:10.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44163330 con 0x7f2a440816e0 2020-11-16T22:28:10.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441635d0 con 0x7f2a44145a30 2020-11-16T22:28:10.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:10.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a41d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145a30 0x7f2a4413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:10.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58156/0 (socket says 172.21.15.181:58156) 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 -- 172.21.15.181:0/4266169481 learned_addr learned my addr 172.21.15.181:0/4266169481 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a42d9d700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a44145e10 0x7f2a4413e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 -- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145a30 msgr2=0x7f2a4413dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145a30 0x7f2a4413dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 -- 172.21.15.181:0/4266169481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a44145e10 msgr2=0x7f2a4413e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a44145e10 0x7f2a4413e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 -- 172.21.15.181:0/4266169481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44163870 con 0x7f2a440816e0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4259c700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f2a3c00ca70 tx=0x7f2a3c0014b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c005ac0 con 0x7f2a440816e0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c005c20 con 0x7f2a440816e0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44163b10 con 0x7f2a440816e0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c007110 con 0x7f2a440816e0 2020-11-16T22:28:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.453+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a44164250 con 0x7f2a440816e0 2020-11-16T22:28:10.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44044f60 con 0x7f2a440816e0 2020-11-16T22:28:10.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2a3c010030 con 0x7f2a440816e0 2020-11-16T22:28:10.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a327fc700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 0x7f2a18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:10.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a41d9b700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 0x7f2a18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:10.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a3c06dda0 con 0x7f2a440816e0 2020-11-16T22:28:10.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.457+0000 7f2a41d9b700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 0x7f2a18057520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2a441702b0 tx=0x7f2a38000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:10.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.461+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2a3c047e60 con 0x7f2a440816e0 2020-11-16T22:28:10.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.609+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a44068e30 con 0x7f2a440816e0 2020-11-16T22:28:10.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a327fc700 1 -- 172.21.15.181:0/4266169481 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f2a3c043f30 con 0x7f2a440816e0 2020-11-16T22:28:10.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 msgr2=0x7f2a18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 0x7f2a18057520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2a441702b0 tx=0x7f2a38000b80).stop 2020-11-16T22:28:10.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 msgr2=0x7f2a441627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:10.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f2a3c00ca70 tx=0x7f2a3c0014b0).stop 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 shutdown_connections 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a18055060 0x7f2a18057520 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a44145e10 0x7f2a4413e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a440816e0 0x7f2a441627e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 --2- 172.21.15.181:0/4266169481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a44145a30 0x7f2a4413dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 >> 172.21.15.181:0/4266169481 conn(0x7f2a440d68c0 msgr2=0x7f2a4413eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:10.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 shutdown_connections 2020-11-16T22:28:10.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:10.613+0000 7f2a4359e700 1 -- 172.21.15.181:0/4266169481 wait complete. 2020-11-16T22:28:10.852 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:10.852 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:07.314280+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:28:11.463 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:28:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.621+0000 7fb50f084700 1 Processor -- start 2020-11-16T22:28:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.621+0000 7fb50f084700 1 -- start start 2020-11-16T22:28:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb508081170 0x7fb508081550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 0x7fb5080d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb508081a40 con 0x7fb5081612c0 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080d1bc0 con 0x7fb508081170 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080d1e60 con 0x7fb5080d1180 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50e883700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb5081612c0 0x7fb508161750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44238/0 (socket says 172.21.15.181:44238) 2020-11-16T22:28:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50e883700 1 -- 172.21.15.181:0/3545743859 learned_addr learned my addr 172.21.15.181:0/3545743859 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 --2- 172.21.15.181:0/3545743859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 0x7fb5080d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50e082700 1 --2- 172.21.15.181:0/3545743859 >> v2:172.21.15.175:3300/0 conn(0x7fb508081170 0x7fb508081550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d080700 1 -- 172.21.15.181:0/3545743859 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921381027 0 0) 0x7fb508081a40 con 0x7fb5081612c0 2020-11-16T22:28:11.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d080700 1 -- 172.21.15.181:0/3545743859 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f0003240 con 0x7fb5081612c0 2020-11-16T22:28:11.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 -- 172.21.15.181:0/3545743859 >> v2:172.21.15.175:3300/0 conn(0x7fb508081170 msgr2=0x7fb508081550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:11.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 --2- 172.21.15.181:0/3545743859 >> v2:172.21.15.175:3300/0 conn(0x7fb508081170 0x7fb508081550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 -- 172.21.15.181:0/3545743859 >> v1:172.21.15.181:6789/0 conn(0x7fb5081612c0 legacy=0x7fb508161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 -- 172.21.15.181:0/3545743859 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080d2100 con 0x7fb5080d1180 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d881700 1 --2- 172.21.15.181:0/3545743859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 0x7fb5080d15e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb50400b9a0 tx=0x7fb504006df0).ready entity=mon.2 client_cookie=7fb56a47bfd55c0e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d080700 1 -- 172.21.15.181:0/3545743859 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504009680 con 0x7fb5080d1180 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d080700 1 -- 172.21.15.181:0/3545743859 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5040097e0 con 0x7fb5080d1180 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50d080700 1 -- 172.21.15.181:0/3545743859 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504009b60 con 0x7fb5080d1180 2020-11-16T22:28:11.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- 172.21.15.181:0/3545743859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 msgr2=0x7fb5080d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 --2- 172.21.15.181:0/3545743859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 0x7fb5080d15e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb50400b9a0 tx=0x7fb504006df0).stop 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- 172.21.15.181:0/3545743859 shutdown_connections 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 --2- 172.21.15.181:0/3545743859 >> v2:172.21.15.175:3300/0 conn(0x7fb508081170 0x7fb508081550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 --2- 172.21.15.181:0/3545743859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5080d1180 0x7fb5080d15e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- 172.21.15.181:0/3545743859 >> 172.21.15.181:0/3545743859 conn(0x7fb5080d67e0 msgr2=0x7fb5080d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- 172.21.15.181:0/3545743859 shutdown_connections 2020-11-16T22:28:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.625+0000 7fb50f084700 1 -- 172.21.15.181:0/3545743859 wait complete. 2020-11-16T22:28:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.629+0000 7fb50f084700 1 Processor -- start 2020-11-16T22:28:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.629+0000 7fb50f084700 1 -- start start 2020-11-16T22:28:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb508081170 0x7fb50816fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 0x7fb50815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508173690 con 0x7fb508081170 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508173150 con 0x7fb5080d1180 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081733f0 con 0x7fb5081612c0 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50e883700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 0x7fb50815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58172/0 (socket says 172.21.15.181:58172) 2020-11-16T22:28:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50e883700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 0x7fb50815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37784/0 (socket says 172.21.15.181:37784) 2020-11-16T22:28:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 -- 172.21.15.181:0/906841458 learned_addr learned my addr 172.21.15.181:0/906841458 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50e082700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb508081170 0x7fb50816fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 -- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 msgr2=0x7fb50815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 0x7fb50815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 -- 172.21.15.181:0/906841458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb508081170 msgr2=0x7fb50816fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb508081170 0x7fb50816fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 -- 172.21.15.181:0/906841458 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50816f420 con 0x7fb5080d1180 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50d881700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb50400c010 tx=0x7fb504015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504011c40 con 0x7fb5080d1180 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504011da0 con 0x7fb5080d1180 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50816f6c0 con 0x7fb5080d1180 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50401d660 con 0x7fb5080d1180 2020-11-16T22:28:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb508172ce0 con 0x7fb5080d1180 2020-11-16T22:28:11.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb504024030 con 0x7fb5080d1180 2020-11-16T22:28:11.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb508069eb0 con 0x7fb5080d1180 2020-11-16T22:28:11.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb50e082700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.633+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb50406f260 con 0x7fb5080d1180 2020-11-16T22:28:11.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.637+0000 7fb50e082700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 0x7fb4e0057520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fb4fc006fd0 tx=0x7fb4fc006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.641+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb5040454a0 con 0x7fb5080d1180 2020-11-16T22:28:11.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.849+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fb508062eb0 con 0x7fb5080d1180 2020-11-16T22:28:11.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb4fa7fc700 1 -- 172.21.15.181:0/906841458 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb504048ac0 con 0x7fb5080d1180 2020-11-16T22:28:11.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 msgr2=0x7fb4e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 0x7fb4e0057520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fb4fc006fd0 tx=0x7fb4fc006df0).stop 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 msgr2=0x7fb5081727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb50400c010 tx=0x7fb504015000).stop 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 shutdown_connections 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5081612c0 0x7fb50815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb508081170 0x7fb50816fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 --2- 172.21.15.181:0/906841458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5080d1180 0x7fb5081727f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 >> 172.21.15.181:0/906841458 conn(0x7fb5080d67e0 msgr2=0x7fb5080821d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:11.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 shutdown_connections 2020-11-16T22:28:11.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:11.861+0000 7fb50f084700 1 -- 172.21.15.181:0/906841458 wait complete. 2020-11-16T22:28:11.848 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:28:11.863 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 Processor -- start 2020-11-16T22:28:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 -- start start 2020-11-16T22:28:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ab815f5b0 0x7f9ab80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab8160e40 con 0x7f9ab80d1830 2020-11-16T22:28:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab8160ba0 con 0x7f9ab80615f0 2020-11-16T22:28:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9abccc1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab81609a0 con 0x7f9ab815f5b0 2020-11-16T22:28:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9ab80615f0 0x7f9ab80622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44252/0 (socket says 172.21.15.181:44252) 2020-11-16T22:28:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 -- 172.21.15.181:0/676885505 learned_addr learned my addr 172.21.15.181:0/676885505 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab659c700 1 --2- 172.21.15.181:0/676885505 >> v2:172.21.15.175:3300/0 conn(0x7f9ab815f5b0 0x7f9ab80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007604753 0 0) 0x7f9ab8160ba0 con 0x7f9ab80615f0 2020-11-16T22:28:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a9c003240 con 0x7f9ab80615f0 2020-11-16T22:28:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1677543804 0 0) 0x7f9a9c003240 con 0x7f9ab80615f0 2020-11-16T22:28:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 >> v2:172.21.15.175:3300/0 conn(0x7f9ab815f5b0 msgr2=0x7f9ab80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 --2- 172.21.15.181:0/676885505 >> v2:172.21.15.175:3300/0 conn(0x7f9ab815f5b0 0x7f9ab80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 >> v1:172.21.15.175:6789/0 conn(0x7f9ab80d1830 legacy=0x7f9ab8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8060970 con 0x7f9ab80615f0 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f9ab0001860 con 0x7f9ab80615f0 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab659c700 1 --2- 172.21.15.181:0/676885505 >> v2:172.21.15.175:3300/0 conn(0x7f9ab815f5b0 0x7f9ab80d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 >> v1:172.21.15.181:6789/0 conn(0x7f9ab80615f0 legacy=0x7f9ab80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c003730 0x7f9a9c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9a9c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c004e50 0x7f9a9c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81609a0 con 0x7f9a9c003730 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c004350 con 0x7f9a9c004e50 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c00bc50 con 0x7f9a9c0044e0 2020-11-16T22:28:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9a9c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab659c700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c003730 0x7f9a9c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab6d9d700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c004e50 0x7f9a9c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 -- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c004e50 msgr2=0x7f9a9c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c004e50 0x7f9a9c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 -- 172.21.15.181:0/676885505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c003730 msgr2=0x7f9a9c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c003730 0x7f9a9c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.001+0000 7f9ab5d9b700 1 -- 172.21.15.181:0/676885505 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c003240 con 0x7f9a9c0044e0 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9ab6d9d700 1 -- 172.21.15.181:0/676885505 reap_dead start 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9a9c004960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9ab000c810 tx=0x7f9ab0007b50).ready entity=mon.2 client_cookie=6b72633b97d9d828 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0005b80 con 0x7f9a9c0044e0 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0005ce0 con 0x7f9a9c0044e0 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9ab559a700 1 -- 172.21.15.181:0/676885505 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0007300 con 0x7f9a9c0044e0 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 -- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 msgr2=0x7f9a9c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9a9c004960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9ab000c810 tx=0x7f9ab0007b50).stop 2020-11-16T22:28:11.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 -- 172.21.15.181:0/676885505 shutdown_connections 2020-11-16T22:28:11.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 --2- 172.21.15.181:0/676885505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9a9c004960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 -- 172.21.15.181:0/676885505 >> 172.21.15.181:0/676885505 conn(0x7f9ab80d69a0 msgr2=0x7f9ab80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:11.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 -- 172.21.15.181:0/676885505 shutdown_connections 2020-11-16T22:28:11.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.005+0000 7f9abccc1700 1 -- 172.21.15.181:0/676885505 wait complete. 2020-11-16T22:28:11.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 Processor -- start 2020-11-16T22:28:11.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 -- start start 2020-11-16T22:28:11.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9ab80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 0x7f9ab8172d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 0x7f9ab816f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8062780 con 0x7f9ab80d05c0 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9ab80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8173690 con 0x7f9ab8173210 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9abccc1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80d3090 con 0x7f9a9c0044e0 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9ab80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37800/0 (socket says 172.21.15.181:37800) 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab659c700 1 -- 172.21.15.181:0/2875625468 learned_addr learned my addr 172.21.15.181:0/2875625468 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:11.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 0x7f9ab816f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 0x7f9ab8172d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 -- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 msgr2=0x7f9ab80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9ab80d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 -- 172.21.15.181:0/2875625468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 msgr2=0x7f9ab8172d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 0x7f9ab8172d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.009+0000 7f9ab6d9d700 1 -- 172.21.15.181:0/2875625468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80d3330 con 0x7f9ab8173210 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9ab5d9b700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 0x7f9ab8172d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9ab6d9d700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 0x7f9ab816f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9ab805d470 tx=0x7f9aa0007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007240 con 0x7f9ab8173210 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa00073c0 con 0x7f9ab8173210 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0006740 con 0x7f9ab8173210 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80d35d0 con 0x7f9ab8173210 2020-11-16T22:28:11.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80d3b10 con 0x7f9ab8173210 2020-11-16T22:28:12.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8069eb0 con 0x7f9ab8173210 2020-11-16T22:28:12.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9aa0006af0 con 0x7f9ab8173210 2020-11-16T22:28:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 0x7f9a88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9aa006d0a0 con 0x7f9ab8173210 2020-11-16T22:28:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9ab659c700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 0x7f9a88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.013+0000 7f9ab659c700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 0x7f9a88057520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9aa8006ca0 tx=0x7f9aa8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:12.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.021+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9aa0043300 con 0x7f9ab8173210 2020-11-16T22:28:12.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab80600f0 con 0x7f9ab8173210 2020-11-16T22:28:12.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9aae7fc700 1 -- 172.21.15.181:0/2875625468 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4626 (secure 0 0 0) 0x7f9aa0018030 con 0x7f9ab8173210 2020-11-16T22:28:12.209 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:12.210 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":30,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":5007,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:09.322519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:12.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 msgr2=0x7f9a88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:12.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 0x7f9a88057520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f9aa8006ca0 tx=0x7f9aa8008040).stop 2020-11-16T22:28:12.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 msgr2=0x7f9ab816f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:12.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 0x7f9ab816f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9ab805d470 tx=0x7f9aa0007b00).stop 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 shutdown_connections 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9a88055060 0x7f9a88057520 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab8173210 0x7f9ab816f7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a9c0044e0 0x7f9ab80d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 --2- 172.21.15.181:0/2875625468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab80d05c0 0x7f9ab8172d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 >> 172.21.15.181:0/2875625468 conn(0x7f9ab80d69a0 msgr2=0x7f9ab8061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:12.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 shutdown_connections 2020-11-16T22:28:12.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:12.225+0000 7f9abccc1700 1 -- 172.21.15.181:0/2875625468 wait complete. 2020-11-16T22:28:12.213 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 30 2020-11-16T22:28:13.131 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:28:06.310129+0000', 'modified': '2020-11-16T22:28:09.322519+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5010}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5010': {'gid': 5010, 'name': 'e', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6841/2161152860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6839', 'nonce': 2161152860}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2161152860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:28:13.131 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:28:13.132 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:28:13.268 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5010,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":6.9636001050000003,"mdsmap_epoch":30,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":20.251713515999999} 2020-11-16T22:28:13.269 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5010, "mdsmap_epoch": 30, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 6.963600105, "state": "up:active", "uptime": 20.251713516, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:28:13.270 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:28:13.270 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:28:13.270 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:28:13.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:28:13.271 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:28:13.271 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:28:13.271 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:28:13.272 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:28:13.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.273 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:28:13.292 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:28:13.292 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:28:13.293 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:28:13.295 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:28:13.296 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.297 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:28:13.297 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:28:13.297 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:28:13.297 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:28:13.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.298 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:28:13.332 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:28:13.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.335 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:28:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:28:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:28:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:28:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:28:13.353 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:28:13.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.354 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:28:13.355 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:28:13.355 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:28:13.355 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:28:13.356 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:28:13.356 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:28:13.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.505 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:28:13.517 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:28:13.517 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:28:13.518 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:28:13.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:13.519 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:28:13.547 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:28:13.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.697+0000 7f690dbe1700 1 Processor -- start 2020-11-16T22:28:13.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.697+0000 7f690dbe1700 1 -- start start 2020-11-16T22:28:13.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690dbe1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f690815d080 0x7f69080cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690dbe1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6908065150 con 0x7f69080600f0 2020-11-16T22:28:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690dbe1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69080653f0 con 0x7f690815d460 2020-11-16T22:28:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690dbe1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6908066a20 con 0x7f690815d080 2020-11-16T22:28:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f6907fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f690815d460 0x7f6908066310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44272/0 (socket says 172.21.15.181:44272) 2020-11-16T22:28:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f6907fff700 1 -- 172.21.15.181:0/1410443799 learned_addr learned my addr 172.21.15.181:0/1410443799 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:13.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690cbdf700 1 --2- 172.21.15.181:0/1410443799 >> v2:172.21.15.175:3300/0 conn(0x7f690815d080 0x7f69080cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:13.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349786304 0 0) 0x7f69080653f0 con 0x7f690815d460 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68ec003240 con 0x7f690815d460 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690cbdf700 1 -- 172.21.15.181:0/1410443799 >> v1:172.21.15.181:6789/0 conn(0x7f690815d460 legacy=0x7f6908066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2092386445 0 0) 0x7f68ec003240 con 0x7f690815d460 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690cbdf700 1 -- 172.21.15.181:0/1410443799 >> v1:172.21.15.175:6789/0 conn(0x7f69080600f0 legacy=0x7f6908060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690cbdf700 1 -- 172.21.15.181:0/1410443799 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908065690 con 0x7f690815d080 2020-11-16T22:28:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f690cbdf700 1 --2- 172.21.15.181:0/1410443799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d080 0x7f69080cdd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f68fc00b950 tx=0x7f68fc006df0).ready entity=mon.0 client_cookie=abc5c6fedc02c019 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40014e0 con 0x7f690815d080 2020-11-16T22:28:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc012460 con 0x7f690815d080 2020-11-16T22:28:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.701+0000 7f69077fe700 1 -- 172.21.15.181:0/1410443799 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0127e0 con 0x7f690815d080 2020-11-16T22:28:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 -- 172.21.15.181:0/1410443799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d080 msgr2=0x7f69080cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 --2- 172.21.15.181:0/1410443799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d080 0x7f69080cdd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f68fc00b950 tx=0x7f68fc006df0).stop 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 -- 172.21.15.181:0/1410443799 shutdown_connections 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 --2- 172.21.15.181:0/1410443799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d080 0x7f69080cdd90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 -- 172.21.15.181:0/1410443799 >> 172.21.15.181:0/1410443799 conn(0x7f69080cfe90 msgr2=0x7f69080d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 -- 172.21.15.181:0/1410443799 shutdown_connections 2020-11-16T22:28:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.705+0000 7f690dbe1700 1 -- 172.21.15.181:0/1410443799 wait complete. 2020-11-16T22:28:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 Processor -- start 2020-11-16T22:28:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 -- start start 2020-11-16T22:28:13.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:13.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690815d080 0x7f6908147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:13.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 0x7f69080d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:13.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690816e0e0 con 0x7f690815d460 2020-11-16T22:28:13.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69080b4820 con 0x7f69080600f0 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690dbe1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69080b4ac0 con 0x7f690815d080 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690cbdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690cbdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58204/0 (socket says 172.21.15.181:58204) 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690cbdf700 1 -- 172.21.15.181:0/3061831424 learned_addr learned my addr 172.21.15.181:0/3061831424 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f6907fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690815d080 0x7f6908147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.709+0000 7f690d3e0700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 0x7f69080d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 -- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690815d080 msgr2=0x7f6908147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690815d080 0x7f6908147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 -- 172.21.15.181:0/3061831424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 msgr2=0x7f69080d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 0x7f69080d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 -- 172.21.15.181:0/3061831424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69080b4d60 con 0x7f69080600f0 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690d3e0700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 0x7f69080d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690cbdf700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f68fc00b150 tx=0x7f69080b35f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0067c0 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc017070 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc011c30 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690815a190 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690815a6d0 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6908044f60 con 0x7f69080600f0 2020-11-16T22:28:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f68fc00f030 con 0x7f69080600f0 2020-11-16T22:28:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 0x7f68e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6907fff700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 0x7f68e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f68fc06e850 con 0x7f69080600f0 2020-11-16T22:28:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.713+0000 7f6907fff700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 0x7f68e0057520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f69080c8f80 tx=0x7f68f4000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:13.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.721+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f68fc044a30 con 0x7f69080600f0 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.877+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69080cfb90 con 0x7f69080600f0 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f6904ff9700 1 -- 172.21.15.181:0/3061831424 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f68fc023020 con 0x7f69080600f0 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:07.314280+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 msgr2=0x7f68e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 0x7f68e0057520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f69080c8f80 tx=0x7f68f4000b80).stop 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 msgr2=0x7f690816dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f68fc00b150 tx=0x7f69080b35f0).stop 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 shutdown_connections 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f68e0055060 0x7f68e0057520 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d460 0x7f69080d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080600f0 0x7f690816dbf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 --2- 172.21.15.181:0/3061831424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690815d080 0x7f6908147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 >> 172.21.15.181:0/3061831424 conn(0x7f69080cfe90 msgr2=0x7f69080d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 shutdown_connections 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:13.881+0000 7f690dbe1700 1 -- 172.21.15.181:0/3061831424 wait complete. 2020-11-16T22:28:13.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:28:03.979538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:28:05.295145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 Processor -- start 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 -- start start 2020-11-16T22:28:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25b8145a30 0x7f25b8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b8061b50 con 0x7f25b80816e0 2020-11-16T22:28:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b80618b0 con 0x7f25b81492c0 2020-11-16T22:28:14.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b80cb930 con 0x7f25b8145a30 2020-11-16T22:28:14.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b5d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f25b81492c0 0x7f25b8080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44288/0 (socket says 172.21.15.181:44288) 2020-11-16T22:28:14.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b5d9b700 1 -- 172.21.15.181:0/4017418784 learned_addr learned my addr 172.21.15.181:0/4017418784 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:14.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b659c700 1 --2- 172.21.15.181:0/4017418784 >> v2:172.21.15.175:3300/0 conn(0x7f25b8145a30 0x7f25b8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b559a700 1 -- 172.21.15.181:0/4017418784 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060987341 0 0) 0x7f25b80618b0 con 0x7f25b81492c0 2020-11-16T22:28:14.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.025+0000 7f25b559a700 1 -- 172.21.15.181:0/4017418784 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c003240 con 0x7f25b81492c0 2020-11-16T22:28:14.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b659c700 1 -- 172.21.15.181:0/4017418784 >> v1:172.21.15.181:6789/0 conn(0x7f25b81492c0 legacy=0x7f25b8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b659c700 1 -- 172.21.15.181:0/4017418784 >> v1:172.21.15.175:6789/0 conn(0x7f25b80816e0 legacy=0x7f25b80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b659c700 1 -- 172.21.15.181:0/4017418784 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80cbbd0 con 0x7f25b8145a30 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b659c700 1 --2- 172.21.15.181:0/4017418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b8145a30 0x7f25b8145e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f25a800b9a0 tx=0x7f25a8006df0).ready entity=mon.0 client_cookie=6d06a65c6b0f2451 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b559a700 1 -- 172.21.15.181:0/4017418784 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80097b0 con 0x7f25b8145a30 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b559a700 1 -- 172.21.15.181:0/4017418784 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8009910 con 0x7f25b8145a30 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b559a700 1 -- 172.21.15.181:0/4017418784 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8009c90 con 0x7f25b8145a30 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- 172.21.15.181:0/4017418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b8145a30 msgr2=0x7f25b8145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 --2- 172.21.15.181:0/4017418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b8145a30 0x7f25b8145e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f25a800b9a0 tx=0x7f25a8006df0).stop 2020-11-16T22:28:14.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- 172.21.15.181:0/4017418784 shutdown_connections 2020-11-16T22:28:14.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 --2- 172.21.15.181:0/4017418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b8145a30 0x7f25b8145e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- 172.21.15.181:0/4017418784 >> 172.21.15.181:0/4017418784 conn(0x7f25b80d68c0 msgr2=0x7f25b80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- 172.21.15.181:0/4017418784 shutdown_connections 2020-11-16T22:28:14.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- 172.21.15.181:0/4017418784 wait complete. 2020-11-16T22:28:14.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 Processor -- start 2020-11-16T22:28:14.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.029+0000 7f25b759e700 1 -- start start 2020-11-16T22:28:14.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b80816e0 0x7f25b8140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 0x7f25b8140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81705b0 con 0x7f25b80816e0 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8170070 con 0x7f25b8145a30 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8170310 con 0x7f25b81492c0 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 0x7f25b8140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37832/0 (socket says 172.21.15.181:37832) 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b6d9d700 1 -- 172.21.15.181:0/2883251858 learned_addr learned my addr 172.21.15.181:0/2883251858 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b659c700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b80816e0 0x7f25b8140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 -- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 msgr2=0x7f25b80ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 -- 172.21.15.181:0/2883251858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b80816e0 msgr2=0x7f25b8140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b80816e0 0x7f25b8140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 -- 172.21.15.181:0/2883251858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80cdd50 con 0x7f25b8145a30 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b6d9d700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b5d9b700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 0x7f25b8140fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f25ac00c710 tx=0x7f25ac007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0072e0 con 0x7f25b8145a30 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac007440 con 0x7f25b8145a30 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b80cdff0 con 0x7f25b8145a30 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac005820 con 0x7f25b8145a30 2020-11-16T22:28:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b80cec20 con 0x7f25b8145a30 2020-11-16T22:28:14.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.033+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8044f60 con 0x7f25b8145a30 2020-11-16T22:28:14.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.037+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f25ac010070 con 0x7f25b8145a30 2020-11-16T22:28:14.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.037+0000 7f25a67fc700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 0x7f258c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.037+0000 7f25b659c700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 0x7f258c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.037+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25ac06d380 con 0x7f25b8145a30 2020-11-16T22:28:14.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.037+0000 7f25b659c700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 0x7f258c057520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f25a8006e30 tx=0x7f25a8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.045+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f25ac043660 con 0x7f25b8145a30 2020-11-16T22:28:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.197+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f25b80d0c00 con 0x7f25b8145a30 2020-11-16T22:28:14.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.197+0000 7f25a67fc700 1 -- 172.21.15.181:0/2883251858 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f25ac046c80 con 0x7f25b8145a30 2020-11-16T22:28:14.183 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:28:14.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 msgr2=0x7f258c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 0x7f258c057520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f25a8006e30 tx=0x7f25a8015000).stop 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 msgr2=0x7f25b8140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 0x7f25b8140fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f25ac00c710 tx=0x7f25ac007a20).stop 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 shutdown_connections 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f258c055060 0x7f258c057520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b81492c0 0x7f25b80ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25b80816e0 0x7f25b8140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 --2- 172.21.15.181:0/2883251858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b8145a30 0x7f25b8140fb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 >> 172.21.15.181:0/2883251858 conn(0x7f25b80d68c0 msgr2=0x7f25b8061080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 shutdown_connections 2020-11-16T22:28:14.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.201+0000 7f25b759e700 1 -- 172.21.15.181:0/2883251858 wait complete. 2020-11-16T22:28:14.198 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:28:14.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.333+0000 7fa8f5f95700 1 Processor -- start 2020-11-16T22:28:14.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.333+0000 7fa8f5f95700 1 -- start start 2020-11-16T22:28:14.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 0x7fa8f0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0161350 con 0x7fa8f0159b40 2020-11-16T22:28:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f01615f0 con 0x7fa8f0162510 2020-11-16T22:28:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0161890 con 0x7fa8f00b2900 2020-11-16T22:28:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5794700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8f0162510 0x7fa8f01629a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44302/0 (socket says 172.21.15.181:44302) 2020-11-16T22:28:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 0x7fa8f0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5794700 1 -- 172.21.15.181:0/1588970372 learned_addr learned my addr 172.21.15.181:0/1588970372 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104486697 0 0) 0x7fa8f01615f0 con 0x7fa8f0162510 2020-11-16T22:28:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8dc003240 con 0x7fa8f0162510 2020-11-16T22:28:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8effff700 1 -- 172.21.15.181:0/1588970372 >> v1:172.21.15.181:6789/0 conn(0x7fa8f0162510 legacy=0x7fa8f01629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454442744 0 0) 0x7fa8f0161350 con 0x7fa8f0159b40 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8effff700 1 -- 172.21.15.181:0/1588970372 >> v1:172.21.15.175:6789/0 conn(0x7fa8f0159b40 legacy=0x7fa8f00b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8effff700 1 -- 172.21.15.181:0/1588970372 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0161b30 con 0x7fa8f00b2900 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8effff700 1 --2- 172.21.15.181:0/1588970372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 0x7fa8f0159f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa8e400b9d0 tx=0x7fa8e4006df0).ready entity=mon.2 client_cookie=40de382776f97968 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40097b0 con 0x7fa8f00b2900 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e4009910 con 0x7fa8f00b2900 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8ef7fe700 1 -- 172.21.15.181:0/1588970372 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4009c90 con 0x7fa8f00b2900 2020-11-16T22:28:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1588970372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 msgr2=0x7fa8f0159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1588970372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 0x7fa8f0159f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa8e400b9d0 tx=0x7fa8e4006df0).stop 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1588970372 shutdown_connections 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1588970372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f00b2900 0x7fa8f0159f20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1588970372 >> 172.21.15.181:0/1588970372 conn(0x7fa8f00d4570 msgr2=0x7fa8f00d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1588970372 shutdown_connections 2020-11-16T22:28:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.337+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1588970372 wait complete. 2020-11-16T22:28:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 Processor -- start 2020-11-16T22:28:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- start start 2020-11-16T22:28:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f0159b40 0x7fa8f0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0151970 0x7fa8f014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0152880 con 0x7fa8f0159b40 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0151480 con 0x7fa8f0151970 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0152340 con 0x7fa8f0162510 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5794700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0151970 0x7fa8f014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37844/0 (socket says 172.21.15.181:37844) 2020-11-16T22:28:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 -- 172.21.15.181:0/1297476439 learned_addr learned my addr 172.21.15.181:0/1297476439 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f4f93700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f0159b40 0x7fa8f0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 -- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0151970 msgr2=0x7fa8f014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0151970 0x7fa8f014fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 -- 172.21.15.181:0/1297476439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f0159b40 msgr2=0x7fa8f0150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f0159b40 0x7fa8f0150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 -- 172.21.15.181:0/1297476439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f01525e0 con 0x7fa8f0162510 2020-11-16T22:28:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8effff700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fa8e4006fd0 tx=0x7fa8e4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4011c80 con 0x7fa8f0162510 2020-11-16T22:28:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e4011de0 con 0x7fa8f0162510 2020-11-16T22:28:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f01505b0 con 0x7fa8f0162510 2020-11-16T22:28:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e401d5d0 con 0x7fa8f0162510 2020-11-16T22:28:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.341+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f0150af0 con 0x7fa8f0162510 2020-11-16T22:28:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f0150cb0 con 0x7fa8f0162510 2020-11-16T22:28:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa8e4026030 con 0x7fa8f0162510 2020-11-16T22:28:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8ecff9700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 0x7fa8c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8f4f93700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 0x7fa8c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa8e406f6e0 con 0x7fa8f0162510 2020-11-16T22:28:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.345+0000 7fa8f4f93700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 0x7fa8c8057520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa8e0003330 tx=0x7fa8e0007910).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.349+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa8e4045830 con 0x7fa8f0162510 2020-11-16T22:28:14.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8f0161d10 con 0x7fa8f0162510 2020-11-16T22:28:14.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8ecff9700 1 -- 172.21.15.181:0/1297476439 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8e4048e50 con 0x7fa8f0162510 2020-11-16T22:28:14.485 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 msgr2=0x7fa8c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 0x7fa8c8057520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa8e0003330 tx=0x7fa8e0007910).stop 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 msgr2=0x7fa8f01708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fa8e4006fd0 tx=0x7fa8e4015000).stop 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 shutdown_connections 2020-11-16T22:28:14.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa8c8055060 0x7fa8c8057520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f0151970 0x7fa8f014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8f0159b40 0x7fa8f0150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 --2- 172.21.15.181:0/1297476439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8f0162510 0x7fa8f01708a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 >> 172.21.15.181:0/1297476439 conn(0x7fa8f00d4570 msgr2=0x7fa8f00d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 shutdown_connections 2020-11-16T22:28:14.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.501+0000 7fa8f5f95700 1 -- 172.21.15.181:0/1297476439 wait complete. 2020-11-16T22:28:14.500 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 Processor -- start 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 -- start start 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 0x7fc7d415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d4066480 con 0x7fc7d40667b0 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d4066ee0 con 0x7fc7d415d200 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7dbd5a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4067180 con 0x7fc7d40ce210 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7da2f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7d9af6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 0x7fc7d415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7da2f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37854/0 (socket says 172.21.15.181:37854) 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.649+0000 7fc7da2f7700 1 -- 172.21.15.181:0/2370307642 learned_addr learned my addr 172.21.15.181:0/2370307642 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7da2f7700 1 -- 172.21.15.181:0/2370307642 >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 msgr2=0x7fc7d415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7da2f7700 1 --2- 172.21.15.181:0/2370307642 >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 0x7fc7d415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7da2f7700 1 -- 172.21.15.181:0/2370307642 >> v1:172.21.15.181:6789/0 conn(0x7fc7d40667b0 legacy=0x7fc7d40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7d8af4700 1 -- 172.21.15.181:0/2370307642 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204171112 0 0) 0x7fc7d4066480 con 0x7fc7d40667b0 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7da2f7700 1 -- 172.21.15.181:0/2370307642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4067420 con 0x7fc7d40ce210 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7d9af6700 1 --2- 172.21.15.181:0/2370307642 >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 0x7fc7d415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7da2f7700 1 --2- 172.21.15.181:0/2370307642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fc7c4006fd0 tx=0x7fc7c4006eb0).ready entity=mon.2 client_cookie=763ab72038b0526a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7d8af4700 1 -- 172.21.15.181:0/2370307642 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4009810 con 0x7fc7d40ce210 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7d8af4700 1 -- 172.21.15.181:0/2370307642 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4009970 con 0x7fc7d40ce210 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7d8af4700 1 -- 172.21.15.181:0/2370307642 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4009cf0 con 0x7fc7d40ce210 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2370307642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 msgr2=0x7fc7d40cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2370307642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fc7c4006fd0 tx=0x7fc7c4006eb0).stop 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2370307642 shutdown_connections 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2370307642 >> v2:172.21.15.181:3300/0 conn(0x7fc7d415d200 0x7fc7d415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2370307642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d40ce210 0x7fc7d40cf920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2370307642 >> 172.21.15.181:0/2370307642 conn(0x7fc7d40d4790 msgr2=0x7fc7d40d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2370307642 shutdown_connections 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2370307642 wait complete. 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 Processor -- start 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.653+0000 7fc7dbd5a700 1 -- start start 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7d40ce210 0x7fc7d40d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 0x7fc7d40d4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40b8950 con 0x7fc7d40ce210 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40b8bf0 con 0x7fc7d40667b0 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4170640 con 0x7fc7d415d200 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58246/0 (socket says 172.21.15.181:58246) 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7da2f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 0x7fc7d40d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 -- 172.21.15.181:0/2198054386 learned_addr learned my addr 172.21.15.181:0/2198054386 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d92f5700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7d40ce210 0x7fc7d40d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 -- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 msgr2=0x7fc7d40d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 0x7fc7d40d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 -- 172.21.15.181:0/2198054386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7d40ce210 msgr2=0x7fc7d40d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7d40ce210 0x7fc7d40d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 -- 172.21.15.181:0/2198054386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d41708e0 con 0x7fc7d40667b0 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7da2f7700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 0x7fc7d40d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d9af6700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc7cc00c150 tx=0x7fc7cc0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00ccc0 con 0x7fc7d40667b0 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc00ce20 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d4170b80 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc012600 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.657+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d40ba6f0 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d4069eb0 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc7cc022070 con 0x7fc7d40667b0 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7d1ffb700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 0x7fc7ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7cc06e6c0 con 0x7fc7d40667b0 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7d92f5700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 0x7fc7ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.661+0000 7fc7d92f5700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 0x7fc7ac057520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc7d4065ad0 tx=0x7fc7c8007970).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.665+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc7cc044920 con 0x7fc7d40667b0 2020-11-16T22:28:14.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.821+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7d4161ba0 con 0x7fc7d40667b0 2020-11-16T22:28:14.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.821+0000 7fc7d1ffb700 1 -- 172.21.15.181:0/2198054386 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc7cc047f40 con 0x7fc7d40667b0 2020-11-16T22:28:14.808 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:28:14.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 msgr2=0x7fc7ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 0x7fc7ac057520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc7d4065ad0 tx=0x7fc7c8007970).stop 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 msgr2=0x7fc7d40ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc7cc00c150 tx=0x7fc7cc0091d0).stop 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 shutdown_connections 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7ac055060 0x7fc7ac057520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7d415d200 0x7fc7d40d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7d40667b0 0x7fc7d40ba200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 --2- 172.21.15.181:0/2198054386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7d40ce210 0x7fc7d40d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 >> 172.21.15.181:0/2198054386 conn(0x7fc7d40d4790 msgr2=0x7fc7d40cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 shutdown_connections 2020-11-16T22:28:14.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.825+0000 7fc7dbd5a700 1 -- 172.21.15.181:0/2198054386 wait complete. 2020-11-16T22:28:14.822 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:28:14.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.961+0000 7f52bf59e700 1 Processor -- start 2020-11-16T22:28:14.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- start start 2020-11-16T22:28:14.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 0x7f52c0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f52c00b21f0 0x7f52c00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52c0162510 0x7f52c01629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0161350 con 0x7f52c00b21f0 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c01615f0 con 0x7f52c0162510 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0161890 con 0x7f52c0159b40 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52be59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 0x7f52c0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52c0162510 0x7f52c01629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52c0162510 0x7f52c01629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58258/0 (socket says 172.21.15.181:58258) 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 -- 172.21.15.181:0/3063995810 learned_addr learned my addr 172.21.15.181:0/3063995810 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52be59c700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 0x7f52c0159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37866/0 (socket says 172.21.15.181:37866) 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bdd9b700 1 --2- 172.21.15.181:0/3063995810 >> v2:172.21.15.175:3300/0 conn(0x7f52c00b21f0 0x7f52c00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 -- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 msgr2=0x7f52c0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 0x7f52c0159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 -- 172.21.15.181:0/3063995810 >> v2:172.21.15.175:3300/0 conn(0x7f52c00b21f0 msgr2=0x7f52c00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:14.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 --2- 172.21.15.181:0/3063995810 >> v2:172.21.15.175:3300/0 conn(0x7f52c00b21f0 0x7f52c00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 -- 172.21.15.181:0/3063995810 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0161b30 con 0x7f52c0162510 2020-11-16T22:28:14.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bed9d700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0162510 0x7f52c01629c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f52c00cec80 tx=0x7f52b8009290).ready entity=mon.1 client_cookie=7bd15087e6bb78c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bd59a700 1 -- 172.21.15.181:0/3063995810 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800bb10 con 0x7f52c0162510 2020-11-16T22:28:14.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bd59a700 1 -- 172.21.15.181:0/3063995810 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b800bc70 con 0x7f52c0162510 2020-11-16T22:28:14.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bd59a700 1 -- 172.21.15.181:0/3063995810 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8011400 con 0x7f52c0162510 2020-11-16T22:28:14.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0162510 msgr2=0x7f52c01629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0162510 0x7f52c01629c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f52c00cec80 tx=0x7f52b8009290).stop 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- 172.21.15.181:0/3063995810 shutdown_connections 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0162510 0x7f52c01629c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- 172.21.15.181:0/3063995810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c0159b40 0x7f52c0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 --2- 172.21.15.181:0/3063995810 >> v2:172.21.15.175:3300/0 conn(0x7f52c00b21f0 0x7f52c00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- 172.21.15.181:0/3063995810 >> 172.21.15.181:0/3063995810 conn(0x7f52c00d4570 msgr2=0x7f52c00d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- 172.21.15.181:0/3063995810 shutdown_connections 2020-11-16T22:28:14.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.965+0000 7f52bf59e700 1 -- 172.21.15.181:0/3063995810 wait complete. 2020-11-16T22:28:14.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 Processor -- start 2020-11-16T22:28:14.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 -- start start 2020-11-16T22:28:14.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0159b40 0x7f52c00caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52c0162510 0x7f52c01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c98d0 con 0x7f52c0162510 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c9b00 con 0x7f52c0159b40 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c9da0 con 0x7f52c00b21f0 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37870/0 (socket says 172.21.15.181:37870) 2020-11-16T22:28:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 -- 172.21.15.181:0/4096589786 learned_addr learned my addr 172.21.15.181:0/4096589786 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bdd9b700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0159b40 0x7f52c00caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bed9d700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52c0162510 0x7f52c01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 -- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0159b40 msgr2=0x7f52c00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0159b40 0x7f52c00caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 -- 172.21.15.181:0/4096589786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52c0162510 msgr2=0x7f52c01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52c0162510 0x7f52c01529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 -- 172.21.15.181:0/4096589786 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00ca040 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52be59c700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f52b000ebd0 tx=0x7f52b0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000b3a0 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b000c9b0 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c00ca2e0 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.969+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000cd70 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00caf10 con 0x7f52c00b21f0 2020-11-16T22:28:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0044f60 con 0x7f52c00b21f0 2020-11-16T22:28:14.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f52b0012600 con 0x7f52c00b21f0 2020-11-16T22:28:14.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52ae7fc700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 0x7f5294057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:14.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52b006e120 con 0x7f52c00b21f0 2020-11-16T22:28:14.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52bdd9b700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 0x7f5294057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:14.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.973+0000 7f52bdd9b700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 0x7f5294057520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f52b4006fd0 tx=0x7f52b4006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:14.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:14.981+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f52b0044320 con 0x7f52c00b21f0 2020-11-16T22:28:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52c0159400 con 0x7f52c00b21f0 2020-11-16T22:28:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52ae7fc700 1 -- 172.21.15.181:0/4096589786 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7f52b0047940 con 0x7f52c00b21f0 2020-11-16T22:28:15.113 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:28:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 msgr2=0x7f5294057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 0x7f5294057520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f52b4006fd0 tx=0x7f52b4006df0).stop 2020-11-16T22:28:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 msgr2=0x7f52c0170480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f52b000ebd0 tx=0x7f52b0006df0).stop 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 shutdown_connections 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5294055060 0x7f5294057520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52c0162510 0x7f52c01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c00b21f0 0x7f52c0170480 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 --2- 172.21.15.181:0/4096589786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c0159b40 0x7f52c00caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 >> 172.21.15.181:0/4096589786 conn(0x7f52c00d4570 msgr2=0x7f52c0061200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:15.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 shutdown_connections 2020-11-16T22:28:15.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.129+0000 7f52bf59e700 1 -- 172.21.15.181:0/4096589786 wait complete. 2020-11-16T22:28:15.127 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:28:15.128 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 Processor -- start 2020-11-16T22:28:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 -- start start 2020-11-16T22:28:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbe80161860 0x7fbe80062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800c8db0 con 0x7fbe80161480 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe80165030 con 0x7fbe80161860 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe872de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800c8330 con 0x7fbe80062740 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe86add700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe86add700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37880/0 (socket says 172.21.15.181:37880) 2020-11-16T22:28:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe86add700 1 -- 172.21.15.181:0/3489786602 learned_addr learned my addr 172.21.15.181:0/3489786602 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:15.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 --2- 172.21.15.181:0/3489786602 >> v2:172.21.15.175:3300/0 conn(0x7fbe80161860 0x7fbe80062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 -- 172.21.15.181:0/3489786602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 msgr2=0x7fbe800644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 -- 172.21.15.181:0/3489786602 >> v1:172.21.15.175:6789/0 conn(0x7fbe80161480 legacy=0x7fbe80164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 -- 172.21.15.181:0/3489786602 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800c85d0 con 0x7fbe80161860 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe86add700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.265+0000 7fbe85adb700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80161860 0x7fbe80062250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fbe7000b9a0 tx=0x7fbe70006df0).ready entity=mon.0 client_cookie=800f5cb2e0e271ff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe852da700 1 -- 172.21.15.181:0/3489786602 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700097b0 con 0x7fbe80161860 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe852da700 1 -- 172.21.15.181:0/3489786602 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70009910 con 0x7fbe80161860 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe852da700 1 -- 172.21.15.181:0/3489786602 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70009c90 con 0x7fbe80161860 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- 172.21.15.181:0/3489786602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80161860 msgr2=0x7fbe80062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80161860 0x7fbe80062250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fbe7000b9a0 tx=0x7fbe70006df0).stop 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- 172.21.15.181:0/3489786602 shutdown_connections 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80062740 0x7fbe800644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 --2- 172.21.15.181:0/3489786602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80161860 0x7fbe80062250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- 172.21.15.181:0/3489786602 >> 172.21.15.181:0/3489786602 conn(0x7fbe800d2dc0 msgr2=0x7fbe800ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- 172.21.15.181:0/3489786602 shutdown_connections 2020-11-16T22:28:15.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- 172.21.15.181:0/3489786602 wait complete. 2020-11-16T22:28:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 Processor -- start 2020-11-16T22:28:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.269+0000 7fbe872de700 1 -- start start 2020-11-16T22:28:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80062740 0x7fbe800d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe80161860 0x7fbe800cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800caa40 con 0x7fbe80062740 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800caca0 con 0x7fbe80161860 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800caf40 con 0x7fbe80161480 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe86add700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe80161860 0x7fbe800cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37886/0 (socket says 172.21.15.181:37886) 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 -- 172.21.15.181:0/2955303951 learned_addr learned my addr 172.21.15.181:0/2955303951 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe862dc700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80062740 0x7fbe800d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 -- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe80161860 msgr2=0x7fbe800cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe80161860 0x7fbe800cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 -- 172.21.15.181:0/2955303951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80062740 msgr2=0x7fbe800d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80062740 0x7fbe800d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 -- 172.21.15.181:0/2955303951 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800cb1e0 con 0x7fbe80161480 2020-11-16T22:28:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe85adb700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fbe7000c010 tx=0x7fbe70015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70011d50 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7001e460 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7001e7e0 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800cb480 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe800d19f0 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.273+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbe7001a040 con 0x7fbe80161480 2020-11-16T22:28:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.277+0000 7fbe7e7fc700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 0x7fbe5c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.277+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe800ca8a0 con 0x7fbe80161480 2020-11-16T22:28:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.277+0000 7fbe862dc700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 0x7fbe5c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.277+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbe7006ed70 con 0x7fbe80161480 2020-11-16T22:28:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.277+0000 7fbe862dc700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 0x7fbe5c057520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbe800c8f40 tx=0x7fbe78000f00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.281+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbe800ca8a0 con 0x7fbe80161480 2020-11-16T22:28:15.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe80068fb0 con 0x7fbe80161480 2020-11-16T22:28:15.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe7e7fc700 1 -- 172.21.15.181:0/2955303951 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4626 (secure 0 0 0) 0x7fbe80068fb0 con 0x7fbe80161480 2020-11-16T22:28:15.465 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:15.466 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":30,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":5007,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:09.322519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:19.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 msgr2=0x7fbe5c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 0x7fbe5c057520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbe800c8f40 tx=0x7fbe78000f00).stop 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 msgr2=0x7fbe8015f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fbe7000c010 tx=0x7fbe70015000).stop 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 shutdown_connections 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbe5c055060 0x7fbe5c057520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe80161860 0x7fbe800cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe80062740 0x7fbe800d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 --2- 172.21.15.181:0/2955303951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe80161480 0x7fbe8015f440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 >> 172.21.15.181:0/2955303951 conn(0x7fbe800d2dc0 msgr2=0x7fbe80163170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 shutdown_connections 2020-11-16T22:28:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:15.481+0000 7fbe872de700 1 -- 172.21.15.181:0/2955303951 wait complete. 2020-11-16T22:28:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 30 2020-11-16T22:28:20.544 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:28:06.310129+0000', 'modified': '2020-11-16T22:28:09.322519+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5010}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5010': {'gid': 5010, 'name': 'e', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6841/2161152860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6839', 'nonce': 2161152860}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2161152860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:28:20.544 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:28:20.545 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:28:20.679 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5010,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.374390125,"mdsmap_epoch":30,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":27.662519824} 2020-11-16T22:28:20.680 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5010, "mdsmap_epoch": 30, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 14.374390125, "state": "up:active", "uptime": 27.662519824, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:28:20.680 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:28:20.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:20.681 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:28:20.699 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:28:20.699 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:28:20.813 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:28:20.813 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:28:20.847 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:28:20.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:28:20.847 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:28:20.876 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:28:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.025+0000 7f527b59e700 1 Processor -- start 2020-11-16T22:28:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.025+0000 7f527b59e700 1 -- start start 2020-11-16T22:28:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527b59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f527c0b2190 0x7f527c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527b59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c0ce460 con 0x7f527c0b2570 2020-11-16T22:28:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527b59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c0ce700 con 0x7f527c0b2190 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0ce9a0 con 0x7f527c15ff80 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37896/0 (socket says 172.21.15.181:37896) 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 -- 172.21.15.181:0/3362644358 learned_addr learned my addr 172.21.15.181:0/3362644358 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527a59c700 1 --2- 172.21.15.181:0/3362644358 >> v2:172.21.15.175:3300/0 conn(0x7f527c0b2190 0x7f527c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 -- 172.21.15.181:0/3362644358 >> v2:172.21.15.175:3300/0 conn(0x7f527c0b2190 msgr2=0x7f527c15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 --2- 172.21.15.181:0/3362644358 >> v2:172.21.15.175:3300/0 conn(0x7f527c0b2190 0x7f527c15de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 -- 172.21.15.181:0/3362644358 >> v1:172.21.15.175:6789/0 conn(0x7f527c0b2570 legacy=0x7f527c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 -- 172.21.15.181:0/3362644358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0cec40 con 0x7f527c15ff80 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527ad9d700 1 --2- 172.21.15.181:0/3362644358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5274009d50 tx=0x7f5274009290).ready entity=mon.2 client_cookie=fceaa795af9a08ed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527959a700 1 -- 172.21.15.181:0/3362644358 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400bc40 con 0x7f527c15ff80 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527959a700 1 -- 172.21.15.181:0/3362644358 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400bda0 con 0x7f527c15ff80 2020-11-16T22:28:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.029+0000 7f527959a700 1 -- 172.21.15.181:0/3362644358 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274011610 con 0x7f527c15ff80 2020-11-16T22:28:21.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- 172.21.15.181:0/3362644358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 msgr2=0x7f527c1603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 --2- 172.21.15.181:0/3362644358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f5274009d50 tx=0x7f5274009290).stop 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- 172.21.15.181:0/3362644358 shutdown_connections 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 --2- 172.21.15.181:0/3362644358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c1603f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 --2- 172.21.15.181:0/3362644358 >> v2:172.21.15.175:3300/0 conn(0x7f527c0b2190 0x7f527c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- 172.21.15.181:0/3362644358 >> 172.21.15.181:0/3362644358 conn(0x7f527c0cfe90 msgr2=0x7f527c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- 172.21.15.181:0/3362644358 shutdown_connections 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- 172.21.15.181:0/3362644358 wait complete. 2020-11-16T22:28:21.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 Processor -- start 2020-11-16T22:28:21.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 -- start start 2020-11-16T22:28:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.033+0000 7f527b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0b2570 0x7f527c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c170630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c171120 con 0x7f527c0b2570 2020-11-16T22:28:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c1713c0 con 0x7f527c0b2190 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c16d060 con 0x7f527c15ff80 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58288/0 (socket says 172.21.15.181:58288) 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 -- 172.21.15.181:0/4290344708 learned_addr learned my addr 172.21.15.181:0/4290344708 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527ad9d700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c170630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f5279d9b700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0b2570 0x7f527c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 -- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 msgr2=0x7f527c170630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c170630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 -- 172.21.15.181:0/4290344708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0b2570 msgr2=0x7f527c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0b2570 0x7f527c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 -- 172.21.15.181:0/4290344708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c16d300 con 0x7f527c0b2190 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527a59c700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f526c00db60 tx=0x7f526c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00be20 con 0x7f527c0b2190 2020-11-16T22:28:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c009790 con 0x7f527c0b2190 2020-11-16T22:28:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c009b30 con 0x7f527c0b2190 2020-11-16T22:28:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c16d5a0 con 0x7f527c0b2190 2020-11-16T22:28:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c156560 con 0x7f527c0b2190 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c15e290 con 0x7f527c0b2190 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f526c013480 con 0x7f527c0b2190 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 0x7f5250057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f526c06df70 con 0x7f527c0b2190 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.037+0000 7f5279d9b700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 0x7f5250057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.041+0000 7f5279d9b700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 0x7f5250057520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f527c15f2f0 tx=0x7f5270007ae0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.045+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f526c097ef0 con 0x7f527c0b2190 2020-11-16T22:28:21.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.249+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f527c044f60 con 0x7f527c0b2190 2020-11-16T22:28:21.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.761+0000 7f526a7fc700 1 -- 172.21.15.181:0/4290344708 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v31) v1 ==== 105+0+0 (secure 0 0 0) 0x7f526c044170 con 0x7f527c0b2190 2020-11-16T22:28:21.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 msgr2=0x7f5250057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 0x7f5250057520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f527c15f2f0 tx=0x7f5270007ae0).stop 2020-11-16T22:28:21.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 msgr2=0x7f527c0b7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f526c00db60 tx=0x7f526c008040).stop 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 shutdown_connections 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5250055060 0x7f5250057520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c15ff80 0x7f527c170630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c0b2190 0x7f527c0b7ce0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 --2- 172.21.15.181:0/4290344708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0b2570 0x7f527c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 >> 172.21.15.181:0/4290344708 conn(0x7f527c0cfe90 msgr2=0x7f527c0cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:21.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 shutdown_connections 2020-11-16T22:28:21.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.765+0000 7f527b59e700 1 -- 172.21.15.181:0/4290344708 wait complete. 2020-11-16T22:28:21.772 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:21.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.909+0000 7f15ecf4f700 1 Processor -- start 2020-11-16T22:28:21.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.909+0000 7f15ecf4f700 1 -- start start 2020-11-16T22:28:21.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15ecf4f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15e8140550 0x7f15e8140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15ecf4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15ecf4f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e80cf390 con 0x7f15e80cef10 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15ecf4f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e815ebb0 con 0x7f15e8140550 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15ecf4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80cf5d0 con 0x7f15e80d0a00 2020-11-16T22:28:21.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37910/0 (socket says 172.21.15.181:37910) 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 -- 172.21.15.181:0/2080323113 learned_addr learned my addr 172.21.15.181:0/2080323113 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e77fe700 1 --2- 172.21.15.181:0/2080323113 >> v2:172.21.15.175:3300/0 conn(0x7f15e8140550 0x7f15e8140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 -- 172.21.15.181:0/2080323113 >> v2:172.21.15.175:3300/0 conn(0x7f15e8140550 msgr2=0x7f15e8140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 --2- 172.21.15.181:0/2080323113 >> v2:172.21.15.175:3300/0 conn(0x7f15e8140550 0x7f15e8140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 -- 172.21.15.181:0/2080323113 >> v1:172.21.15.175:6789/0 conn(0x7f15e80cef10 legacy=0x7f15e80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 -- 172.21.15.181:0/2080323113 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e80cf870 con 0x7f15e80d0a00 2020-11-16T22:28:21.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e7fff700 1 --2- 172.21.15.181:0/2080323113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f15e0009d50 tx=0x7f15e0009290).ready entity=mon.2 client_cookie=37a8b6a96c3e45e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e67fc700 1 -- 172.21.15.181:0/2080323113 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e000bbc0 con 0x7f15e80d0a00 2020-11-16T22:28:21.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e67fc700 1 -- 172.21.15.181:0/2080323113 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e000bd20 con 0x7f15e80d0a00 2020-11-16T22:28:21.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.913+0000 7f15e67fc700 1 -- 172.21.15.181:0/2080323113 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0011610 con 0x7f15e80d0a00 2020-11-16T22:28:21.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- 172.21.15.181:0/2080323113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 msgr2=0x7f15e8063ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/2080323113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f15e0009d50 tx=0x7f15e0009290).stop 2020-11-16T22:28:21.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- 172.21.15.181:0/2080323113 shutdown_connections 2020-11-16T22:28:21.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/2080323113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e80d0a00 0x7f15e8063ce0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/2080323113 >> v2:172.21.15.175:3300/0 conn(0x7f15e8140550 0x7f15e8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- 172.21.15.181:0/2080323113 >> 172.21.15.181:0/2080323113 conn(0x7f15e80d4570 msgr2=0x7f15e80d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:21.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- 172.21.15.181:0/2080323113 shutdown_connections 2020-11-16T22:28:21.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- 172.21.15.181:0/2080323113 wait complete. 2020-11-16T22:28:21.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 Processor -- start 2020-11-16T22:28:21.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.917+0000 7f15ecf4f700 1 -- start start 2020-11-16T22:28:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 0x7f15e80b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 0x7f15e815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80cd7f0 con 0x7f15e80cef10 2020-11-16T22:28:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815a9a0 con 0x7f15e80d0a00 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815ac40 con 0x7f15e8140550 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 0x7f15e815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37916/0 (socket says 172.21.15.181:37916) 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e7fff700 1 -- 172.21.15.181:0/3199031241 learned_addr learned my addr 172.21.15.181:0/3199031241 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e77fe700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 0x7f15e80b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 -- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 msgr2=0x7f15e80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 -- 172.21.15.181:0/3199031241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 msgr2=0x7f15e80b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 0x7f15e80b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 -- 172.21.15.181:0/3199031241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e815aee0 con 0x7f15e80d0a00 2020-11-16T22:28:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e77fe700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 0x7f15e80b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e7fff700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15e6ffd700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 0x7f15e815a040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f15dc00c710 tx=0x7f15dc007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc007020 con 0x7f15e80d0a00 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc007180 con 0x7f15e80d0a00 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e80cda20 con 0x7f15e80d0a00 2020-11-16T22:28:21.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0064f0 con 0x7f15e80d0a00 2020-11-16T22:28:21.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.921+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e815a530 con 0x7f15e80d0a00 2020-11-16T22:28:21.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e815a7b0 con 0x7f15e80d0a00 2020-11-16T22:28:21.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f15dc01a060 con 0x7f15e80d0a00 2020-11-16T22:28:21.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15c77fe700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:21.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15dc06d2d0 con 0x7f15e80d0a00 2020-11-16T22:28:21.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15e77fe700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:21.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.925+0000 7f15e77fe700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 0x7f15c0057520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f15d8006fd0 tx=0x7f15d8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:21.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:21.933+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f15dc043420 con 0x7f15e80d0a00 2020-11-16T22:28:22.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15e80cf390 con 0x7f15e80d0a00 2020-11-16T22:28:22.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15c77fe700 1 -- 172.21.15.181:0/3199031241 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f15e80cf390 con 0x7f15e80d0a00 2020-11-16T22:28:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:22.117 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:21.765884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5010,"mds_1":5007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:creating","state_seq":1,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:22.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 msgr2=0x7f15c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:22.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 0x7f15c0057520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f15d8006fd0 tx=0x7f15d8006df0).stop 2020-11-16T22:28:22.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 msgr2=0x7f15e815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:22.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 0x7f15e815a040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f15dc00c710 tx=0x7f15dc007b90).stop 2020-11-16T22:28:22.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 shutdown_connections 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15e8140550 0x7f15e80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e80cef10 0x7f15e80b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 --2- 172.21.15.181:0/3199031241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15e80d0a00 0x7f15e815a040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 >> 172.21.15.181:0/3199031241 conn(0x7f15e80d4570 msgr2=0x7f15e80c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 shutdown_connections 2020-11-16T22:28:22.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:22.133+0000 7f15ecf4f700 1 -- 172.21.15.181:0/3199031241 wait complete. 2020-11-16T22:28:22.121 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2020-11-16T22:28:23.039 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:28:06.310129+0000', 'modified': '2020-11-16T22:28:21.765884+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5010, 'mds_1': 5007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5007': {'gid': 5007, 'name': 'f', 'rank': 1, 'incarnation': 32, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.175:6840/2720169666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 2720169666}, {'type': 'v1', 'addr': '172.21.15.175:6840', 'nonce': 2720169666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5010': {'gid': 5010, 'name': 'e', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6841/2161152860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6839', 'nonce': 2161152860}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2161152860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:28:23.039 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5007:up:creating 2020-11-16T22:28:23.853 INFO:tasks.ceph.mds.e.smithi175.stderr:2020-11-16T22:28:23.868+0000 7f9f35d4c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:28:23.853 INFO:tasks.ceph.mds.e.smithi175.stderr:2020-11-16T22:28:23.868+0000 7f9f35d4c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:28:24.040 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:24.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.185+0000 7f28bc8eb700 1 Processor -- start 2020-11-16T22:28:24.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.185+0000 7f28bc8eb700 1 -- start start 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28b8142660 0x7f28b8142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 0x7f28b8063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b8061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b8062700 con 0x7f28b8063620 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b80629a0 con 0x7f28b8142660 2020-11-16T22:28:24.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8062c40 con 0x7f28b8160530 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28b8142660 0x7f28b8142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b8061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28b8142660 0x7f28b8142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58320/0 (socket says 172.21.15.181:58320) 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b8061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37930/0 (socket says 172.21.15.181:37930) 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 -- 172.21.15.181:0/2400497765 learned_addr learned my addr 172.21.15.181:0/2400497765 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b6ffd700 1 --2- 172.21.15.181:0/2400497765 >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 0x7f28b8063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 -- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 msgr2=0x7f28b8061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:24.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 --2- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b8061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 -- 172.21.15.181:0/2400497765 >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 msgr2=0x7f28b8063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 --2- 172.21.15.181:0/2400497765 >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 0x7f28b8063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 -- 172.21.15.181:0/2400497765 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8062ee0 con 0x7f28b8142660 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b6ffd700 1 --2- 172.21.15.181:0/2400497765 >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 0x7f28b8063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b77fe700 1 --2- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8142660 0x7f28b8142a40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f28b8066500 tx=0x7f28a0006eb0).ready entity=mon.1 client_cookie=1b29d851c3cadb38 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b67fc700 1 -- 172.21.15.181:0/2400497765 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a00098a0 con 0x7f28b8142660 2020-11-16T22:28:24.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b67fc700 1 -- 172.21.15.181:0/2400497765 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a0009a00 con 0x7f28b8142660 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28b67fc700 1 -- 172.21.15.181:0/2400497765 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0009d80 con 0x7f28b8142660 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8142660 msgr2=0x7f28b8142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8142660 0x7f28b8142a40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f28b8066500 tx=0x7f28a0006eb0).stop 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- 172.21.15.181:0/2400497765 shutdown_connections 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b8061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/2400497765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8142660 0x7f28b8142a40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/2400497765 >> v2:172.21.15.175:3300/0 conn(0x7f28b8063620 0x7f28b8063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- 172.21.15.181:0/2400497765 >> 172.21.15.181:0/2400497765 conn(0x7f28b80d2dc0 msgr2=0x7f28b80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- 172.21.15.181:0/2400497765 shutdown_connections 2020-11-16T22:28:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.189+0000 7f28bc8eb700 1 -- 172.21.15.181:0/2400497765 wait complete. 2020-11-16T22:28:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 Processor -- start 2020-11-16T22:28:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- start start 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b8142660 0x7f28b8146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b80c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8147230 con 0x7f28b8142660 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80c9370 con 0x7f28b8063620 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8146a50 con 0x7f28b8160530 2020-11-16T22:28:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b80c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58324/0 (socket says 172.21.15.181:58324) 2020-11-16T22:28:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b77fe700 1 -- 172.21.15.181:0/1899376684 learned_addr learned my addr 172.21.15.181:0/1899376684 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b6ffd700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b8142660 0x7f28b8146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 -- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 msgr2=0x7f28b80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 -- 172.21.15.181:0/1899376684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b8142660 msgr2=0x7f28b8146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b8142660 0x7f28b8146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 -- 172.21.15.181:0/1899376684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8146cf0 con 0x7f28b8160530 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b77fe700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28b7fff700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b80c9ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f28b000ed20 tx=0x7f28b00091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b000eee0 con 0x7f28b8160530 2020-11-16T22:28:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b000cb00 con 0x7f28b8160530 2020-11-16T22:28:24.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8146f90 con 0x7f28b8160530 2020-11-16T22:28:24.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b000cea0 con 0x7f28b8160530 2020-11-16T22:28:24.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.193+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8146680 con 0x7f28b8160530 2020-11-16T22:28:24.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b8044f60 con 0x7f28b8160530 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f28b0022070 con 0x7f28b8160530 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28af7fe700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 0x7f2894057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28b006e4d0 con 0x7f28b8160530 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28b77fe700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 0x7f2894057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.197+0000 7f28b77fe700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 0x7f2894057520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f28a0009fd0 tx=0x7f28a0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:24.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.205+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f28b0044730 con 0x7f28b8160530 2020-11-16T22:28:24.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.405+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28b805d550 con 0x7f28b8160530 2020-11-16T22:28:24.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28af7fe700 1 -- 172.21.15.181:0/1899376684 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f28b0016020 con 0x7f28b8160530 2020-11-16T22:28:24.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 msgr2=0x7f2894057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:24.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 0x7f2894057520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f28a0009fd0 tx=0x7f28a0015000).stop 2020-11-16T22:28:24.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 msgr2=0x7f28b80c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:24.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b80c9ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f28b000ed20 tx=0x7f28b00091d0).stop 2020-11-16T22:28:24.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 shutdown_connections 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2894055060 0x7f2894057520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28b8160530 0x7f28b80c9ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b8063620 0x7f28b80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 --2- 172.21.15.181:0/1899376684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b8142660 0x7f28b8146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 >> 172.21.15.181:0/1899376684 conn(0x7f28b80d2dc0 msgr2=0x7f28b815cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 shutdown_connections 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:24.409+0000 7f28bc8eb700 1 -- 172.21.15.181:0/1899376684 wait complete. 2020-11-16T22:28:24.795 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 35 2020-11-16T22:28:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:24.796 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:23.782577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:25.903 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:28:06.310129+0000', 'modified': '2020-11-16T22:28:23.782577+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5010, 'mds_1': 5007, 'mds_2': 4990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4990': {'gid': 4990, 'name': 'a', 'rank': 2, 'incarnation': 34, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.181:6837/4047172210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 4047172210}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 4047172210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5007': {'gid': 5007, 'name': 'f', 'rank': 1, 'incarnation': 32, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.175:6840/2720169666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 2720169666}, {'type': 'v1', 'addr': '172.21.15.175:6840', 'nonce': 2720169666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5010': {'gid': 5010, 'name': 'e', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6841/2161152860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6839', 'nonce': 2161152860}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2161152860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:28:25.904 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:28:25.904 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:28:26.041 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":4990,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.2657034130000002,"mdsmap_epoch":35,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":33.085709989000001} 2020-11-16T22:28:26.042 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 4990, "mdsmap_epoch": 35, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 3.265703413, "state": "up:active", "uptime": 33.085709989, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:28:26.042 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json status 2020-11-16T22:28:26.180 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":5007,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.4191712159999996,"mdsmap_epoch":35,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":33.163720775000002} 2020-11-16T22:28:26.180 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5007, "mdsmap_epoch": 35, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 4.419171216, "state": "up:active", "uptime": 33.163720775, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:28:26.181 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:28:26.313 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5010,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.008196260999998,"mdsmap_epoch":35,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":33.296317377000001} 2020-11-16T22:28:26.313 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5010, "mdsmap_epoch": 35, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 20.008196261, "state": "up:active", "uptime": 33.296317377, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:28:26.313 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 Processor -- start 2020-11-16T22:28:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 -- start start 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85dc061810 0x7f85dc15e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc15d180 con 0x7f85dc13e870 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc15d810 con 0x7f85dc13ec50 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e181f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc15dab0 con 0x7f85dc061810 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85dbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f85dc13ec50 0x7f85dc061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44406/0 (socket says 172.21.15.181:44406) 2020-11-16T22:28:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85dbfff700 1 -- 172.21.15.181:0/2145911243 learned_addr learned my addr 172.21.15.181:0/2145911243 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85e101e700 1 --2- 172.21.15.181:0/2145911243 >> v2:172.21.15.175:3300/0 conn(0x7f85dc061810 0x7f85dc15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767810190 0 0) 0x7f85dc15d810 con 0x7f85dc13ec50 2020-11-16T22:28:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.461+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c0003240 con 0x7f85dc13ec50 2020-11-16T22:28:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218044633 0 0) 0x7f85dc15d180 con 0x7f85dc13e870 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85dc15d810 con 0x7f85dc13e870 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1762964198 0 0) 0x7f85c0003240 con 0x7f85dc13ec50 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 >> v2:172.21.15.175:3300/0 conn(0x7f85dc061810 msgr2=0x7f85dc15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 --2- 172.21.15.181:0/2145911243 >> v2:172.21.15.175:3300/0 conn(0x7f85dc061810 0x7f85dc15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 >> v1:172.21.15.175:6789/0 conn(0x7f85dc13e870 legacy=0x7f85dc15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc15dd50 con 0x7f85dc13ec50 2020-11-16T22:28:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f85c80014e0 con 0x7f85dc13ec50 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 >> v1:172.21.15.181:6789/0 conn(0x7f85dc13ec50 legacy=0x7f85dc061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85c00038c0 0x7f85c0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85c0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c0004e50 0x7f85c0005700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc15dab0 con 0x7f85c0004e50 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0005bf0 con 0x7f85c00044e0 2020-11-16T22:28:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0005db0 con 0x7f85c00038c0 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85c0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e081d700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85c00038c0 0x7f85c0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e101e700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c0004e50 0x7f85c0005700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 -- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85c00038c0 msgr2=0x7f85c0003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85c00038c0 0x7f85c0003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 -- 172.21.15.181:0/2145911243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c0004e50 msgr2=0x7f85c0005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c0004e50 0x7f85c0005700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 -- 172.21.15.181:0/2145911243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c0003240 con 0x7f85c00044e0 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e101e700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85c0004e50 0x7f85c0005700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e101e700 1 -- 172.21.15.181:0/2145911243 reap_dead start 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85dbfff700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85c0004960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f85c800c440 tx=0x7f85dc15d9a0).ready entity=mon.1 client_cookie=6f3352b06827f2d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800ce00 con 0x7f85c00044e0 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80072e0 con 0x7f85c00044e0 2020-11-16T22:28:26.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85db7fe700 1 -- 172.21.15.181:0/2145911243 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8005840 con 0x7f85c00044e0 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 -- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 msgr2=0x7f85c0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85c0004960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f85c800c440 tx=0x7f85dc15d9a0).stop 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 -- 172.21.15.181:0/2145911243 shutdown_connections 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 --2- 172.21.15.181:0/2145911243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85c0004960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 -- 172.21.15.181:0/2145911243 >> 172.21.15.181:0/2145911243 conn(0x7f85dc0cf630 msgr2=0x7f85dc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 -- 172.21.15.181:0/2145911243 shutdown_connections 2020-11-16T22:28:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.465+0000 7f85e181f700 1 -- 172.21.15.181:0/2145911243 wait complete. 2020-11-16T22:28:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 Processor -- start 2020-11-16T22:28:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- start start 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc13d4e0 0x7f85dc16bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0b4130 0x7f85dc0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc16b730 con 0x7f85dc0b4130 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b5b20 con 0x7f85c00044e0 2020-11-16T22:28:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b5dc0 con 0x7f85dc13d4e0 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85dbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc13d4e0 0x7f85dc16bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58344/0 (socket says 172.21.15.181:58344) 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 -- 172.21.15.181:0/1316205655 learned_addr learned my addr 172.21.15.181:0/1316205655 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e101e700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0b4130 0x7f85dc0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 -- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc13d4e0 msgr2=0x7f85dc16bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc13d4e0 0x7f85dc16bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 -- 172.21.15.181:0/1316205655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0b4130 msgr2=0x7f85dc0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0b4130 0x7f85dc0d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 -- 172.21.15.181:0/1316205655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0b6060 con 0x7f85c00044e0 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e081d700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f85d0002a90 tx=0x7f85d0007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0007e20 con 0x7f85c00044e0 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d000f8a0 con 0x7f85c00044e0 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d000fc20 con 0x7f85c00044e0 2020-11-16T22:28:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc0b6300 con 0x7f85c00044e0 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.469+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc0b6f30 con 0x7f85c00044e0 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc044f60 con 0x7f85c00044e0 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f85d001b030 con 0x7f85c00044e0 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85d8ff9700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 0x7f85b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85d006da90 con 0x7f85c00044e0 2020-11-16T22:28:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85dbfff700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 0x7f85b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.473+0000 7f85dbfff700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 0x7f85b4057520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f85c8000bc0 tx=0x7f85c8013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.477+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f85d0043d40 con 0x7f85c00044e0 2020-11-16T22:28:26.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85dc05c310 con 0x7f85c00044e0 2020-11-16T22:28:26.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85d8ff9700 1 -- 172.21.15.181:0/1316205655 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f85d0047360 con 0x7f85c00044e0 2020-11-16T22:28:26.665 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:26.665 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:23.782577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:26.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 msgr2=0x7f85b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 0x7f85b4057520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f85c8000bc0 tx=0x7f85c8013000).stop 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 msgr2=0x7f85dc0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f85d0002a90 tx=0x7f85d0007b90).stop 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 shutdown_connections 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f85b4055060 0x7f85b4057520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0b4130 0x7f85dc0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85c00044e0 0x7f85dc0b6a40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 --2- 172.21.15.181:0/1316205655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc13d4e0 0x7f85dc16bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 >> 172.21.15.181:0/1316205655 conn(0x7f85dc0cf630 msgr2=0x7f85dc15e490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 shutdown_connections 2020-11-16T22:28:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:26.681+0000 7f85e181f700 1 -- 172.21.15.181:0/1316205655 wait complete. 2020-11-16T22:28:26.669 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 35 2020-11-16T22:28:26.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:26.688 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:28:26.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:26.753 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:28:28.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:28.030 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:28:28.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:28.087 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:28:28.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:28.124 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:28:28.158 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:28.295 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:28:28.296 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:28.296 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:28.732 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:28:28.738 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:28:28.740 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:30.437 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:28:30.438 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:30.439 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:28:32.577 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:32.577 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:34.578 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:34.714 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:28:34.715 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:34.716 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:36.717 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:28:36.856 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:36.857 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:38.858 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:28:38.995 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:28:38.996 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:28:38.996 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:28:38.997 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json get subtrees 2020-11-16T22:28:39.122 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:28:39.123 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:28:39.123 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:28:41.125 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json get subtrees 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:28:41.261 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:28:41.261 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:28:43.263 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json get subtrees 2020-11-16T22:28:43.399 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:28:43.400 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:28:43.401 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:28:45.402 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok --format=json get subtrees 2020-11-16T22:28:45.537 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:28:45.538 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:28:45.539 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:28:45.539 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:28:45.678 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"15","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:28:45.679 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:28:45.679 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:28:45.680 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:45.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 Processor -- start 2020-11-16T22:28:46.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 -- start start 2020-11-16T22:28:46.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f603c0615f0 0x7f603c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c160e40 con 0x7f603c062790 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c160ba0 con 0x7f603c15f5b0 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6044250700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c1609a0 con 0x7f603c0615f0 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6041fec700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f603c15f5b0 0x7f603c0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44424/0 (socket says 172.21.15.181:44424) 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6041fec700 1 -- 172.21.15.181:0/2724813451 learned_addr learned my addr 172.21.15.181:0/2724813451 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f60417eb700 1 --2- 172.21.15.181:0/2724813451 >> v2:172.21.15.175:3300/0 conn(0x7f603c0615f0 0x7f603c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622727439 0 0) 0x7f603c160ba0 con 0x7f603c15f5b0 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6024003240 con 0x7f603c15f5b0 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2990262329 0 0) 0x7f6024003240 con 0x7f603c15f5b0 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 >> v2:172.21.15.175:3300/0 conn(0x7f603c0615f0 msgr2=0x7f603c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 --2- 172.21.15.181:0/2724813451 >> v2:172.21.15.175:3300/0 conn(0x7f603c0615f0 0x7f603c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 >> v1:172.21.15.175:6789/0 conn(0x7f603c062790 legacy=0x7f603c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c060970 con 0x7f603c15f5b0 2020-11-16T22:28:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f60380019e0 con 0x7f603c15f5b0 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f60417eb700 1 --2- 172.21.15.181:0/2724813451 >> v2:172.21.15.175:3300/0 conn(0x7f603c0615f0 0x7f603c0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 >> v1:172.21.15.181:6789/0 conn(0x7f603c15f5b0 legacy=0x7f603c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024003730 0x7f6024003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60240044e0 0x7f6024004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 0x7f602400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1609a0 con 0x7f6024003730 2020-11-16T22:28:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6024004350 con 0x7f6024004e50 2020-11-16T22:28:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602400bc50 con 0x7f60240044e0 2020-11-16T22:28:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.813+0000 7f60427ed700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 0x7f602400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60417eb700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60240044e0 0x7f6024004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6041fec700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024003730 0x7f6024003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 -- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60240044e0 msgr2=0x7f6024004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60240044e0 0x7f6024004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 -- 172.21.15.181:0/2724813451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024003730 msgr2=0x7f6024003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024003730 0x7f6024003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 -- 172.21.15.181:0/2724813451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024003240 con 0x7f6024004e50 2020-11-16T22:28:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6041fec700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024003730 0x7f6024003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 -- 172.21.15.181:0/2724813451 reap_dead start 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f60427ed700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 0x7f602400b730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f6034001770 tx=0x7f6034007b00).ready entity=mon.1 client_cookie=b3c777c309f4b05 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034007020 con 0x7f6024004e50 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6034007180 con 0x7f6024004e50 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6040fea700 1 -- 172.21.15.181:0/2724813451 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400c6c0 con 0x7f6024004e50 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 msgr2=0x7f602400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 0x7f602400b730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f6034001770 tx=0x7f6034007b00).stop 2020-11-16T22:28:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- 172.21.15.181:0/2724813451 shutdown_connections 2020-11-16T22:28:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 --2- 172.21.15.181:0/2724813451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024004e50 0x7f602400b730 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- 172.21.15.181:0/2724813451 >> 172.21.15.181:0/2724813451 conn(0x7f603c0d69a0 msgr2=0x7f603c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- 172.21.15.181:0/2724813451 shutdown_connections 2020-11-16T22:28:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- 172.21.15.181:0/2724813451 wait complete. 2020-11-16T22:28:46.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 Processor -- start 2020-11-16T22:28:46.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.817+0000 7f6044250700 1 -- start start 2020-11-16T22:28:46.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 0x7f603c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 0x7f603c172190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0cb900 con 0x7f6024004e50 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0cbb30 con 0x7f603c171d10 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0cbdd0 con 0x7f603c0d16d0 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 0x7f603c172190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60427ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60427ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37978/0 (socket says 172.21.15.181:37978) 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60427ed700 1 -- 172.21.15.181:0/65677467 learned_addr learned my addr 172.21.15.181:0/65677467 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6041fec700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 0x7f603c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 -- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 msgr2=0x7f603c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 -- 172.21.15.181:0/65677467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 msgr2=0x7f603c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 0x7f603c0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 -- 172.21.15.181:0/65677467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0cc070 con 0x7f603c171d10 2020-11-16T22:28:46.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6041fec700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 0x7f603c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:46.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60427ed700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:46.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f60417eb700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 0x7f603c172190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f602c0108a0 tx=0x7f602c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c0126d0 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c012830 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c0cc310 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c012bb0 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c170700 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.821+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f602c014430 con 0x7f603c171d10 2020-11-16T22:28:46.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.825+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c069eb0 con 0x7f603c171d10 2020-11-16T22:28:46.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.825+0000 7f6029ffb700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 0x7f6014057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:46.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.825+0000 7f6041fec700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 0x7f6014057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:46.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.825+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f602c06e150 con 0x7f603c171d10 2020-11-16T22:28:46.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.825+0000 7f6041fec700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 0x7f6014057520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6038001230 tx=0x7f6038009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:46.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:45.829+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f602c044350 con 0x7f603c171d10 2020-11-16T22:28:46.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.033+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603c0cf640 con 0x7f603c171d10 2020-11-16T22:28:46.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.033+0000 7f6029ffb700 1 -- 172.21.15.181:0/65677467 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f602c047970 con 0x7f603c171d10 2020-11-16T22:28:46.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 msgr2=0x7f6014057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 0x7f6014057520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6038001230 tx=0x7f6038009040).stop 2020-11-16T22:28:46.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 msgr2=0x7f603c172190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 0x7f603c172190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f602c0108a0 tx=0x7f602c00b040).stop 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 shutdown_connections 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6014055060 0x7f6014057520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f603c0d16d0 0x7f603c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6024004e50 0x7f603c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 --2- 172.21.15.181:0/65677467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603c171d10 0x7f603c172190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:46.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 >> 172.21.15.181:0/65677467 conn(0x7f603c0d69a0 msgr2=0x7f603c061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:46.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 shutdown_connections 2020-11-16T22:28:46.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:46.037+0000 7f6044250700 1 -- 172.21.15.181:0/65677467 wait complete. 2020-11-16T22:28:46.830 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 37 2020-11-16T22:28:46.830 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:46.830 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":37,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:43.783511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:47.034 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e flush journal 2020-11-16T22:28:47.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 Processor -- start 2020-11-16T22:28:47.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 -- start start 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 0x7fbd90164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd90164e10 0x7fbd90062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900c8db0 con 0x7fbd90161860 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900c8330 con 0x7fbd90164e10 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd950e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c85d0 con 0x7fbd90161480 2020-11-16T22:28:47.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 0x7fbd90164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd948e4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbd90161860 0x7fbd90062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44448/0 (socket says 172.21.15.181:44448) 2020-11-16T22:28:47.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd948e4700 1 -- 172.21.15.181:0/3541695677 learned_addr learned my addr 172.21.15.181:0/3541695677 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:47.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.185+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/3541695677 >> v2:172.21.15.175:3300/0 conn(0x7fbd90164e10 0x7fbd90062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8effd700 1 -- 172.21.15.181:0/3541695677 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416892219 0 0) 0x7fbd900c8db0 con 0x7fbd90161860 2020-11-16T22:28:47.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8ffff700 1 -- 172.21.15.181:0/3541695677 >> v2:172.21.15.175:3300/0 conn(0x7fbd90164e10 msgr2=0x7fbd90062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:47.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3541695677 >> v2:172.21.15.175:3300/0 conn(0x7fbd90164e10 0x7fbd90062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8ffff700 1 -- 172.21.15.181:0/3541695677 >> v1:172.21.15.181:6789/0 conn(0x7fbd90161860 legacy=0x7fbd90062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8ffff700 1 -- 172.21.15.181:0/3541695677 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900c8870 con 0x7fbd90161480 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3541695677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 0x7fbd90164920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fbd88009c90 tx=0x7fbd880091d0).ready entity=mon.2 client_cookie=ac82f7aa629a94ee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8effd700 1 -- 172.21.15.181:0/3541695677 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800ba70 con 0x7fbd90161480 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8effd700 1 -- 172.21.15.181:0/3541695677 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8800bbd0 con 0x7fbd90161480 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd8effd700 1 -- 172.21.15.181:0/3541695677 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011400 con 0x7fbd90161480 2020-11-16T22:28:47.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- 172.21.15.181:0/3541695677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 msgr2=0x7fbd90164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:47.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3541695677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 0x7fbd90164920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fbd88009c90 tx=0x7fbd880091d0).stop 2020-11-16T22:28:47.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- 172.21.15.181:0/3541695677 shutdown_connections 2020-11-16T22:28:47.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3541695677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd90161480 0x7fbd90164920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3541695677 >> v2:172.21.15.175:3300/0 conn(0x7fbd90164e10 0x7fbd90062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- 172.21.15.181:0/3541695677 >> 172.21.15.181:0/3541695677 conn(0x7fbd900d2dc0 msgr2=0x7fbd900ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:47.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- 172.21.15.181:0/3541695677 shutdown_connections 2020-11-16T22:28:47.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- 172.21.15.181:0/3541695677 wait complete. 2020-11-16T22:28:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 Processor -- start 2020-11-16T22:28:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.189+0000 7fbd950e5700 1 -- start start 2020-11-16T22:28:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 0x7fbd901779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90161860 0x7fbd90149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90177ea0 con 0x7fbd90161860 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901499b0 con 0x7fbd90161480 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90149c50 con 0x7fbd900d0540 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 0x7fbd901779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 0x7fbd901779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58378/0 (socket says 172.21.15.181:58378) 2020-11-16T22:28:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37990/0 (socket says 172.21.15.181:37990) 2020-11-16T22:28:47.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd8ffff700 1 -- 172.21.15.181:0/3918476455 learned_addr learned my addr 172.21.15.181:0/3918476455 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:47.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90161860 0x7fbd90149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 -- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 msgr2=0x7fbd901779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:47.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 0x7fbd901779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 -- 172.21.15.181:0/3918476455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90161860 msgr2=0x7fbd90149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90161860 0x7fbd90149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 -- 172.21.15.181:0/3918476455 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90149ef0 con 0x7fbd900d0540 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd948e4700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fbd7800c730 tx=0x7fbd780078e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd87fff700 1 -- 172.21.15.181:0/3918476455 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780072e0 con 0x7fbd900d0540 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd87fff700 1 -- 172.21.15.181:0/3918476455 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78007460 con 0x7fbd900d0540 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd90144940 con 0x7fbd900d0540 2020-11-16T22:28:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd87fff700 1 -- 172.21.15.181:0/3918476455 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78005840 con 0x7fbd900d0540 2020-11-16T22:28:47.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.193+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd90161f60 con 0x7fbd900d0540 2020-11-16T22:28:47.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.197+0000 7fbd87fff700 1 -- 172.21.15.181:0/3918476455 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbd78010070 con 0x7fbd900d0540 2020-11-16T22:28:47.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.197+0000 7fbd87fff700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 0x7fbd60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.197+0000 7fbd87fff700 1 -- 172.21.15.181:0/3918476455 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd7806d260 con 0x7fbd900d0540 2020-11-16T22:28:47.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.197+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 0x7fbd60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.197+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 0x7fbd60057520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbd8800b770 tx=0x7fbd88016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:47.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 Processor -- start 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- start start 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 0x23ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22534b0 con 0x213bbb0 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2307930 con 0x23c7fd0 2020-11-16T22:28:47.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21c76f0 con 0x21a97b0 2020-11-16T22:28:47.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 0x23ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58390/0 (socket says 172.21.15.181:58390) 2020-11-16T22:28:47.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 0x23ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:37998/0 (socket says 172.21.15.181:37998) 2020-11-16T22:28:47.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 -- 172.21.15.181:0/1143736862 learned_addr learned my addr 172.21.15.181:0/1143736862 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:47.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 -- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 msgr2=0x23ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 0x23ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 -- 172.21.15.181:0/1143736862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 msgr2=0x23c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 -- 172.21.15.181:0/1143736862 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21c25a0 con 0x23c7fd0 2020-11-16T22:28:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd948e4700 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fbd7800a000 tx=0x7fbd78010000).ready entity=mon.1 client_cookie=852082a31e418037 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd857fa700 1 -- 172.21.15.181:0/1143736862 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7804e070 con 0x23c7fd0 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd857fa700 1 -- 172.21.15.181:0/1143736862 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78045390 con 0x23c7fd0 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbd857fa700 1 -- 172.21.15.181:0/1143736862 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7806d0d0 con 0x23c7fd0 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 msgr2=0x23c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fbd7800a000 tx=0x7fbd78010000).stop 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- 172.21.15.181:0/1143736862 shutdown_connections 2020-11-16T22:28:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23c7fd0 0x23c1700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21a97b0 0x23ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/1143736862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- 172.21.15.181:0/1143736862 >> 172.21.15.181:0/1143736862 conn(0x2377e40 msgr2=0x23c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- 172.21.15.181:0/1143736862 shutdown_connections 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.201+0000 7fbda4dd8740 1 -- 172.21.15.181:0/1143736862 wait complete. 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- start start 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 Processor -- start 2020-11-16T22:28:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f46e0 con 0x213bbb0 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f4820 con 0x21a97b0 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22f5a90 con 0x23c7fd0 2020-11-16T22:28:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 10 client.0 ll_register_callbacks cb 0x22eaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58396/0 (socket says 172.21.15.181:58396) 2020-11-16T22:28:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38004/0 (socket says 172.21.15.181:38004) 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/3926907934 learned_addr learned my addr 172.21.15.181:0/3926907934 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 msgr2=0x23e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 msgr2=0x23c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/3926907934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x22e5280 con 0x21a97b0 2020-11-16T22:28:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd948e4700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fbd90061220 tx=0x7fbd80006df0).ready entity=mon.1 client_cookie=49f73ac0e4891142 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd7e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:28:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800156f0 con 0x21a97b0 2020-11-16T22:28:48.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd80015850 con 0x21a97b0 2020-11-16T22:28:48.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23a86c0 con 0x21a97b0 2020-11-16T22:28:48.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd80015a10 con 0x21a97b0 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 10 client.5239 fetch_fsmap learned FSMap version 37 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.205+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x211afd0 con 0x21a97b0 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23a86c0 con 0x21a97b0 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fbd80015b70 con 0x21a97b0 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 10 client.5239 fetch_fsmap finished waiting for FSMap version 37 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 10 client.5239 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:48.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 20 client.5239 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2418ce0 0x23c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 4 client.5239 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x21e5c50 con 0x2418ce0 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2418ce0 0x23c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 10 client.5239 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2418ce0 0x23c6b10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=88aa1ad28ce7d013 server_cookie=f13c04a971bfdf14 in_seq=0 out_seq=0 2020-11-16T22:28:48.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/3926907934 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x21e5c50 con 0x2418ce0 2020-11-16T22:28:48.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 10 client.5239 handle_command_reply: tid=0 2020-11-16T22:28:48.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 client.5239 shutdown 2020-11-16T22:28:48.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 msgr2=0x24187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fbd90061220 tx=0x7fbd80006df0).stop 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 shutdown_connections 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2418ce0 0x23c6b10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23c7fd0 0x23e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x213bbb0 0x23c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbd7e7fc700 0 client.5239 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:48.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/3926907934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21a97b0 0x24187f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 >> 172.21.15.181:0/3926907934 conn(0x2377e40 msgr2=0x23c2560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 shutdown_connections 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 1 -- 172.21.15.181:0/3926907934 wait complete. 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.209+0000 7fbda4dd8740 20 client.5239 trim_cache size 0 max 16384 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.229+0000 7fbda4dd8740 1 Processor -- start 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.229+0000 7fbda4dd8740 1 -- start start 2020-11-16T22:28:48.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 0x23bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x24125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:48.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cce40 con 0x21a97b0 2020-11-16T22:28:48.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c3600 con 0x213bbb0 2020-11-16T22:28:48.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2412d60 con 0x2377e40 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 0x23bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58402/0 (socket says 172.21.15.181:58402) 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/309772485 learned_addr learned my addr 172.21.15.181:0/309772485 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd948e4700 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x24125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 msgr2=0x23bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:48.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 0x23bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/309772485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 msgr2=0x24125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x24125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 -- 172.21.15.181:0/309772485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580081d0 con 0x213bbb0 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 0x23bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fbd8001d840 tx=0x7fbd8001d000).ready entity=mon.1 client_cookie=19029e0a11a12832 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd85ffb700 1 -- 172.21.15.181:0/309772485 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80014c80 con 0x213bbb0 2020-11-16T22:28:48.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd85ffb700 1 -- 172.21.15.181:0/309772485 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80006a60 con 0x213bbb0 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbd85ffb700 1 -- 172.21.15.181:0/309772485 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800156f0 con 0x213bbb0 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 msgr2=0x23faa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fbd8001d840 tx=0x7fbd8001d000).stop 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- 172.21.15.181:0/309772485 shutdown_connections 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x24125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2377e40 0x23bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/309772485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x213bbb0 0x23faa20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- 172.21.15.181:0/309772485 >> 172.21.15.181:0/309772485 conn(0x7fbd58002320 msgr2=0x23e8040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- 172.21.15.181:0/309772485 shutdown_connections 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- 172.21.15.181:0/309772485 wait complete. 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 -- start start 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.233+0000 7fbda4dd8740 1 Processor -- start 2020-11-16T22:28:48.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x213bbb0 0x23c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x23bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2479e50 con 0x21a97b0 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c2db0 con 0x2377e40 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x247a040 con 0x213bbb0 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 10 client.0 ll_register_callbacks cb 0x23c8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x213bbb0 0x23c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58410/0 (socket says 172.21.15.181:58410) 2020-11-16T22:28:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 -- 172.21.15.181:0/4282550885 learned_addr learned my addr 172.21.15.181:0/4282550885 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd8f7fe700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x23bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 -- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x213bbb0 msgr2=0x23c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x213bbb0 0x23c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 -- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 msgr2=0x23bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x23bc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 -- 172.21.15.181:0/4282550885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2159720 con 0x2377e40 2020-11-16T22:28:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd948e4700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fbd78049040 tx=0x7fbd7806d000).ready entity=mon.1 client_cookie=9cf907b6cef14c75 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78058070 con 0x2377e40 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd78045390 con 0x2377e40 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd78005e00 con 0x2377e40 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd7806d0d0 con 0x2377e40 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 10 client.5245 fetch_fsmap learned FSMap version 37 2020-11-16T22:28:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x2164e00 con 0x2377e40 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78045390 con 0x2377e40 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fbd7801b020 con 0x2377e40 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 10 client.5245 fetch_fsmap finished waiting for FSMap version 37 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 10 client.5245 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 20 client.5245 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23c7fd0 0x247a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 4 client.5245 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:28:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x21ecf50 con 0x23c7fd0 2020-11-16T22:28:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23c7fd0 0x247a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd7e7fc700 10 client.5245 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.237+0000 7fbd8ffff700 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23c7fd0 0x247a4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=796a3f6bf72c5e01 server_cookie=fe65b9641749dba3 in_seq=0 out_seq=0 2020-11-16T22:28:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd7e7fc700 1 -- 172.21.15.181:0/4282550885 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x21ecf50 con 0x23c7fd0 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd7e7fc700 10 client.5245 handle_command_reply: tid=0 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 client.5245 shutdown 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 msgr2=0x246f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fbd78049040 tx=0x7fbd7806d000).stop 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 shutdown_connections 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23c7fd0 0x247a4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2377e40 0x246f9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd7e7fc700 0 client.5245 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x213bbb0 0x23c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 --2- 172.21.15.181:0/4282550885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21a97b0 0x23bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 >> 172.21.15.181:0/4282550885 conn(0x7fbd58002320 msgr2=0x23c3770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 shutdown_connections 2020-11-16T22:28:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 1 -- 172.21.15.181:0/4282550885 wait complete. 2020-11-16T22:28:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbda4dd8740 20 client.5245 trim_cache size 0 max 16384 2020-11-16T22:28:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 msgr2=0x7fbd60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 0x7fbd60057520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbd8800b770 tx=0x7fbd88016000).stop 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 msgr2=0x7fbd90163460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fbd7800c730 tx=0x7fbd780078e0).stop 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 shutdown_connections 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbd60055060 0x7fbd60057520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd900d0540 0x7fbd90163460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd90161480 0x7fbd901779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 --2- 172.21.15.181:0/3918476455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90161860 0x7fbd90149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 >> 172.21.15.181:0/3918476455 conn(0x7fbd900d2dc0 msgr2=0x7fbd90147390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 shutdown_connections 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:47.269+0000 7fbd950e5700 1 -- 172.21.15.181:0/3918476455 wait complete. 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:28:49.640 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:28:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:28:49.641 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:28:49.641 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:49.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 Processor -- start 2020-11-16T22:28:49.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- start start 2020-11-16T22:28:49.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f19dc1492c0 0x7f19dc080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc061b50 con 0x7f19dc0814c0 2020-11-16T22:28:49.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc0618b0 con 0x7f19dc1492c0 2020-11-16T22:28:49.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0cb930 con 0x7f19dc145a30 2020-11-16T22:28:49.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38024/0 (socket says 172.21.15.181:38024) 2020-11-16T22:28:49.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 -- 172.21.15.181:0/3529416907 learned_addr learned my addr 172.21.15.181:0/3529416907 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:49.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19dbfff700 1 --2- 172.21.15.181:0/3529416907 >> v2:172.21.15.175:3300/0 conn(0x7f19dc1492c0 0x7f19dc080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 -- 172.21.15.181:0/3529416907 >> v2:172.21.15.175:3300/0 conn(0x7f19dc1492c0 msgr2=0x7f19dc080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:49.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 --2- 172.21.15.181:0/3529416907 >> v2:172.21.15.175:3300/0 conn(0x7f19dc1492c0 0x7f19dc080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 -- 172.21.15.181:0/3529416907 >> v1:172.21.15.175:6789/0 conn(0x7f19dc0814c0 legacy=0x7f19dc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 -- 172.21.15.181:0/3529416907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0cbbd0 con 0x7f19dc145a30 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e0ebe700 1 --2- 172.21.15.181:0/3529416907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f19d4009d50 tx=0x7f19d4009290).ready entity=mon.2 client_cookie=80b637637305fb2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19db7fe700 1 -- 172.21.15.181:0/3529416907 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400bc40 con 0x7f19dc145a30 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19db7fe700 1 -- 172.21.15.181:0/3529416907 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d400bda0 con 0x7f19dc145a30 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19db7fe700 1 -- 172.21.15.181:0/3529416907 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4011610 con 0x7f19dc145a30 2020-11-16T22:28:49.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- 172.21.15.181:0/3529416907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 msgr2=0x7f19dc145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/3529416907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f19d4009d50 tx=0x7f19d4009290).stop 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- 172.21.15.181:0/3529416907 shutdown_connections 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/3529416907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc145a30 0x7f19dc145e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/3529416907 >> v2:172.21.15.175:3300/0 conn(0x7f19dc1492c0 0x7f19dc080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- 172.21.15.181:0/3529416907 >> 172.21.15.181:0/3529416907 conn(0x7f19dc0d68c0 msgr2=0x7f19dc0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- 172.21.15.181:0/3529416907 shutdown_connections 2020-11-16T22:28:49.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.785+0000 7f19e1ec0700 1 -- 172.21.15.181:0/3529416907 wait complete. 2020-11-16T22:28:49.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 Processor -- start 2020-11-16T22:28:49.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 -- start start 2020-11-16T22:28:49.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 0x7f19dc142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1492c0 0x7f19dc060620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0d3250 con 0x7f19dc1492c0 2020-11-16T22:28:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc141fa0 con 0x7f19dc145a30 2020-11-16T22:28:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e1ec0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc142240 con 0x7f19dc0814c0 2020-11-16T22:28:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38030/0 (socket says 172.21.15.181:38030) 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 -- 172.21.15.181:0/1912378430 learned_addr learned my addr 172.21.15.181:0/1912378430 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19dbfff700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 0x7f19dc142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e16bf700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1492c0 0x7f19dc060620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 -- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 msgr2=0x7f19dc142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 0x7f19dc142c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 -- 172.21.15.181:0/1912378430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1492c0 msgr2=0x7f19dc060620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1492c0 0x7f19dc060620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 -- 172.21.15.181:0/1912378430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc1424e0 con 0x7f19dc0814c0 2020-11-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19dbfff700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 0x7f19dc142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:50.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.789+0000 7f19e0ebe700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f19dc0d78a0 tx=0x7f19d4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d401b5e0 con 0x7f19dc0814c0 2020-11-16T22:28:50.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d401b740 con 0x7f19dc0814c0 2020-11-16T22:28:50.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc142780 con 0x7f19dc0814c0 2020-11-16T22:28:50.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d401bac0 con 0x7f19dc0814c0 2020-11-16T22:28:50.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc143110 con 0x7f19dc0814c0 2020-11-16T22:28:50.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc142940 con 0x7f19dc0814c0 2020-11-16T22:28:50.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f19d4017070 con 0x7f19dc0814c0 2020-11-16T22:28:50.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19d8ff9700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 0x7f19b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.793+0000 7f19dbfff700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 0x7f19b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.797+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19d4070980 con 0x7f19dc0814c0 2020-11-16T22:28:50.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.797+0000 7f19dbfff700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 0x7f19b4057520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f19dc0d1310 tx=0x7f19cc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:49.801+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f19d4046b50 con 0x7f19dc0814c0 2020-11-16T22:28:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19dc044f60 con 0x7f19dc0814c0 2020-11-16T22:28:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19d8ff9700 1 -- 172.21.15.181:0/1912378430 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f19d4015030 con 0x7f19dc0814c0 2020-11-16T22:28:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 msgr2=0x7f19b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 0x7f19b4057520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f19dc0d1310 tx=0x7f19cc008040).stop 2020-11-16T22:28:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 msgr2=0x7f19dc0d5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f19dc0d78a0 tx=0x7f19d4016000).stop 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 shutdown_connections 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19b4055060 0x7f19b4057520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1492c0 0x7f19dc060620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19dc0814c0 0x7f19dc0d5030 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 --2- 172.21.15.181:0/1912378430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19dc145a30 0x7f19dc142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.005+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 >> 172.21.15.181:0/1912378430 conn(0x7f19dc0d68c0 msgr2=0x7f19dc13ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.009+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 shutdown_connections 2020-11-16T22:28:50.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.009+0000 7f19e1ec0700 1 -- 172.21.15.181:0/1912378430 wait complete. 2020-11-16T22:28:50.392 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f flush journal 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.745+0000 7f0b44956700 1 Processor -- start 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.745+0000 7f0b44956700 1 -- start start 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b401426d0 0x7f0b40142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b40160bf0 0x7f0b40061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b40142fa0 con 0x7f0b40160740 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b40064af0 con 0x7f0b401426d0 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b40064d90 con 0x7f0b40160bf0 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b40160bf0 0x7f0b40061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0b40160740 0x7f0b40061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44500/0 (socket says 172.21.15.181:44500) 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1684794204 learned_addr learned my addr 172.21.15.181:0/1684794204 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1684794204 >> v2:172.21.15.175:3300/0 conn(0x7f0b401426d0 0x7f0b40142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1684794204 >> v2:172.21.15.175:3300/0 conn(0x7f0b401426d0 msgr2=0x7f0b40142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1684794204 >> v2:172.21.15.175:3300/0 conn(0x7f0b401426d0 0x7f0b40142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1684794204 >> v1:172.21.15.181:6789/0 conn(0x7f0b40160740 legacy=0x7f0b40061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1684794204 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40065030 con 0x7f0b40160bf0 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3d59a700 1 -- 172.21.15.181:0/1684794204 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689838502 0 0) 0x7f0b40142fa0 con 0x7f0b40160740 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1684794204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40061810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0b40069200 tx=0x7f0b38006df0).ready entity=mon.1 client_cookie=6871dad573e63598 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3d59a700 1 -- 172.21.15.181:0/1684794204 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b380097b0 con 0x7f0b40160bf0 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3d59a700 1 -- 172.21.15.181:0/1684794204 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38009910 con 0x7f0b40160bf0 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b3d59a700 1 -- 172.21.15.181:0/1684794204 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38009c90 con 0x7f0b40160bf0 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- 172.21.15.181:0/1684794204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 msgr2=0x7f0b40061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 --2- 172.21.15.181:0/1684794204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40061810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0b40069200 tx=0x7f0b38006df0).stop 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- 172.21.15.181:0/1684794204 shutdown_connections 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 --2- 172.21.15.181:0/1684794204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40061810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 --2- 172.21.15.181:0/1684794204 >> v2:172.21.15.175:3300/0 conn(0x7f0b401426d0 0x7f0b40142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- 172.21.15.181:0/1684794204 >> 172.21.15.181:0/1684794204 conn(0x7f0b400cf890 msgr2=0x7f0b400cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- 172.21.15.181:0/1684794204 shutdown_connections 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.749+0000 7f0b44956700 1 -- 172.21.15.181:0/1684794204 wait complete. 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 Processor -- start 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 -- start start 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 0x7f0b4015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 0x7f0b4015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4016eeb0 con 0x7f0b401426d0 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b401730a0 con 0x7f0b40160bf0 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b44956700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4016f0a0 con 0x7f0b40160740 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 0x7f0b4015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 0x7f0b4015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38046/0 (socket says 172.21.15.181:38046) 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58438/0 (socket says 172.21.15.181:58438) 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1219164976 learned_addr learned my addr 172.21.15.181:0/1219164976 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 0x7f0b4015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 msgr2=0x7f0b4015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 0x7f0b4015a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1219164976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 msgr2=0x7f0b4015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 0x7f0b4015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.753+0000 7f0b3ed9d700 1 -- 172.21.15.181:0/1219164976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4016f340 con 0x7f0b40160bf0 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 0x7f0b4015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0b38012000 tx=0x7f0b38015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 -- 172.21.15.181:0/1219164976 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38009e40 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 -- 172.21.15.181:0/1219164976 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3801d5d0 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4016f5e0 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 -- 172.21.15.181:0/1219164976 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3801d950 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4016fd20 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 -- 172.21.15.181:0/1219164976 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0b3801a040 con 0x7f0b40160bf0 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 0x7f0b14057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b2e7fc700 1 -- 172.21.15.181:0/1219164976 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b3806f3d0 con 0x7f0b40160bf0 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.757+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 0x7f0b14057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 0x7f0b14057520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f0b3000c790 tx=0x7f0b30008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 Processor -- start 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 -- start start 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x1598cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 0x15f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x142b4b0 con 0x159a470 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14df930 con 0x1313bb0 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x139f6f0 con 0x13817b0 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x1598cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58444/0 (socket says 172.21.15.181:58444) 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x1598cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38052/0 (socket says 172.21.15.181:38052) 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1638566631 learned_addr learned my addr 172.21.15.181:0/1638566631 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 0x15f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 msgr2=0x1598cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x1598cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1638566631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 msgr2=0x15f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 0x15f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/1638566631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x139a5a0 con 0x1313bb0 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 0x15f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0b34006930 tx=0x7f0b34007a60).ready entity=mon.1 client_cookie=d2254a6853de63c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b1b7fe700 1 -- 172.21.15.181:0/1638566631 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340017c0 con 0x1313bb0 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b1b7fe700 1 -- 172.21.15.181:0/1638566631 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34001920 con 0x1313bb0 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.761+0000 7f0b1b7fe700 1 -- 172.21.15.181:0/1638566631 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34001cc0 con 0x1313bb0 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 msgr2=0x15d7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0b34006930 tx=0x7f0b34007a60).stop 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- 172.21.15.181:0/1638566631 shutdown_connections 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x159a470 0x15f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x1598cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1638566631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1313bb0 0x15d7530 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- 172.21.15.181:0/1638566631 >> 172.21.15.181:0/1638566631 conn(0x154fe40 msgr2=0x15f4580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- 172.21.15.181:0/1638566631 shutdown_connections 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- 172.21.15.181:0/1638566631 wait complete. 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 -- start start 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 Processor -- start 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.765+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x15ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x159a470 0x1601d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cc6e0 con 0x13817b0 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cc820 con 0x159a470 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14cda90 con 0x1313bb0 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 10 client.0 ll_register_callbacks cb 0x14c2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38058/0 (socket says 172.21.15.181:38058) 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 -- 172.21.15.181:0/1333916326 learned_addr learned my addr 172.21.15.181:0/1333916326 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x159a470 0x1601d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x15ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 -- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x159a470 msgr2=0x1601d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x159a470 0x1601d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 -- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 msgr2=0x15ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x15ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 -- 172.21.15.181:0/1333916326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x14bd280 con 0x1313bb0 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x15ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0b3000fa30 tx=0x7f0b30015000).ready entity=mon.2 client_cookie=75f3a53045f752c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300147b0 con 0x1313bb0 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b300139d0 con 0x1313bb0 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1594e10 con 0x1313bb0 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b30013da0 con 0x1313bb0 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 10 client.5018 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x12f2fd0 con 0x1313bb0 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1594e10 con 0x1313bb0 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f0b3002f020 con 0x1313bb0 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 10 client.5018 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 10 client.5018 resolve_mds: resolved ID 'f' to GID 5007 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 20 client.5018 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x15f0510 0x1602560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 4 client.5018 mds_command: new command op to 5007 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 --> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x13bdc50 con 0x15f0510 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x15f0510 0x1602560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 10 client.5018 ms_handle_connect on v2:172.21.15.175:6838/2720169666 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x15f0510 0x1602560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=d2adadb4c61b3dea server_cookie=5a4edebef6c98371 in_seq=0 out_seq=0 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 1 -- 172.21.15.181:0/1333916326 <== mds.1 v2:172.21.15.175:6838/2720169666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x13bdc50 con 0x15f0510 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b18ff9700 10 client.5018 handle_command_reply: tid=0 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.769+0000 7f0b533e7740 1 client.5018 shutdown 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 msgr2=0x1600460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0b3000fa30 tx=0x7f0b30015000).stop 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 shutdown_connections 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x15f0510 0x1602560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x159a470 0x1601d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x1600460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 --2- 172.21.15.181:0/1333916326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x15ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b18ff9700 0 client.5018 ms_handle_reset on v2:172.21.15.175:6838/2720169666 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 >> 172.21.15.181:0/1333916326 conn(0x154fe40 msgr2=0x15d72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 shutdown_connections 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 1 -- 172.21.15.181:0/1333916326 wait complete. 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.773+0000 7f0b533e7740 20 client.5018 trim_cache size 0 max 16384 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 Processor -- start 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 -- start start 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x159af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x159b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14a4e40 con 0x13817b0 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x15f5250 con 0x154fe40 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15d7ec0 con 0x1313bb0 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x159af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58456/0 (socket says 172.21.15.181:58456) 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 -- 172.21.15.181:0/4196967200 learned_addr learned my addr 172.21.15.181:0/4196967200 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3ed9d700 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x159b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 -- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 msgr2=0x159af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x159af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 -- 172.21.15.181:0/4196967200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 msgr2=0x159b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x159b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 -- 172.21.15.181:0/4196967200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080081d0 con 0x154fe40 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x159af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0b30006ea0 tx=0x7f0b30025000).ready entity=mon.1 client_cookie=913e75aef248f21d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b1bfff700 1 -- 172.21.15.181:0/4196967200 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30013640 con 0x154fe40 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b1bfff700 1 -- 172.21.15.181:0/4196967200 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b30014d60 con 0x154fe40 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.789+0000 7f0b1bfff700 1 -- 172.21.15.181:0/4196967200 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30020850 con 0x154fe40 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 msgr2=0x15c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0b30006ea0 tx=0x7f0b30025000).stop 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/4196967200 shutdown_connections 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13817b0 0x159b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15c0830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- 172.21.15.181:0/4196967200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1313bb0 0x159af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/4196967200 >> 172.21.15.181:0/4196967200 conn(0x7f0b08002320 msgr2=0x1601540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/4196967200 shutdown_connections 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/4196967200 wait complete. 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- start start 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 Processor -- start 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 0x15c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x15f3db0 con 0x1313bb0 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1652080 con 0x154fe40 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15c23a0 con 0x13817b0 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 10 client.0 ll_register_callbacks cb 0x15d9c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38074/0 (socket says 172.21.15.181:38074) 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/2332871444 learned_addr learned my addr 172.21.15.181:0/2332871444 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 0x15c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 msgr2=0x15de670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15de670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 msgr2=0x15c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 0x15c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 -- 172.21.15.181:0/2332871444 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1331720 con 0x13817b0 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 0x15c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b3dd9b700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f0b340018f0 tx=0x7f0b34016000).ready entity=mon.2 client_cookie=8f0d43efa39b9002 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34001ca0 con 0x13817b0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b34022870 con 0x13817b0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.793+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b34001920 con 0x13817b0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b34022c40 con 0x13817b0 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 10 client.5024 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x133ce00 con 0x13817b0 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34001920 con 0x13817b0 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f0b3401c040 con 0x13817b0 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 10 client.5024 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 10 client.5024 resolve_mds: resolved ID 'f' to GID 5007 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 20 client.5024 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x159a470 0x15deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 4 client.5024 mds_command: new command op to 5007 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 --> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x13c4f50 con 0x159a470 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x159a470 0x15deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b18ff9700 10 client.5024 ms_handle_connect on v2:172.21.15.175:6838/2720169666 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.797+0000 7f0b3e59c700 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x159a470 0x15deb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=155e760303b1fd70 server_cookie=94570efdedd8b320 in_seq=0 out_seq=0 2020-11-16T22:28:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.813+0000 7f0b18ff9700 1 -- 172.21.15.181:0/2332871444 <== mds.1 v2:172.21.15.175:6838/2720169666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x13c4f50 con 0x159a470 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.813+0000 7f0b18ff9700 10 client.5024 handle_command_reply: tid=0 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 client.5024 shutdown 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 msgr2=0x164caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f0b340018f0 tx=0x7f0b34016000).stop 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 shutdown_connections 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:6838/2720169666,v1:172.21.15.175:6840/2720169666] conn(0x159a470 0x15deb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x154fe40 0x15de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1313bb0 0x15c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 --2- 172.21.15.181:0/2332871444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13817b0 0x164caa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b18ff9700 0 client.5024 ms_handle_reset on v2:172.21.15.175:6838/2720169666 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 >> 172.21.15.181:0/2332871444 conn(0x7f0b08002320 msgr2=0x15c2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 shutdown_connections 2020-11-16T22:28:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 1 -- 172.21.15.181:0/2332871444 wait complete. 2020-11-16T22:28:50.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b533e7740 20 client.5024 trim_cache size 0 max 16384 2020-11-16T22:28:50.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 msgr2=0x7f0b14057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 0x7f0b14057520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f0b3000c790 tx=0x7f0b30008040).stop 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 msgr2=0x7f0b40172400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0b38012000 tx=0x7f0b38015000).stop 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 shutdown_connections 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b14055060 0x7f0b14057520 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b40160bf0 0x7f0b40172400 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b401426d0 0x7f0b4015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 --2- 172.21.15.181:0/1219164976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b40160740 0x7f0b4015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 >> 172.21.15.181:0/1219164976 conn(0x7f0b400cf890 msgr2=0x7f0b400615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 shutdown_connections 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.817+0000 7f0b44956700 1 -- 172.21.15.181:0/1219164976 wait complete. 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:28:50.804 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:28:50.805 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:28:50.820 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.953+0000 7fb11e6b6700 1 Processor -- start 2020-11-16T22:28:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.953+0000 7fb11e6b6700 1 -- start start 2020-11-16T22:28:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1180b2190 0x7fb11815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180ce460 con 0x7fb1180b2570 2020-11-16T22:28:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180ce700 con 0x7fb11815ff80 2020-11-16T22:28:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1180ce9a0 con 0x7fb1180b2190 2020-11-16T22:28:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11d6b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1180b2190 0x7fb11815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11deb5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb11815ff80 0x7fb1181603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44546/0 (socket says 172.21.15.181:44546) 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11deb5700 1 -- 172.21.15.181:0/1675964749 learned_addr learned my addr 172.21.15.181:0/1675964749 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653914745 0 0) 0x7fb1180ce700 con 0x7fb11815ff80 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb100003240 con 0x7fb11815ff80 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595369066 0 0) 0x7fb1180ce460 con 0x7fb1180b2570 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11d6b4700 1 -- 172.21.15.181:0/1675964749 >> v1:172.21.15.181:6789/0 conn(0x7fb11815ff80 legacy=0x7fb1181603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11d6b4700 1 -- 172.21.15.181:0/1675964749 >> v1:172.21.15.175:6789/0 conn(0x7fb1180b2570 legacy=0x7fb1180cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11d6b4700 1 -- 172.21.15.181:0/1675964749 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180cec40 con 0x7fb1180b2190 2020-11-16T22:28:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11d6b4700 1 --2- 172.21.15.181:0/1675964749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1180b2190 0x7fb11815de80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb114009c90 tx=0x7fb1140091d0).ready entity=mon.1 client_cookie=49154e3b0733a99 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11400ba70 con 0x7fb1180b2190 2020-11-16T22:28:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11400bbd0 con 0x7fb1180b2190 2020-11-16T22:28:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb107fff700 1 -- 172.21.15.181:0/1675964749 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114011400 con 0x7fb1180b2190 2020-11-16T22:28:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1675964749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1180b2190 msgr2=0x7fb11815de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1675964749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1180b2190 0x7fb11815de80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb114009c90 tx=0x7fb1140091d0).stop 2020-11-16T22:28:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1675964749 shutdown_connections 2020-11-16T22:28:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1675964749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1180b2190 0x7fb11815de80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.957+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1675964749 >> 172.21.15.181:0/1675964749 conn(0x7fb1180cfe90 msgr2=0x7fb1180d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1675964749 shutdown_connections 2020-11-16T22:28:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1675964749 wait complete. 2020-11-16T22:28:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 Processor -- start 2020-11-16T22:28:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- start start 2020-11-16T22:28:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 0x7fb118151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1180b9270 0x7fb118151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb118060f80 con 0x7fb1180b9270 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11816d360 con 0x7fb11815ff80 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11e6b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11816d600 con 0x7fb1180b2570 2020-11-16T22:28:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11d6b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 0x7fb118151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11d6b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38086/0 (socket says 172.21.15.181:38086) 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11d6b4700 1 -- 172.21.15.181:0/1908784461 learned_addr learned my addr 172.21.15.181:0/1908784461 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11deb5700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1180b9270 0x7fb118151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 -- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 msgr2=0x7fb1180b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 -- 172.21.15.181:0/1908784461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1180b9270 msgr2=0x7fb118151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1180b9270 0x7fb118151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11ceb3700 1 -- 172.21.15.181:0/1908784461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11816d8a0 con 0x7fb11815ff80 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.961+0000 7fb11d6b4700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11ceb3700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 0x7fb118151840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb10800c730 tx=0x7fb108007970).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800e490 con 0x7fb11815ff80 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800e5f0 con 0x7fb11815ff80 2020-11-16T22:28:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11816db40 con 0x7fb11815ff80 2020-11-16T22:28:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800e9b0 con 0x7fb11815ff80 2020-11-16T22:28:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11816d010 con 0x7fb11815ff80 2020-11-16T22:28:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb118044f60 con 0x7fb11815ff80 2020-11-16T22:28:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb108010030 con 0x7fb11815ff80 2020-11-16T22:28:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 0x7fb0f0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb10806da80 con 0x7fb11815ff80 2020-11-16T22:28:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11d6b4700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 0x7fb0f0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.965+0000 7fb11d6b4700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 0x7fb0f0057550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fb118064d90 tx=0x7fb114016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:50.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:50.973+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb108043ce0 con 0x7fb11815ff80 2020-11-16T22:28:51.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.173+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb11805c4a0 con 0x7fb11815ff80 2020-11-16T22:28:51.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.173+0000 7fb1057fa700 1 -- 172.21.15.181:0/1908784461 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fb108047300 con 0x7fb11815ff80 2020-11-16T22:28:51.158 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:51.158 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 msgr2=0x7fb0f0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 0x7fb0f0057550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fb118064d90 tx=0x7fb114016000).stop 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 msgr2=0x7fb118151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 0x7fb118151840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb10800c730 tx=0x7fb108007970).stop 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 shutdown_connections 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb0f0055090 0x7fb0f0057550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1180b9270 0x7fb118151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1180b2570 0x7fb1180b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 --2- 172.21.15.181:0/1908784461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb11815ff80 0x7fb118151840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 >> 172.21.15.181:0/1908784461 conn(0x7fb1180cfe90 msgr2=0x7fb11815f870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 shutdown_connections 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.177+0000 7fb11e6b6700 1 -- 172.21.15.181:0/1908784461 wait complete. 2020-11-16T22:28:51.161 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:28:51.172 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.313+0000 7fac22892700 1 Processor -- start 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.313+0000 7fac22892700 1 -- start start 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac1c15d5e0 0x7fac1c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c066480 con 0x7fac1c15d200 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c066ee0 con 0x7fac1c0667b0 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c067180 con 0x7fac1c15d5e0 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac1c15d5e0 0x7fac1c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac20e2f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fac1c0667b0 0x7fac1c0cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44560/0 (socket says 172.21.15.181:44560) 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac1c15d5e0 0x7fac1c0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58486/0 (socket says 172.21.15.181:58486) 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac20e2f700 1 -- 172.21.15.181:0/168620446 learned_addr learned my addr 172.21.15.181:0/168620446 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896985574 0 0) 0x7fac1c066ee0 con 0x7fac1c0667b0 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac08003240 con 0x7fac1c0667b0 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 102014986 0 0) 0x7fac08003240 con 0x7fac1c0667b0 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 -- 172.21.15.181:0/168620446 >> v1:172.21.15.181:6789/0 conn(0x7fac1c0667b0 legacy=0x7fac1c0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 -- 172.21.15.181:0/168620446 >> v1:172.21.15.175:6789/0 conn(0x7fac1c15d200 legacy=0x7fac1c0cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 -- 172.21.15.181:0/168620446 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c067420 con 0x7fac1c15d5e0 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/168620446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c0ce210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fac1000b9a0 tx=0x7fac10006df0).ready entity=mon.1 client_cookie=1d2a1355f60ef86d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100096b0 con 0x7fac1c15d5e0 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10009810 con 0x7fac1c15d5e0 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac1affd700 1 -- 172.21.15.181:0/168620446 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10009b90 con 0x7fac1c15d5e0 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- 172.21.15.181:0/168620446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 msgr2=0x7fac1c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 --2- 172.21.15.181:0/168620446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c0ce210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fac1000b9a0 tx=0x7fac10006df0).stop 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- 172.21.15.181:0/168620446 shutdown_connections 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 --2- 172.21.15.181:0/168620446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c0ce210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- 172.21.15.181:0/168620446 >> 172.21.15.181:0/168620446 conn(0x7fac1c0d4790 msgr2=0x7fac1c0d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- 172.21.15.181:0/168620446 shutdown_connections 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.317+0000 7fac22892700 1 -- 172.21.15.181:0/168620446 wait complete. 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.321+0000 7fac22892700 1 Processor -- start 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.321+0000 7fac22892700 1 -- start start 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac1c0667b0 0x7fac1c165c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 0x7fac1c0b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c16f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c166150 con 0x7fac1c0667b0 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c16fc60 con 0x7fac1c15d5e0 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c16ff00 con 0x7fac1c15d200 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c16f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 0x7fac1c0b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c16f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58494/0 (socket says 172.21.15.181:58494) 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac20e2f700 1 -- 172.21.15.181:0/347214265 learned_addr learned my addr 172.21.15.181:0/347214265 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1bfff700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac1c0667b0 0x7fac1c165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 -- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 msgr2=0x7fac1c16f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c16f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 -- 172.21.15.181:0/347214265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac1c0667b0 msgr2=0x7fac1c165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac1c0667b0 0x7fac1c165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 -- 172.21.15.181:0/347214265 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c1701a0 con 0x7fac1c15d200 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 0x7fac1c0b88a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fac10012000 tx=0x7fac10015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fabfbfff700 1 -- 172.21.15.181:0/347214265 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10011b40 con 0x7fac1c15d200 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fabfbfff700 1 -- 172.21.15.181:0/347214265 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10011ca0 con 0x7fac1c15d200 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fabfbfff700 1 -- 172.21.15.181:0/347214265 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1001e690 con 0x7fac1c15d200 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c165280 con 0x7fac1c15d200 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.325+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c1657c0 con 0x7fac1c15d200 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fabfbfff700 1 -- 172.21.15.181:0/347214265 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fac1001a040 con 0x7fac1c15d200 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fabfbfff700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 0x7fabf4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fac1bfff700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 0x7fabf4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fabfbfff700 1 -- 172.21.15.181:0/347214265 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac1006ee40 con 0x7fac1c15d200 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fac1bfff700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 0x7fabf4057520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fac0c0010a0 tx=0x7fac0c007b70).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fac31323740 1 Processor -- start 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.329+0000 7fac31323740 1 -- start start 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 0xf27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xd5e4b0 con 0xc46bb0 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe12930 con 0xecd470 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xcd26f0 con 0xcb47b0 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 0xf27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38110/0 (socket says 172.21.15.181:38110) 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac1bfff700 1 -- 172.21.15.181:0/4026977194 learned_addr learned my addr 172.21.15.181:0/4026977194 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 -- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 msgr2=0xecbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 -- 172.21.15.181:0/4026977194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 msgr2=0xf0a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf0a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 -- 172.21.15.181:0/4026977194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xccd5a0 con 0xecd470 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac20e2f700 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 0xf27c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fac14004fc0 tx=0x7fac140079f0).ready entity=mon.1 client_cookie=ab3c34c8ce6c984a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fabf97fa700 1 -- 172.21.15.181:0/4026977194 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14001c30 con 0xecd470 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fabf97fa700 1 -- 172.21.15.181:0/4026977194 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14001d90 con 0xecd470 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fabf97fa700 1 -- 172.21.15.181:0/4026977194 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14012350 con 0xecd470 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 msgr2=0xf27c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 0xf27c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fac14004fc0 tx=0x7fac140079f0).stop 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- 172.21.15.181:0/4026977194 shutdown_connections 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xecd470 0xf27c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 --2- 172.21.15.181:0/4026977194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf0a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- 172.21.15.181:0/4026977194 >> 172.21.15.181:0/4026977194 conn(0xe82e40 msgr2=0xf27580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- 172.21.15.181:0/4026977194 shutdown_connections 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- 172.21.15.181:0/4026977194 wait complete. 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 -- start start 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.333+0000 7fac31323740 1 Processor -- start 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc46bb0 0xefa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcb47b0 0xf08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xdff6e0 con 0xcb47b0 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xdff820 con 0xc46bb0 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xe00a90 con 0xecd470 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 10 client.0 ll_register_callbacks cb 0xdf5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38118/0 (socket says 172.21.15.181:38118) 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 -- 172.21.15.181:0/2673430817 learned_addr learned my addr 172.21.15.181:0/2673430817 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac1bfff700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc46bb0 0xefa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcb47b0 0xf08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 -- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc46bb0 msgr2=0xefa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc46bb0 0xefa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 -- 172.21.15.181:0/2673430817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcb47b0 msgr2=0xf08870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcb47b0 0xf08870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 -- 172.21.15.181:0/2673430817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdf0280 con 0xecd470 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac20e2f700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fac14007a70 tx=0x7fac14017000).ready entity=mon.2 client_cookie=c48bbd145228baa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14012e10 con 0xecd470 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac1401bae0 con 0xecd470 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xec7e10 con 0xecd470 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac14020430 con 0xecd470 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 10 client.5036 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0xc25fd0 con 0xecd470 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xec7e10 con 0xecd470 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fac1401d070 con 0xecd470 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 10 client.5036 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 10 client.5036 resolve_mds: resolved ID 'a' to GID 4990 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 20 client.5036 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xf32930 0xef41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 4 client.5036 mds_command: new command op to 4990 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 --> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xcf0c50 con 0xf32930 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac1bfff700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xf32930 0xef41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fabf27fc700 10 client.5036 ms_handle_connect on v2:172.21.15.181:6836/4047172210 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.337+0000 7fac1bfff700 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xf32930 0xef41e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=991cf53293da20b2 server_cookie=b8cc9b7d1eb8f587 in_seq=0 out_seq=0 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fabf27fc700 1 -- 172.21.15.181:0/2673430817 <== mds.2 v2:172.21.15.181:6836/4047172210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xcf0c50 con 0xf32930 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fabf27fc700 10 client.5036 handle_command_reply: tid=0 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 client.5036 shutdown 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 msgr2=0xf08d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fac14007a70 tx=0x7fac14017000).stop 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 shutdown_connections 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xf32930 0xef41e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xecd470 0xf08d60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc46bb0 0xefa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 --2- 172.21.15.181:0/2673430817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcb47b0 0xf08870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 >> 172.21.15.181:0/2673430817 conn(0xe82e40 msgr2=0xf0a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fabf27fc700 0 client.5036 ms_handle_reset on v2:172.21.15.181:6836/4047172210 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 shutdown_connections 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 1 -- 172.21.15.181:0/2673430817 wait complete. 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.341+0000 7fac31323740 20 client.5036 trim_cache size 0 max 16384 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 Processor -- start 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 -- start start 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf08d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xdd7e40 con 0xc46bb0 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xef7f80 con 0xe82e40 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xef81b0 con 0xcb47b0 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac20e2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58512/0 (socket says 172.21.15.181:58512) 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 -- 172.21.15.181:0/3818552456 learned_addr learned my addr 172.21.15.181:0/3818552456 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf08d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 -- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 msgr2=0xecaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 -- 172.21.15.181:0/3818552456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 msgr2=0xf08d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf08d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 -- 172.21.15.181:0/3818552456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80081d0 con 0xe82e40 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf08d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fac0c015c20 tx=0x7fac0c018040).ready entity=mon.1 client_cookie=503a16e16a815a93 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fabf9ffb700 1 -- 172.21.15.181:0/3818552456 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c020900 con 0xe82e40 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fabf9ffb700 1 -- 172.21.15.181:0/3818552456 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c020a60 con 0xe82e40 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fabf9ffb700 1 -- 172.21.15.181:0/3818552456 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c020de0 con 0xe82e40 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 -- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 msgr2=0xf08870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.361+0000 7fac31323740 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fac0c015c20 tx=0x7fac0c018040).stop 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 -- 172.21.15.181:0/3818552456 shutdown_connections 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcb47b0 0xecaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe82e40 0xf08870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 --2- 172.21.15.181:0/3818552456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc46bb0 0xf08d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 -- 172.21.15.181:0/3818552456 >> 172.21.15.181:0/3818552456 conn(0x7fabe8002320 msgr2=0xecbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 -- 172.21.15.181:0/3818552456 shutdown_connections 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 -- 172.21.15.181:0/3818552456 wait complete. 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 -- start start 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 Processor -- start 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.365+0000 7fac31323740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 0xf77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf85a20 con 0xe82e40 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xece040 con 0xcb47b0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xece2e0 con 0xc46bb0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 10 client.0 ll_register_callbacks cb 0xf0c4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38128/0 (socket says 172.21.15.181:38128) 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58520/0 (socket says 172.21.15.181:58520) 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 -- 172.21.15.181:0/3795143867 learned_addr learned my addr 172.21.15.181:0/3795143867 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac20e2f700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 0xf77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 -- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 msgr2=0xf26200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 -- 172.21.15.181:0/3795143867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 msgr2=0xf77140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 0xf77140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 -- 172.21.15.181:0/3795143867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc64720 con 0xcb47b0 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac20e2f700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 0xf77140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1b7fe700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fac10048d40 tx=0x7fac10029040).ready entity=mon.1 client_cookie=458cb23099dba175 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10048150 con 0xcb47b0 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac14001d90 con 0xcb47b0 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac1001ea80 con 0xcb47b0 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1001ee50 con 0xcb47b0 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 10 client.5299 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0xc6fe00 con 0xcb47b0 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14001d90 con 0xcb47b0 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fac1003b020 con 0xcb47b0 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 10 client.5299 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 10 client.5299 resolve_mds: resolved ID 'a' to GID 4990 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 20 client.5299 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xecd470 0xf35130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 4 client.5299 mds_command: new command op to 4990 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 --> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0xcf7f50 con 0xecd470 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xecd470 0xf35130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fabf27fc700 10 client.5299 ms_handle_connect on v2:172.21.15.181:6836/4047172210 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.369+0000 7fac1bfff700 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xecd470 0xf35130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=3f0f3ed3078f279f server_cookie=231ff57901da874e in_seq=0 out_seq=0 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fabf27fc700 1 -- 172.21.15.181:0/3795143867 <== mds.2 v2:172.21.15.181:6836/4047172210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0xcf7f50 con 0xecd470 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fabf27fc700 10 client.5299 handle_command_reply: tid=0 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 client.5299 shutdown 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 msgr2=0xf76c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fac10048d40 tx=0x7fac10029040).stop 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 shutdown_connections 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:6836/4047172210,v1:172.21.15.181:6837/4047172210] conn(0xecd470 0xf35130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe82e40 0xf77140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc46bb0 0xf26200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 --2- 172.21.15.181:0/3795143867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcb47b0 0xf76c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 >> 172.21.15.181:0/3795143867 conn(0x7fabe8002320 msgr2=0xef41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fabf27fc700 0 client.5299 ms_handle_reset on v2:172.21.15.181:6836/4047172210 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 shutdown_connections 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 1 -- 172.21.15.181:0/3795143867 wait complete. 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac31323740 20 client.5299 trim_cache size 0 max 16384 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 msgr2=0x7fabf4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 0x7fabf4057520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fac0c0010a0 tx=0x7fac0c007b70).stop 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 msgr2=0x7fac1c0b88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 0x7fac1c0b88a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fac10012000 tx=0x7fac10015000).stop 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 shutdown_connections 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabf4055060 0x7fabf4057520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac1c15d5e0 0x7fac1c16f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac1c0667b0 0x7fac1c165c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 --2- 172.21.15.181:0/347214265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac1c15d200 0x7fac1c0b88a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 >> 172.21.15.181:0/347214265 conn(0x7fac1c0d4790 msgr2=0x7fac1c15b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 shutdown_connections 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:51.393+0000 7fac22892700 1 -- 172.21.15.181:0/347214265 wait complete. 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:28:51.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:28:51.395 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:28:51.453 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/ 2020-11-16T22:28:51.453 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4 2020-11-16T22:28:51.453 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5 2020-11-16T22:28:51.453 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:28:51.454 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:28:51.454 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:28:51.459 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:51.459 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:51.459 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:51.460 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:51.460 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:28:51.460 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:51.571 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2020-11-16T22:28:51.761 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:51.762 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:51.762 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:51.762 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:51.762 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:28:51.763 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:51.873 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2020-11-16T22:28:52.064 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:52.065 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:52.065 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:52.065 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:52.066 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:28:52.066 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:52.221 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2020-11-16T22:28:52.410 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:52.410 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:52.410 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:52.411 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:52.411 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:28:52.411 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:52.524 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627781 2020-11-16T22:28:52.715 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:52.716 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:52.716 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:52.717 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:52.717 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:28:52.717 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:52.824 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627782 2020-11-16T22:28:53.013 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:28:53.013 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:28:53.013 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:28:53.014 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:28:53.014 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:28:53.014 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:28:53.165 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627783 2020-11-16T22:28:53.356 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:53.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.505+0000 7f4df359e700 1 Processor -- start 2020-11-16T22:28:53.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.505+0000 7f4df359e700 1 -- start start 2020-11-16T22:28:53.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.505+0000 7f4df359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df4060800 0x7f4df40660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.505+0000 7f4df359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40cfbf0 con 0x7f4df40cf390 2020-11-16T22:28:53.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.505+0000 7f4df359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df4065190 con 0x7f4df40cf770 2020-11-16T22:28:53.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df4065430 con 0x7f4df4060800 2020-11-16T22:28:53.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df4060800 0x7f4df40660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df4060800 0x7f4df40660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58530/0 (socket says 172.21.15.181:58530) 2020-11-16T22:28:53.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 -- 172.21.15.181:0/1276196130 learned_addr learned my addr 172.21.15.181:0/1276196130 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 -- 172.21.15.181:0/1276196130 >> v1:172.21.15.181:6789/0 conn(0x7f4df40cf770 legacy=0x7f4df40600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 -- 172.21.15.181:0/1276196130 >> v1:172.21.15.175:6789/0 conn(0x7f4df40cf390 legacy=0x7f4df4163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df159a700 1 -- 172.21.15.181:0/1276196130 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784027087 0 0) 0x7f4df4065190 con 0x7f4df40cf770 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 -- 172.21.15.181:0/1276196130 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40656d0 con 0x7f4df4060800 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df2d9d700 1 --2- 172.21.15.181:0/1276196130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df4060800 0x7f4df40660b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f4dec006fd0 tx=0x7f4dec006eb0).ready entity=mon.1 client_cookie=e19744d61df72165 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df159a700 1 -- 172.21.15.181:0/1276196130 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0098d0 con 0x7f4df4060800 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df159a700 1 -- 172.21.15.181:0/1276196130 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec009a30 con 0x7f4df4060800 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df159a700 1 -- 172.21.15.181:0/1276196130 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec009db0 con 0x7f4df4060800 2020-11-16T22:28:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- 172.21.15.181:0/1276196130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df4060800 msgr2=0x7f4df40660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 --2- 172.21.15.181:0/1276196130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df4060800 0x7f4df40660b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f4dec006fd0 tx=0x7f4dec006eb0).stop 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- 172.21.15.181:0/1276196130 shutdown_connections 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 --2- 172.21.15.181:0/1276196130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df4060800 0x7f4df40660b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- 172.21.15.181:0/1276196130 >> 172.21.15.181:0/1276196130 conn(0x7f4df40cfe90 msgr2=0x7f4df40d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- 172.21.15.181:0/1276196130 shutdown_connections 2020-11-16T22:28:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- 172.21.15.181:0/1276196130 wait complete. 2020-11-16T22:28:53.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 Processor -- start 2020-11-16T22:28:53.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.509+0000 7f4df359e700 1 -- start start 2020-11-16T22:28:53.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df4060800 0x7f4df40b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4172450 con 0x7f4df4060800 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4173300 con 0x7f4df40cf770 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40b8c60 con 0x7f4df40cf390 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58536/0 (socket says 172.21.15.181:58536) 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38144/0 (socket says 172.21.15.181:38144) 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 -- 172.21.15.181:0/889552151 learned_addr learned my addr 172.21.15.181:0/889552151 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df259c700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df4060800 0x7f4df40b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 -- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 msgr2=0x7f4df4066170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 -- 172.21.15.181:0/889552151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df4060800 msgr2=0x7f4df40b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df4060800 0x7f4df40b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 -- 172.21.15.181:0/889552151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4172680 con 0x7f4df40cf770 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df1d9b700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df2d9d700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f4dec011dd0 tx=0x7f4dec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0098d0 con 0x7f4df40cf770 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec01d5d0 con 0x7f4df40cf770 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4172920 con 0x7f4df40cf770 2020-11-16T22:28:53.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec01d950 con 0x7f4df40cf770 2020-11-16T22:28:53.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df40b92e0 con 0x7f4df40cf770 2020-11-16T22:28:53.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.513+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df4044f60 con 0x7f4df40cf770 2020-11-16T22:28:53.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.517+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4dec025030 con 0x7f4df40cf770 2020-11-16T22:28:53.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.517+0000 7f4de27fc700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 0x7f4dc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.517+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4dec06f6b0 con 0x7f4df40cf770 2020-11-16T22:28:53.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.517+0000 7f4df259c700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 0x7f4dc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.517+0000 7f4df259c700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 0x7f4dc8057520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f4de4004250 tx=0x7f4de4007f10).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.521+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4dec0459c0 con 0x7f4df40cf770 2020-11-16T22:28:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.725+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4df405c4a0 con 0x7f4df40cf770 2020-11-16T22:28:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.725+0000 7f4de27fc700 1 -- 172.21.15.181:0/889552151 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f4dec02e250 con 0x7f4df40cf770 2020-11-16T22:28:53.710 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 msgr2=0x7f4dc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 0x7f4dc8057520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f4de4004250 tx=0x7f4de4007f10).stop 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 msgr2=0x7f4df4066660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f4dec011dd0 tx=0x7f4dec015000).stop 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 shutdown_connections 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4dc8055060 0x7f4dc8057520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df40cf770 0x7f4df4066660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df4060800 0x7f4df40b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 --2- 172.21.15.181:0/889552151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4df40cf390 0x7f4df4066170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 >> 172.21.15.181:0/889552151 conn(0x7f4df40cfe90 msgr2=0x7f4df40600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 shutdown_connections 2020-11-16T22:28:53.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.729+0000 7f4df359e700 1 -- 172.21.15.181:0/889552151 wait complete. 2020-11-16T22:28:53.714 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:28:53.724 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e tag path /d1/d2/d3 tag123 2020-11-16T22:28:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 Processor -- start 2020-11-16T22:28:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 -- start start 2020-11-16T22:28:53.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f94400c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400ccbf0 con 0x7f944015d740 2020-11-16T22:28:53.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944015dba0 con 0x7f944015d360 2020-11-16T22:28:53.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f9444eca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400cbe30 con 0x7f94400b4720 2020-11-16T22:28:53.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f943f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f944015d360 0x7f94400cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44612/0 (socket says 172.21.15.181:44612) 2020-11-16T22:28:53.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f943f7fe700 1 -- 172.21.15.181:0/2921433548 learned_addr learned my addr 172.21.15.181:0/2921433548 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:53.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.861+0000 7f943ffff700 1 --2- 172.21.15.181:0/2921433548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f94400c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943e7fc700 1 -- 172.21.15.181:0/2921433548 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439079509 0 0) 0x7f944015dba0 con 0x7f944015d360 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943ffff700 1 -- 172.21.15.181:0/2921433548 >> v1:172.21.15.181:6789/0 conn(0x7f944015d360 legacy=0x7f94400cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943ffff700 1 -- 172.21.15.181:0/2921433548 >> v1:172.21.15.175:6789/0 conn(0x7f944015d740 legacy=0x7f94400b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943ffff700 1 -- 172.21.15.181:0/2921433548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400cc0d0 con 0x7f94400b4720 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943ffff700 1 --2- 172.21.15.181:0/2921433548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f94400c8590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9438009cc0 tx=0x7f94380091d0).ready entity=mon.2 client_cookie=9a0a288739af7738 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943e7fc700 1 -- 172.21.15.181:0/2921433548 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943800bbc0 con 0x7f94400b4720 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943e7fc700 1 -- 172.21.15.181:0/2921433548 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943800bd20 con 0x7f94400b4720 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f943e7fc700 1 -- 172.21.15.181:0/2921433548 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438011610 con 0x7f94400b4720 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- 172.21.15.181:0/2921433548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 msgr2=0x7f94400c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 --2- 172.21.15.181:0/2921433548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f94400c8590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9438009cc0 tx=0x7f94380091d0).stop 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- 172.21.15.181:0/2921433548 shutdown_connections 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 --2- 172.21.15.181:0/2921433548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f94400c8590 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- 172.21.15.181:0/2921433548 >> 172.21.15.181:0/2921433548 conn(0x7f9440060bc0 msgr2=0x7f94400d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- 172.21.15.181:0/2921433548 shutdown_connections 2020-11-16T22:28:53.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- 172.21.15.181:0/2921433548 wait complete. 2020-11-16T22:28:53.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 Processor -- start 2020-11-16T22:28:53.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.865+0000 7f9444eca700 1 -- start start 2020-11-16T22:28:53.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f944015d360 0x7f9440156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944015d740 0x7f944016cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944016e450 con 0x7f944015d740 2020-11-16T22:28:53.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944016db90 con 0x7f944015d360 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944016de30 con 0x7f94400b4720 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38156/0 (socket says 172.21.15.181:38156) 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 -- 172.21.15.181:0/564952469 learned_addr learned my addr 172.21.15.181:0/564952469 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943effd700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f944015d360 0x7f9440156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943ffff700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944015d740 0x7f944016cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 -- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f944015d360 msgr2=0x7f9440156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f944015d360 0x7f9440156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 -- 172.21.15.181:0/564952469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944015d740 msgr2=0x7f944016cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944015d740 0x7f944016cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 -- 172.21.15.181:0/564952469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944016e6b0 con 0x7f94400b4720 2020-11-16T22:28:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f943f7fe700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f943000c6c0 tx=0x7f9430007a80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f941f7fe700 1 -- 172.21.15.181:0/564952469 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300072e0 con 0x7f94400b4720 2020-11-16T22:28:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f941f7fe700 1 -- 172.21.15.181:0/564952469 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430007440 con 0x7f94400b4720 2020-11-16T22:28:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944016e950 con 0x7f94400b4720 2020-11-16T22:28:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f941f7fe700 1 -- 172.21.15.181:0/564952469 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430005800 con 0x7f94400b4720 2020-11-16T22:28:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.869+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944016d510 con 0x7f94400b4720 2020-11-16T22:28:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.873+0000 7f941f7fe700 1 -- 172.21.15.181:0/564952469 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f943001a060 con 0x7f94400b4720 2020-11-16T22:28:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.873+0000 7f941f7fe700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 0x7f9418057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.873+0000 7f943effd700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 0x7f9418057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.873+0000 7f941f7fe700 1 -- 172.21.15.181:0/564952469 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f943006d5b0 con 0x7f94400b4720 2020-11-16T22:28:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.873+0000 7f943effd700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 0x7f9418057520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f94400d1d50 tx=0x7f9434007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 Processor -- start 2020-11-16T22:28:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 -- start start 2020-11-16T22:28:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 0xfb9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfb7040 0xfb0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xe42520 con 0xd98820 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xef69a0 con 0xfb7040 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xdb6760 con 0xd2ac20 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38166/0 (socket says 172.21.15.181:38166) 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 -- 172.21.15.181:0/2511899703 learned_addr learned my addr 172.21.15.181:0/2511899703 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943ffff700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfb7040 0xfb0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943f7fe700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 0xfb9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 -- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfb7040 msgr2=0xfb0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfb7040 0xfb0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 -- 172.21.15.181:0/2511899703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 msgr2=0xfb9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 0xfb9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 -- 172.21.15.181:0/2511899703 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xdb1610 con 0xd2ac20 2020-11-16T22:28:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943f7fe700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 0xfb9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.877+0000 7f943effd700 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f9434017a80 tx=0x7f943400a000).ready entity=mon.2 client_cookie=8f95a73c15d86a2b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f941cff9700 1 -- 172.21.15.181:0/2511899703 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434017240 con 0xd2ac20 2020-11-16T22:28:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f941cff9700 1 -- 172.21.15.181:0/2511899703 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94340157e0 con 0xd2ac20 2020-11-16T22:28:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f941cff9700 1 -- 172.21.15.181:0/2511899703 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434015b60 con 0xd2ac20 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 msgr2=0xfb76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f9434017a80 tx=0x7f943400a000).stop 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/2511899703 shutdown_connections 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfb7040 0xfb0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd98820 0xfb9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- 172.21.15.181:0/2511899703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2ac20 0xfb76e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/2511899703 >> 172.21.15.181:0/2511899703 conn(0xf66eb0 msgr2=0xfb3410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/2511899703 shutdown_connections 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/2511899703 wait complete. 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- start start 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 Processor -- start 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0xfd4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x100a2f0 0x100a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xee3750 con 0x100a2f0 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xee3890 con 0xd98820 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xee4b00 con 0xfb7040 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 10 client.0 ll_register_callbacks cb 0xed9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58560/0 (socket says 172.21.15.181:58560) 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 -- 172.21.15.181:0/156430107 learned_addr learned my addr 172.21.15.181:0/156430107 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0xfd4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943ffff700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x100a2f0 0x100a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 -- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 msgr2=0xfd4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0xfd4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 -- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x100a2f0 msgr2=0x100a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x100a2f0 0x100a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 -- 172.21.15.181:0/156430107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xed42f0 con 0xd98820 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f943f7fe700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f94300439d0 tx=0x7f9430010000).ready entity=mon.1 client_cookie=2695199c7335279d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430043dc0 con 0xd98820 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f943006d2e0 con 0xd98820 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf97730 con 0xd98820 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94300434d0 con 0xd98820 2020-11-16T22:28:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 10 client.5320 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0xd0a040 con 0xd98820 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.881+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf97730 con 0xd98820 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f9430046430 con 0xd98820 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 10 client.5320 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 10 client.5320 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 20 client.5320 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0xfb8b00 0xfb8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 4 client.5320 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:28:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xdd4cc0 con 0xfb8b00 2020-11-16T22:28:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f943effd700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0xfb8b00 0xfb8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9415ffb700 10 client.5320 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f943effd700 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0xfb8b00 0xfb8ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5a44299e79018681 server_cookie=7ccbae184cfb356 in_seq=0 out_seq=0 2020-11-16T22:28:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9415ffb700 1 -- 172.21.15.181:0/156430107 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xdd4cc0 con 0xfb8b00 2020-11-16T22:28:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9415ffb700 10 client.5320 handle_command_reply: tid=0 2020-11-16T22:28:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 client.5320 shutdown 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 msgr2=0x1019ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f94300439d0 tx=0x7f9430010000).stop 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 shutdown_connections 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0xfb8b00 0xfb8ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x100a2f0 0x100a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0x1019ad0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 --2- 172.21.15.181:0/156430107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0xfd4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9415ffb700 0 client.5320 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:53.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 >> 172.21.15.181:0/156430107 conn(0xf66eb0 msgr2=0xfecae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 shutdown_connections 2020-11-16T22:28:53.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 1 -- 172.21.15.181:0/156430107 wait complete. 2020-11-16T22:28:53.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.885+0000 7f9454bbd740 20 client.5320 trim_cache size 0 max 16384 2020-11-16T22:28:53.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 Processor -- start 2020-11-16T22:28:53.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- start start 2020-11-16T22:28:53.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 0xfb5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb7040 0xfb64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xebbeb0 con 0xfb7040 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xfac4f0 con 0xf66eb0 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1002120 con 0xd98820 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58568/0 (socket says 172.21.15.181:58568) 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 -- 172.21.15.181:0/1765798529 learned_addr learned my addr 172.21.15.181:0/1765798529 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943effd700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 0xfb5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 -- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 msgr2=0xfb5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 0xfb5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 -- 172.21.15.181:0/1765798529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb7040 msgr2=0xfb64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943ffff700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb7040 0xfb64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb7040 0xfb64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 -- 172.21.15.181:0/1765798529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0081d0 con 0xf66eb0 2020-11-16T22:28:53.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943effd700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 0xfb5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:53.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f943f7fe700 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f9430047400 tx=0x7f943004d000).ready entity=mon.1 client_cookie=32b53883644e57d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f941d7fa700 1 -- 172.21.15.181:0/1765798529 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300467f0 con 0xf66eb0 2020-11-16T22:28:53.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f941d7fa700 1 -- 172.21.15.181:0/1765798529 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430047e90 con 0xf66eb0 2020-11-16T22:28:53.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f941d7fa700 1 -- 172.21.15.181:0/1765798529 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430043810 con 0xf66eb0 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 msgr2=0x1002a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f9430047400 tx=0x7f943004d000).stop 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- 172.21.15.181:0/1765798529 shutdown_connections 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb7040 0xfb64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf66eb0 0x1002a30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1765798529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd98820 0xfb5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- 172.21.15.181:0/1765798529 >> 172.21.15.181:0/1765798529 conn(0x7f940c002320 msgr2=0x1000b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- 172.21.15.181:0/1765798529 shutdown_connections 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- 172.21.15.181:0/1765798529 wait complete. 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 -- start start 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.909+0000 7f9454bbd740 1 Processor -- start 2020-11-16T22:28:53.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0xfb82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 0x1008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0x1000db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x10012a0 con 0xf66eb0 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x106e3b0 con 0xd98820 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1071c20 con 0xfb7040 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 10 client.0 ll_register_callbacks cb 0xfb7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0xfb82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0x1000db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0xfb82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58574/0 (socket says 172.21.15.181:58574) 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943f7fe700 1 -- 172.21.15.181:0/1890875476 learned_addr learned my addr 172.21.15.181:0/1890875476 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943effd700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 0x1008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 -- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 msgr2=0xfb82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0xfb82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 -- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 msgr2=0x1008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 0x1008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 -- 172.21.15.181:0/1890875476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd48790 con 0xfb7040 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943effd700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 0x1008650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943ffff700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0x1000db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9438009cc0 tx=0x7f9438016000).ready entity=mon.2 client_cookie=901faf6adfeed54f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943801d600 con 0xfb7040 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f943801f870 con 0xfb7040 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94340157e0 con 0xfb7040 2020-11-16T22:28:53.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f943801fc40 con 0xfb7040 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 10 client.5063 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0xd53e70 con 0xfb7040 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943802e020 con 0xfb7040 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f943801fda0 con 0xfb7040 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 10 client.5063 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 10 client.5063 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:53.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 20 client.5063 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x100a2f0 0x106e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 4 client.5063 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",t,a,g, ,p,a,t,h,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",t,a,g,",:, ,",t,a,g,1,2,3,",}] 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " t a g p a t h " , " p a t h " : " / d 1 / d 2 / d 3 " , " t a g " : " t a g 1 2 3 " }) v1 -- 0xddbfc0 con 0x100a2f0 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943f7fe700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x100a2f0 0x106e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f9415ffb700 10 client.5063 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:53.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.913+0000 7f943f7fe700 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x100a2f0 0x106e7e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cbea5804dccb621a server_cookie=478cfef4d53a202a in_seq=0 out_seq=0 2020-11-16T22:28:53.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9415ffb700 1 -- 172.21.15.181:0/1890875476 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+80 (crc 0 0 0) 0xddbfc0 con 0x100a2f0 2020-11-16T22:28:53.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9415ffb700 10 client.5063 handle_command_reply: tid=0 2020-11-16T22:28:53.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 client.5063 shutdown 2020-11-16T22:28:53.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 msgr2=0x1000db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0x1000db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9438009cc0 tx=0x7f9438016000).stop 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 shutdown_connections 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x100a2f0 0x106e7e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9415ffb700 0 client.5063 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb7040 0x1000db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd98820 0xfb82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 --2- 172.21.15.181:0/1890875476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf66eb0 0x1008650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 >> 172.21.15.181:0/1890875476 conn(0x7f940c002320 msgr2=0x1002820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 shutdown_connections 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 1 -- 172.21.15.181:0/1890875476 wait complete. 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9454bbd740 20 client.5063 trim_cache size 0 max 16384 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 msgr2=0x7f9418057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 0x7f9418057520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f94400d1d50 tx=0x7f9434007b00).stop 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 msgr2=0x7f9440156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f943000c6c0 tx=0x7f9430007a80).stop 2020-11-16T22:28:53.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 shutdown_connections 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9418055060 0x7f9418057520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944015d740 0x7f944016cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94400b4720 0x7f9440156430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 --2- 172.21.15.181:0/564952469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f944015d360 0x7f9440156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.917+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 >> 172.21.15.181:0/564952469 conn(0x7f9440060bc0 msgr2=0x7f94400d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.921+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 shutdown_connections 2020-11-16T22:28:53.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:53.921+0000 7f9444eca700 1 -- 172.21.15.181:0/564952469 wait complete. 2020-11-16T22:28:53.911 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:28:53.911 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2020-11-16T22:28:53.911 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "tag123", 2020-11-16T22:28:53.911 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2020-11-16T22:28:53.911 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:28:53.920 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:28:54.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 Processor -- start 2020-11-16T22:28:54.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 -- start start 2020-11-16T22:28:54.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85bc163a50 0x7f85bc163e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 0x7f85bc0813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc061900 con 0x7f85bc081890 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc061bd0 con 0x7f85bc163a50 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85c1b33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc061e70 con 0x7f85bc062af0 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 0x7f85bc0813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85bbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f85bc081890 0x7f85bc0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44656/0 (socket says 172.21.15.181:44656) 2020-11-16T22:28:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85bbfff700 1 -- 172.21.15.181:0/4173210881 learned_addr learned my addr 172.21.15.181:0/4173210881 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:28:54.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/4173210881 >> v2:172.21.15.175:3300/0 conn(0x7f85bc163a50 0x7f85bc163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 -- 172.21.15.181:0/4173210881 >> v2:172.21.15.175:3300/0 conn(0x7f85bc163a50 msgr2=0x7f85bc163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 --2- 172.21.15.181:0/4173210881 >> v2:172.21.15.175:3300/0 conn(0x7f85bc163a50 0x7f85bc163e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 -- 172.21.15.181:0/4173210881 >> v1:172.21.15.181:6789/0 conn(0x7f85bc081890 legacy=0x7f85bc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85ba7fc700 1 -- 172.21.15.181:0/4173210881 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231062532 0 0) 0x7f85bc061900 con 0x7f85bc081890 2020-11-16T22:28:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 -- 172.21.15.181:0/4173210881 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc062110 con 0x7f85bc062af0 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85baffd700 1 --2- 172.21.15.181:0/4173210881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 0x7f85bc0813a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f85b4009c90 tx=0x7f85b40091d0).ready entity=mon.2 client_cookie=c0d0d10e975a2ff6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85ba7fc700 1 -- 172.21.15.181:0/4173210881 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400ba70 con 0x7f85bc062af0 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85ba7fc700 1 -- 172.21.15.181:0/4173210881 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b400bbd0 con 0x7f85bc062af0 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.057+0000 7f85ba7fc700 1 -- 172.21.15.181:0/4173210881 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4011400 con 0x7f85bc062af0 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- 172.21.15.181:0/4173210881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 msgr2=0x7f85bc0813a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 --2- 172.21.15.181:0/4173210881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 0x7f85bc0813a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f85b4009c90 tx=0x7f85b40091d0).stop 2020-11-16T22:28:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- 172.21.15.181:0/4173210881 shutdown_connections 2020-11-16T22:28:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 --2- 172.21.15.181:0/4173210881 >> v2:172.21.15.175:3300/0 conn(0x7f85bc163a50 0x7f85bc163e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 --2- 172.21.15.181:0/4173210881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc062af0 0x7f85bc0813a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- 172.21.15.181:0/4173210881 >> 172.21.15.181:0/4173210881 conn(0x7f85bc0d69a0 msgr2=0x7f85bc0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- 172.21.15.181:0/4173210881 shutdown_connections 2020-11-16T22:28:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- 172.21.15.181:0/4173210881 wait complete. 2020-11-16T22:28:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 Processor -- start 2020-11-16T22:28:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 -- start start 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.061+0000 7f85c1b33700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc062af0 0x7f85bc1624f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 0x7f85bc164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1779c0 con 0x7f85bc062af0 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1736e0 con 0x7f85bc081890 2020-11-16T22:28:54.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc173980 con 0x7f85bc163a50 2020-11-16T22:28:54.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 0x7f85bc164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58590/0 (socket says 172.21.15.181:58590) 2020-11-16T22:28:54.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85baffd700 1 -- 172.21.15.181:0/2702736178 learned_addr learned my addr 172.21.15.181:0/2702736178 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:54.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc062af0 0x7f85bc1624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 -- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 msgr2=0x7f85bc1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 -- 172.21.15.181:0/2702736178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc062af0 msgr2=0x7f85bc1624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc062af0 0x7f85bc1624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 -- 172.21.15.181:0/2702736178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc173c20 con 0x7f85bc163a50 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85baffd700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85bbfff700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 0x7f85bc164cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f85a400c730 tx=0x7f85a40078e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40072e0 con 0x7f85bc163a50 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4007440 con 0x7f85bc163a50 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc173ec0 con 0x7f85bc163a50 2020-11-16T22:28:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005820 con 0x7f85bc163a50 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc1651c0 con 0x7f85bc163a50 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc174520 con 0x7f85bc163a50 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f85a401a060 con 0x7f85bc163a50 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.065+0000 7f85b37fe700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.069+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85a406d010 con 0x7f85bc163a50 2020-11-16T22:28:54.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.069+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.069+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f85bc0a7620 tx=0x7f85ac008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.077+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f85a4043270 con 0x7f85bc163a50 2020-11-16T22:28:54.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85bc069eb0 con 0x7f85bc163a50 2020-11-16T22:28:54.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85b37fe700 1 -- 172.21.15.181:0/2702736178 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f85a4046890 con 0x7f85bc163a50 2020-11-16T22:28:54.257 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:28:54.257 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 msgr2=0x7f8590057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f85bc0a7620 tx=0x7f85ac008040).stop 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 msgr2=0x7f85bc164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 0x7f85bc164cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f85a400c730 tx=0x7f85a40078e0).stop 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 shutdown_connections 2020-11-16T22:28:54.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc163a50 0x7f85bc164cd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc062af0 0x7f85bc1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 --2- 172.21.15.181:0/2702736178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc081890 0x7f85bc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.273+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 >> 172.21.15.181:0/2702736178 conn(0x7f85bc0d69a0 msgr2=0x7f85bc081160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.277+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 shutdown_connections 2020-11-16T22:28:54.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.277+0000 7f85c1b33700 1 -- 172.21.15.181:0/2702736178 wait complete. 2020-11-16T22:28:54.261 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:28:54.271 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:28:54.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.417+0000 7f060ac0d700 1 Processor -- start 2020-11-16T22:28:54.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.417+0000 7f060ac0d700 1 -- start start 2020-11-16T22:28:54.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0604167ac0 0x7f0604167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06041672c0 0x7f06040b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0604168390 con 0x7f06040b3220 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040b5f00 con 0x7f0604167ac0 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0604167730 con 0x7f06041672c0 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06041672c0 0x7f06040b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06041672c0 0x7f06040b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58600/0 (socket says 172.21.15.181:58600) 2020-11-16T22:28:54.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 -- 172.21.15.181:0/2453702592 learned_addr learned my addr 172.21.15.181:0/2453702592 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:54.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06089a9700 1 --2- 172.21.15.181:0/2453702592 >> v2:172.21.15.175:3300/0 conn(0x7f0604167ac0 0x7f0604167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 -- 172.21.15.181:0/2453702592 >> v2:172.21.15.175:3300/0 conn(0x7f0604167ac0 msgr2=0x7f0604167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:54.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 --2- 172.21.15.181:0/2453702592 >> v2:172.21.15.175:3300/0 conn(0x7f0604167ac0 0x7f0604167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 -- 172.21.15.181:0/2453702592 >> v1:172.21.15.175:6789/0 conn(0x7f06040b3220 legacy=0x7f0604166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 -- 172.21.15.181:0/2453702592 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040b2060 con 0x7f06041672c0 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06091aa700 1 --2- 172.21.15.181:0/2453702592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06041672c0 0x7f06040b36a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f06040ceff0 tx=0x7f05fc009290).ready entity=mon.1 client_cookie=dad8164955e17eb7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06037fe700 1 -- 172.21.15.181:0/2453702592 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00bbc0 con 0x7f06041672c0 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06037fe700 1 -- 172.21.15.181:0/2453702592 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00bd20 con 0x7f06041672c0 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f06037fe700 1 -- 172.21.15.181:0/2453702592 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc011610 con 0x7f06041672c0 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- 172.21.15.181:0/2453702592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06041672c0 msgr2=0x7f06040b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 --2- 172.21.15.181:0/2453702592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06041672c0 0x7f06040b36a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f06040ceff0 tx=0x7f05fc009290).stop 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- 172.21.15.181:0/2453702592 shutdown_connections 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 --2- 172.21.15.181:0/2453702592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06041672c0 0x7f06040b36a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 --2- 172.21.15.181:0/2453702592 >> v2:172.21.15.175:3300/0 conn(0x7f0604167ac0 0x7f0604167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- 172.21.15.181:0/2453702592 >> 172.21.15.181:0/2453702592 conn(0x7f0604060bc0 msgr2=0x7f06040d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- 172.21.15.181:0/2453702592 shutdown_connections 2020-11-16T22:28:54.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- 172.21.15.181:0/2453702592 wait complete. 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 Processor -- start 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.421+0000 7f060ac0d700 1 -- start start 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06040b3220 0x7f06040cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 0x7f0604161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060414cde0 con 0x7f06040b3220 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040cd4a0 con 0x7f0604167ac0 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06040cd740 con 0x7f06041672c0 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0603fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 0x7f0604161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58606/0 (socket says 172.21.15.181:58606) 2020-11-16T22:28:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 -- 172.21.15.181:0/616949764 learned_addr learned my addr 172.21.15.181:0/616949764 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0603fff700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 0x7f0604161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38212/0 (socket says 172.21.15.181:38212) 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06089a9700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06040b3220 0x7f06040cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 -- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 msgr2=0x7f0604161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 0x7f0604161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 -- 172.21.15.181:0/616949764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06040b3220 msgr2=0x7f06040cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06040b3220 0x7f06040cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 -- 172.21.15.181:0/616949764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040cd9e0 con 0x7f0604167ac0 2020-11-16T22:28:54.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f06091aa700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0604168990 tx=0x7f05fc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0600ff9700 1 -- 172.21.15.181:0/616949764 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00bbc0 con 0x7f0604167ac0 2020-11-16T22:28:54.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0600ff9700 1 -- 172.21.15.181:0/616949764 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00be90 con 0x7f0604167ac0 2020-11-16T22:28:54.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06040cdc80 con 0x7f0604167ac0 2020-11-16T22:28:54.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0600ff9700 1 -- 172.21.15.181:0/616949764 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc010cb0 con 0x7f0604167ac0 2020-11-16T22:28:54.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060409e6d0 con 0x7f0604167ac0 2020-11-16T22:28:54.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.425+0000 7f0600ff9700 1 -- 172.21.15.181:0/616949764 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f05fc017070 con 0x7f0604167ac0 2020-11-16T22:28:54.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f0600ff9700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 0x7f05dc057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f06089a9700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 0x7f05dc057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f0600ff9700 1 -- 172.21.15.181:0/616949764 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05fc06feb0 con 0x7f0604167ac0 2020-11-16T22:28:54.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f06089a9700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 0x7f05dc057550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f05f4006fd0 tx=0x7f05f4006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f061969e740 1 Processor -- start 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.429+0000 7f061969e740 1 -- start start 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x1478cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x130b4b0 con 0x12617b0 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13bf930 con 0x147a470 2020-11-16T22:28:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x127f6f0 con 0x11f3bb0 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f0603fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58614/0 (socket says 172.21.15.181:58614) 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 -- 172.21.15.181:0/219399883 learned_addr learned my addr 172.21.15.181:0/219399883 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06089a9700 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x1478cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:54.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 -- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 msgr2=0x14b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 -- 172.21.15.181:0/219399883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 msgr2=0x1478cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x1478cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 -- 172.21.15.181:0/219399883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x127a5a0 con 0x147a470 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f0603fff700 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f06091aa700 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f05fc049b60 tx=0x7f05fc012000).ready entity=mon.1 client_cookie=92d2b60cc413a8e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f05e1ffb700 1 -- 172.21.15.181:0/219399883 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc006ca0 con 0x147a470 2020-11-16T22:28:54.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f05e1ffb700 1 -- 172.21.15.181:0/219399883 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc025cd0 con 0x147a470 2020-11-16T22:28:54.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f05e1ffb700 1 -- 172.21.15.181:0/219399883 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0491b0 con 0x147a470 2020-11-16T22:28:54.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 msgr2=0x14d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:54.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f05fc049b60 tx=0x7f05fc012000).stop 2020-11-16T22:28:54.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- 172.21.15.181:0/219399883 shutdown_connections 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14d4c90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x1478cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 --2- 172.21.15.181:0/219399883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- 172.21.15.181:0/219399883 >> 172.21.15.181:0/219399883 conn(0x142fe40 msgr2=0x14d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- 172.21.15.181:0/219399883 shutdown_connections 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.433+0000 7f061969e740 1 -- 172.21.15.181:0/219399883 wait complete. 2020-11-16T22:28:54.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 -- start start 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 Processor -- start 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x14b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13ac6e0 con 0x12617b0 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13ac820 con 0x147a470 2020-11-16T22:28:54.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13ada90 con 0x11f3bb0 2020-11-16T22:28:55.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f061969e740 10 client.0 ll_register_callbacks cb 0x13a2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:55.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38226/0 (socket says 172.21.15.181:38226) 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58620/0 (socket says 172.21.15.181:58620) 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 -- 172.21.15.181:0/2194075275 learned_addr learned my addr 172.21.15.181:0/2194075275 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f0603fff700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x14b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 -- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 msgr2=0x14b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 -- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 msgr2=0x14b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:55.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x14b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 -- 172.21.15.181:0/2194075275 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x139d280 con 0x11f3bb0 2020-11-16T22:28:55.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f06089a9700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f05f4011d60 tx=0x7f05f4013000).ready entity=mon.2 client_cookie=d64984098fd4738a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:55.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.437+0000 7f05d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:55.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011e60 con 0x11f3bb0 2020-11-16T22:28:55.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05f4012a30 con 0x11f3bb0 2020-11-16T22:28:55.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1474e10 con 0x11f3bb0 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05f4012e00 con 0x11f3bb0 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 10 client.5078 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x11d2fd0 con 0x11f3bb0 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1474e10 con 0x11f3bb0 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f05f401f350 con 0x11f3bb0 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 10 client.5078 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:55.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 10 client.5078 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 20 client.5078 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x14d0500 0x14e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 4 client.5078 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x129dc50 con 0x14d0500 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f0603fff700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x14d0500 0x14e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f0603fff700 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x14d0500 0x14e0810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f701e1505cdb2246 server_cookie=8a17a85a85ff52da in_seq=0 out_seq=0 2020-11-16T22:28:55.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 10 client.5078 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:55.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 1 -- 172.21.15.181:0/2194075275 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x129dc50 con 0x14d0500 2020-11-16T22:28:55.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 10 client.5078 handle_command_reply: tid=0 2020-11-16T22:28:55.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 client.5078 shutdown 2020-11-16T22:28:55.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 msgr2=0x14e2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f05f4011d60 tx=0x7f05f4013000).stop 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 shutdown_connections 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x14d0500 0x14e0810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x147a470 0x14b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 --2- 172.21.15.181:0/2194075275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12617b0 0x14b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f05d2ffd700 0 client.5078 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 >> 172.21.15.181:0/2194075275 conn(0x142fe40 msgr2=0x14a5340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 shutdown_connections 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 1 -- 172.21.15.181:0/2194075275 wait complete. 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.441+0000 7f061969e740 20 client.5078 trim_cache size 0 max 16384 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 Processor -- start 2020-11-16T22:28:55.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- start start 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x142fe40 0x14e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1384e40 con 0x11f3bb0 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1480660 con 0x12617b0 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x147b5c0 con 0x142fe40 2020-11-16T22:28:55.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06089a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x142fe40 0x14e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58626/0 (socket says 172.21.15.181:58626) 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 -- 172.21.15.181:0/2309953378 learned_addr learned my addr 172.21.15.181:0/2309953378 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f0603fff700 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 -- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x142fe40 msgr2=0x14e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x142fe40 0x14e0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 -- 172.21.15.181:0/2309953378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 msgr2=0x14e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:55.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 -- 172.21.15.181:0/2309953378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80081d0 con 0x12617b0 2020-11-16T22:28:55.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f06091aa700 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f05fc0461c0 tx=0x7f05fc027000).ready entity=mon.1 client_cookie=8f087b26af7aaae3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:55.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f05e27fc700 1 -- 172.21.15.181:0/2309953378 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0491b0 con 0x12617b0 2020-11-16T22:28:55.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f05e27fc700 1 -- 172.21.15.181:0/2309953378 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc006ca0 con 0x12617b0 2020-11-16T22:28:55.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f05e27fc700 1 -- 172.21.15.181:0/2309953378 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc045cc0 con 0x12617b0 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 msgr2=0x14cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f05fc0461c0 tx=0x7f05fc027000).stop 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- 172.21.15.181:0/2309953378 shutdown_connections 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12617b0 0x14cfa30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x142fe40 0x14e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 --2- 172.21.15.181:0/2309953378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x14e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:55.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- 172.21.15.181:0/2309953378 >> 172.21.15.181:0/2309953378 conn(0x7f05d8002320 msgr2=0x14b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:55.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- 172.21.15.181:0/2309953378 shutdown_connections 2020-11-16T22:28:55.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.465+0000 7f061969e740 1 -- 172.21.15.181:0/2309953378 wait complete. 2020-11-16T22:28:55.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- start start 2020-11-16T22:28:55.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 Processor -- start 2020-11-16T22:28:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x152e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 0x14cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x147e450 con 0x11f3bb0 2020-11-16T22:28:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x147e6f0 con 0x142fe40 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x147e990 con 0x12617b0 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 10 client.0 ll_register_callbacks cb 0x14b9770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 0x14cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38240/0 (socket says 172.21.15.181:38240) 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 -- 172.21.15.181:0/4044016992 learned_addr learned my addr 172.21.15.181:0/4044016992 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:28:55.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f06091aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 0x14cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58634/0 (socket says 172.21.15.181:58634) 2020-11-16T22:28:55.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f06089a9700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x152e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:55.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 -- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 msgr2=0x14cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:55.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 0x14cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 -- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 msgr2=0x152e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:28:56.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x152e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 -- 172.21.15.181:0/4044016992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1211720 con 0x12617b0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f0603fff700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f05f8007fd0 tx=0x7f05f8018040).ready entity=mon.2 client_cookie=3974eac16fed9eab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8016ce0 con 0x12617b0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05f8016e40 con 0x12617b0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05fc025cd0 con 0x12617b0 2020-11-16T22:28:56.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05f8024e70 con 0x12617b0 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 10 client.5084 fetch_fsmap learned FSMap version 38 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x121ce00 con 0x12617b0 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc025cd0 con 0x12617b0 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f05f802d420 con 0x12617b0 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 10 client.5084 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 10 client.5084 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:28:56.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 20 client.5084 populate_metadata read hostname 'smithi181' 2020-11-16T22:28:56.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x147a470 0x14d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:28:56.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 4 client.5084 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:28:56.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x12a4f50 con 0x147a470 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f06089a9700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x147a470 0x14d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f05d2ffd700 10 client.5084 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.469+0000 7f06089a9700 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x147a470 0x14d3630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1ed03cb57742ce39 server_cookie=76b6c63e58490b95 in_seq=0 out_seq=0 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f05d2ffd700 1 -- 172.21.15.181:0/4044016992 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x12a4f50 con 0x147a470 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f05d2ffd700 10 client.5084 handle_command_reply: tid=0 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 client.5084 shutdown 2020-11-16T22:28:56.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 msgr2=0x14a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f05f8007fd0 tx=0x7f05f8018040).stop 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 shutdown_connections 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x147a470 0x14d3630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142fe40 0x14cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11f3bb0 0x152e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f05d2ffd700 0 client.5084 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:28:56.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 --2- 172.21.15.181:0/4044016992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12617b0 0x14a5340 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 >> 172.21.15.181:0/4044016992 conn(0x7f05d8002320 msgr2=0x14a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 shutdown_connections 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 1 -- 172.21.15.181:0/4044016992 wait complete. 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f061969e740 20 client.5084 trim_cache size 0 max 16384 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 msgr2=0x7f05dc057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 0x7f05dc057550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f05f4006fd0 tx=0x7f05f4006df0).stop 2020-11-16T22:28:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 msgr2=0x7f0604174a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f0604168990 tx=0x7f05fc016000).stop 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 shutdown_connections 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f05dc055090 0x7f05dc057550 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0604167ac0 0x7f0604174a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06040b3220 0x7f06040cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 --2- 172.21.15.181:0/616949764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06041672c0 0x7f0604161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:28:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 >> 172.21.15.181:0/616949764 conn(0x7f0604060bc0 msgr2=0x7f0604066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 shutdown_connections 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:28:54.473+0000 7f060ac0d700 1 -- 172.21.15.181:0/616949764 wait complete. 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:28:56.267 INFO:teuthology.orchestra.run.smithi181.stdout: "tag123": { 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "tag123", 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:28:56.268 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:28:56.269 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:28:58.906 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:28:58.912 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:28:58.914 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:01.270 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:01.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 Processor -- start 2020-11-16T22:29:01.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 -- start start 2020-11-16T22:29:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe390145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3901492c0 0x7fe390080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 0x7fe3900cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe390061b50 con 0x7fe3900814c0 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3900618b0 con 0x7fe3901492c0 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe396fbf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900cb930 con 0x7fe390145a30 2020-11-16T22:29:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe3957bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3901492c0 0x7fe390080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe395fbd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe390145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe3957bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3901492c0 0x7fe390080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58640/0 (socket says 172.21.15.181:58640) 2020-11-16T22:29:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe3957bc700 1 -- 172.21.15.181:0/861443635 learned_addr learned my addr 172.21.15.181:0/861443635 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.421+0000 7fe3967be700 1 --2- 172.21.15.181:0/861443635 >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 0x7fe3900cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 -- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 msgr2=0x7fe390145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 --2- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe390145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 -- 172.21.15.181:0/861443635 >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 msgr2=0x7fe3900cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 --2- 172.21.15.181:0/861443635 >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 0x7fe3900cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 -- 172.21.15.181:0/861443635 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900cbbd0 con 0x7fe3901492c0 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3967be700 1 --2- 172.21.15.181:0/861443635 >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 0x7fe3900cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe3957bc700 1 --2- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3901492c0 0x7fe390080fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fe3900cdbd0 tx=0x7fe384006eb0).ready entity=mon.1 client_cookie=fb0bf64fb0853f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:01.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe394fbb700 1 -- 172.21.15.181:0/861443635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384009810 con 0x7fe3901492c0 2020-11-16T22:29:01.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe394fbb700 1 -- 172.21.15.181:0/861443635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384009970 con 0x7fe3901492c0 2020-11-16T22:29:01.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe394fbb700 1 -- 172.21.15.181:0/861443635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384009cf0 con 0x7fe3901492c0 2020-11-16T22:29:01.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3901492c0 msgr2=0x7fe390080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:01.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 --2- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3901492c0 0x7fe390080fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fe3900cdbd0 tx=0x7fe384006eb0).stop 2020-11-16T22:29:01.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- 172.21.15.181:0/861443635 shutdown_connections 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 --2- 172.21.15.181:0/861443635 >> v2:172.21.15.175:3300/0 conn(0x7fe3900814c0 0x7fe3900cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 --2- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe390145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 --2- 172.21.15.181:0/861443635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3901492c0 0x7fe390080fd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- 172.21.15.181:0/861443635 >> 172.21.15.181:0/861443635 conn(0x7fe3900d68c0 msgr2=0x7fe3900d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- 172.21.15.181:0/861443635 shutdown_connections 2020-11-16T22:29:01.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- 172.21.15.181:0/861443635 wait complete. 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 Processor -- start 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.425+0000 7fe396fbf700 1 -- start start 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3900814c0 0x7fe3901709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3901492c0 0x7fe3900cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:01.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe390170ee0 con 0x7fe3901492c0 2020-11-16T22:29:01.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39013e730 con 0x7fe3900814c0 2020-11-16T22:29:01.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900cdd80 con 0x7fe390145a30 2020-11-16T22:29:01.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38256/0 (socket says 172.21.15.181:38256) 2020-11-16T22:29:01.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe395fbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3900814c0 0x7fe3901709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 -- 172.21.15.181:0/1714896891 learned_addr learned my addr 172.21.15.181:0/1714896891 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:01.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3967be700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3901492c0 0x7fe3900cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 -- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3900814c0 msgr2=0x7fe3901709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3900814c0 0x7fe3901709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 -- 172.21.15.181:0/1714896891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3901492c0 msgr2=0x7fe3900cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3901492c0 0x7fe3900cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 -- 172.21.15.181:0/1714896891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900ce020 con 0x7fe390145a30 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe3957bc700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fe384010000 tx=0x7fe384015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:01.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384011c80 con 0x7fe390145a30 2020-11-16T22:29:01.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384011de0 con 0x7fe390145a30 2020-11-16T22:29:01.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3900ce2c0 con 0x7fe390145a30 2020-11-16T22:29:01.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38401e670 con 0x7fe390145a30 2020-11-16T22:29:02.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3900d08c0 con 0x7fe390145a30 2020-11-16T22:29:02.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe384025030 con 0x7fe390145a30 2020-11-16T22:29:02.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.429+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39013e930 con 0x7fe390145a30 2020-11-16T22:29:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.433+0000 7fe381ffb700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 0x7fe368057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.433+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe38406f3d0 con 0x7fe390145a30 2020-11-16T22:29:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.433+0000 7fe395fbd700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 0x7fe368057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.433+0000 7fe395fbd700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 0x7fe368057520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe38c006fd0 tx=0x7fe38c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.441+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe384045630 con 0x7fe390145a30 2020-11-16T22:29:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.669+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe390044f60 con 0x7fe390145a30 2020-11-16T22:29:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe381ffb700 1 -- 172.21.15.181:0/1714896891 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fe384023090 con 0x7fe390145a30 2020-11-16T22:29:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 msgr2=0x7fe368057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 0x7fe368057520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe38c006fd0 tx=0x7fe38c006df0).stop 2020-11-16T22:29:02.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 msgr2=0x7fe3900d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fe384010000 tx=0x7fe384015000).stop 2020-11-16T22:29:02.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 shutdown_connections 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe368055060 0x7fe368057520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3901492c0 0x7fe3900cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3900814c0 0x7fe3901709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 --2- 172.21.15.181:0/1714896891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe390145a30 0x7fe3900d03d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 >> 172.21.15.181:0/1714896891 conn(0x7fe3900d68c0 msgr2=0x7fe39013eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 shutdown_connections 2020-11-16T22:29:02.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:01.673+0000 7fe396fbf700 1 -- 172.21.15.181:0/1714896891 wait complete. 2020-11-16T22:29:02.236 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:29:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:02.236 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:29:02.438 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:29:02.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 Processor -- start 2020-11-16T22:29:02.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 -- start start 2020-11-16T22:29:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff24160e40 con 0x7eff2415f5b0 2020-11-16T22:29:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff24160ba0 con 0x7eff240615f0 2020-11-16T22:29:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff29e61700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff241609a0 con 0x7eff240629b0 2020-11-16T22:29:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38264/0 (socket says 172.21.15.181:38264) 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 -- 172.21.15.181:0/56427554 learned_addr learned my addr 172.21.15.181:0/56427554 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff227fc700 1 -- 172.21.15.181:0/56427554 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400972007 0 0) 0x7eff24160ba0 con 0x7eff240615f0 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff227fc700 1 -- 172.21.15.181:0/56427554 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff10003240 con 0x7eff240615f0 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 -- 172.21.15.181:0/56427554 >> v1:172.21.15.181:6789/0 conn(0x7eff240615f0 legacy=0x7eff240622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 -- 172.21.15.181:0/56427554 >> v1:172.21.15.175:6789/0 conn(0x7eff2415f5b0 legacy=0x7eff240d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 -- 172.21.15.181:0/56427554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24060970 con 0x7eff240629b0 2020-11-16T22:29:02.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff23fff700 1 --2- 172.21.15.181:0/56427554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7eff1c009d80 tx=0x7eff1c009290).ready entity=mon.2 client_cookie=33ce807fa5731bcb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff227fc700 1 -- 172.21.15.181:0/56427554 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00bbc0 con 0x7eff240629b0 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff227fc700 1 -- 172.21.15.181:0/56427554 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c00bd20 con 0x7eff240629b0 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.593+0000 7eff227fc700 1 -- 172.21.15.181:0/56427554 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c011610 con 0x7eff240629b0 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- 172.21.15.181:0/56427554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 msgr2=0x7eff24080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 --2- 172.21.15.181:0/56427554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7eff1c009d80 tx=0x7eff1c009290).stop 2020-11-16T22:29:02.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- 172.21.15.181:0/56427554 shutdown_connections 2020-11-16T22:29:02.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 --2- 172.21.15.181:0/56427554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240629b0 0x7eff24080fd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- 172.21.15.181:0/56427554 >> 172.21.15.181:0/56427554 conn(0x7eff240d69a0 msgr2=0x7eff240d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- 172.21.15.181:0/56427554 shutdown_connections 2020-11-16T22:29:02.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- 172.21.15.181:0/56427554 wait complete. 2020-11-16T22:29:02.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 Processor -- start 2020-11-16T22:29:02.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.597+0000 7eff29e61700 1 -- start start 2020-11-16T22:29:02.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff240629b0 0x7eff24173380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 0x7eff24161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24173870 con 0x7eff240629b0 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24157390 con 0x7eff2415f5b0 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2416fd70 con 0x7eff240615f0 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 0x7eff24161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 0x7eff24161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58660/0 (socket says 172.21.15.181:58660) 2020-11-16T22:29:02.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff23fff700 1 -- 172.21.15.181:0/3360370426 learned_addr learned my addr 172.21.15.181:0/3360370426 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38268/0 (socket says 172.21.15.181:38268) 2020-11-16T22:29:02.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff240629b0 0x7eff24173380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 -- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 msgr2=0x7eff24161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 0x7eff24161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 -- 172.21.15.181:0/3360370426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff240629b0 msgr2=0x7eff24173380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff240629b0 0x7eff24173380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 -- 172.21.15.181:0/3360370426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24170010 con 0x7eff240615f0 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff237fe700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7eff1400c6e0 tx=0x7eff14007ab0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff037fe700 1 -- 172.21.15.181:0/3360370426 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400f470 con 0x7eff240615f0 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff037fe700 1 -- 172.21.15.181:0/3360370426 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400f5d0 con 0x7eff240615f0 2020-11-16T22:29:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff241702b0 con 0x7eff240615f0 2020-11-16T22:29:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff037fe700 1 -- 172.21.15.181:0/3360370426 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400f950 con 0x7eff240615f0 2020-11-16T22:29:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.601+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff240d5a90 con 0x7eff240615f0 2020-11-16T22:29:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.605+0000 7eff037fe700 1 -- 172.21.15.181:0/3360370426 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7eff14019030 con 0x7eff240615f0 2020-11-16T22:29:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.605+0000 7eff037fe700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 0x7efefc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.605+0000 7eff037fe700 1 -- 172.21.15.181:0/3360370426 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff1406cf10 con 0x7eff240615f0 2020-11-16T22:29:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.605+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 0x7efefc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.605+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 0x7efefc057520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7eff240b8f40 tx=0x7eff18007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 Processor -- start 2020-11-16T22:29:02.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 -- start start 2020-11-16T22:29:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 0x23d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 0x240f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 0x242cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22634b0 con 0x23d2470 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2317930 con 0x21b97b0 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21d76f0 con 0x214bbb0 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 0x23d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 0x240f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 0x23d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58666/0 (socket says 172.21.15.181:58666) 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff237fe700 1 -- 172.21.15.181:0/2745258065 learned_addr learned my addr 172.21.15.181:0/2745258065 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff23fff700 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 0x242cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 -- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 msgr2=0x23d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 0x23d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 -- 172.21.15.181:0/2745258065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 msgr2=0x242cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 0x242cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 -- 172.21.15.181:0/2745258065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21d25a0 con 0x214bbb0 2020-11-16T22:29:02.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff23fff700 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 0x242cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:02.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 0x240f530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7eff2415d170 tx=0x7eff1800a000).ready entity=mon.2 client_cookie=916eba8cb37a32d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff00ff9700 1 -- 172.21.15.181:0/2745258065 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18017240 con 0x214bbb0 2020-11-16T22:29:02.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff00ff9700 1 -- 172.21.15.181:0/2745258065 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180157e0 con 0x214bbb0 2020-11-16T22:29:02.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff00ff9700 1 -- 172.21.15.181:0/2745258065 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18015b60 con 0x214bbb0 2020-11-16T22:29:02.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 -- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 msgr2=0x240f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.609+0000 7eff388f2740 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 0x240f530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7eff2415d170 tx=0x7eff1800a000).stop 2020-11-16T22:29:02.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- 172.21.15.181:0/2745258065 shutdown_connections 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d2470 0x242cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b97b0 0x23d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- 172.21.15.181:0/2745258065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x214bbb0 0x240f530 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- 172.21.15.181:0/2745258065 >> 172.21.15.181:0/2745258065 conn(0x2387e40 msgr2=0x242c580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- 172.21.15.181:0/2745258065 shutdown_connections 2020-11-16T22:29:02.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- 172.21.15.181:0/2745258065 wait complete. 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- start start 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 Processor -- start 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x2439e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d2470 0x2438480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23046e0 con 0x214bbb0 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2304820 con 0x23d2470 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2305a90 con 0x21b97b0 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff388f2740 10 client.0 ll_register_callbacks cb 0x22faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:29:02.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38284/0 (socket says 172.21.15.181:38284) 2020-11-16T22:29:02.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 -- 172.21.15.181:0/2753114252 learned_addr learned my addr 172.21.15.181:0/2753114252 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff23fff700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d2470 0x2438480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff237fe700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x2439e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 -- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d2470 msgr2=0x2438480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d2470 0x2438480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 -- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 msgr2=0x2439e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x2439e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 -- 172.21.15.181:0/2753114252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x22f5280 con 0x21b97b0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.613+0000 7eff22ffd700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7eff24061a50 tx=0x7eff1801b000).ready entity=mon.2 client_cookie=2898631336c0096e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18028070 con 0x21b97b0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23cce10 con 0x21b97b0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff18017240 con 0x21b97b0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff18015b60 con 0x21b97b0 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 10 client.5102 fetch_fsmap learned FSMap version 38 2020-11-16T22:29:02.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x212afd0 con 0x21b97b0 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23cce10 con 0x21b97b0 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7eff18023080 con 0x21b97b0 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 10 client.5102 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 10 client.5102 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 20 client.5102 populate_metadata read hostname 'smithi181' 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23ff020 0x243a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 4 client.5102 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x21f5c50 con 0x23ff020 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff237fe700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23ff020 0x243a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff237fe700 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23ff020 0x243a580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=79996ebfaa08087a server_cookie=204c2767ce2dfed8 in_seq=0 out_seq=0 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 10 client.5102 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 1 -- 172.21.15.181:0/2753114252 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x21f5c50 con 0x23ff020 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 10 client.5102 handle_command_reply: tid=0 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 client.5102 shutdown 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 msgr2=0x2428300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7eff24061a50 tx=0x7eff1801b000).stop 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 shutdown_connections 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23ff020 0x243a580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d2470 0x2438480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x2439e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 --2- 172.21.15.181:0/2753114252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x2428300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7efef9ffb700 0 client.5102 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 >> 172.21.15.181:0/2753114252 conn(0x2387e40 msgr2=0x240e650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 shutdown_connections 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 1 -- 172.21.15.181:0/2753114252 wait complete. 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.617+0000 7eff388f2740 20 client.5102 trim_cache size 0 max 16384 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 Processor -- start 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 -- start start 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x214bbb0 0x2411d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 0x23d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22dce40 con 0x21b97b0 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x243a160 con 0x214bbb0 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24381d0 con 0x2387e40 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38292/0 (socket says 172.21.15.181:38292) 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 -- 172.21.15.181:0/3992965296 learned_addr learned my addr 172.21.15.181:0/3992965296 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x214bbb0 0x2411d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff23fff700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 0x23d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 -- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x214bbb0 msgr2=0x2411d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x214bbb0 0x2411d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 -- 172.21.15.181:0/3992965296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 msgr2=0x23d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 0x23d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 -- 172.21.15.181:0/3992965296 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00081d0 con 0x2387e40 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff23fff700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 0x23d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff237fe700 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7eff1400cfd0 tx=0x7eff14012000).ready entity=mon.2 client_cookie=215f3c366157af23 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff017fa700 1 -- 172.21.15.181:0/3992965296 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff140450a0 con 0x2387e40 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff017fa700 1 -- 172.21.15.181:0/3992965296 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff14045280 con 0x2387e40 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.637+0000 7eff017fa700 1 -- 172.21.15.181:0/3992965296 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff140463d0 con 0x2387e40 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 msgr2=0x23fdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7eff1400cfd0 tx=0x7eff14012000).stop 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- 172.21.15.181:0/3992965296 shutdown_connections 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b97b0 0x23d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2387e40 0x23fdb30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 --2- 172.21.15.181:0/3992965296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x214bbb0 0x2411d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- 172.21.15.181:0/3992965296 >> 172.21.15.181:0/3992965296 conn(0x7efef0002320 msgr2=0x23fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- 172.21.15.181:0/3992965296 shutdown_connections 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- 172.21.15.181:0/3992965296 wait complete. 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 -- start start 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.641+0000 7eff388f2740 1 Processor -- start 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x240e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2387e40 0x23d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x242fd80 con 0x214bbb0 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x240ddf0 con 0x2387e40 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x240e090 con 0x21b97b0 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 10 client.0 ll_register_callbacks cb 0x2411c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38296/0 (socket says 172.21.15.181:38296) 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 -- 172.21.15.181:0/3620673926 learned_addr learned my addr 172.21.15.181:0/3620673926 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff23fff700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2387e40 0x23d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff237fe700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x240e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 -- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2387e40 msgr2=0x23d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2387e40 0x23d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 -- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 msgr2=0x240e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x240e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 -- 172.21.15.181:0/3620673926 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2169720 con 0x21b97b0 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff237fe700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x240e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff22ffd700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7eff1802c5f0 tx=0x7eff18028000).ready entity=mon.2 client_cookie=79a21d131dbf2726 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1803a070 con 0x21b97b0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff18023820 con 0x21b97b0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff180157e0 con 0x21b97b0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff18015660 con 0x21b97b0 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 10 client.5108 fetch_fsmap learned FSMap version 38 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x2174e00 con 0x21b97b0 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180157e0 con 0x21b97b0 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7eff1802c680 con 0x21b97b0 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 10 client.5108 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 10 client.5108 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 20 client.5108 populate_metadata read hostname 'smithi181' 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23d2470 0x248c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 4 client.5108 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x21fcf50 con 0x23d2470 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff237fe700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23d2470 0x248c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 10 client.5108 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff237fe700 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23d2470 0x248c980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ab74f00666a644a server_cookie=bf22939af189cd0d in_seq=0 out_seq=0 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 1 -- 172.21.15.181:0/3620673926 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x21fcf50 con 0x23d2470 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 10 client.5108 handle_command_reply: tid=0 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 client.5108 shutdown 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 msgr2=0x23cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7eff1802c5f0 tx=0x7eff18028000).stop 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 shutdown_connections 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x23d2470 0x248c980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2387e40 0x23d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7efef9ffb700 0 client.5108 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x214bbb0 0x240e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 --2- 172.21.15.181:0/3620673926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21b97b0 0x23cfd40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.645+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 >> 172.21.15.181:0/3620673926 conn(0x7efef0002320 msgr2=0x23d9140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 shutdown_connections 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff388f2740 1 -- 172.21.15.181:0/3620673926 wait complete. 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff388f2740 20 client.5108 trim_cache size 0 max 16384 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 msgr2=0x7efefc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 0x7efefc057520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7eff240b8f40 tx=0x7eff18007a20).stop 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 msgr2=0x7eff241612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7eff1400c6e0 tx=0x7eff14007ab0).stop 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 shutdown_connections 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efefc055060 0x7efefc057520 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff2415f5b0 0x7eff24161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff240615f0 0x7eff241612c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 --2- 172.21.15.181:0/3360370426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff240629b0 0x7eff24173380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 >> 172.21.15.181:0/3360370426 conn(0x7eff240d69a0 msgr2=0x7eff24061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 shutdown_connections 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:02.649+0000 7eff29e61700 1 -- 172.21.15.181:0/3360370426 wait complete. 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "tag123": { 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "tag123", 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:29:02.833 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:29:07.834 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 Processor -- start 2020-11-16T22:29:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 -- start start 2020-11-16T22:29:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bc8161860 0x7f9bc8062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc80c8db0 con 0x7f9bc8164e10 2020-11-16T22:29:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc80c8330 con 0x7f9bc8161860 2020-11-16T22:29:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.981+0000 7f9bcef52700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c85d0 con 0x7f9bc8161480 2020-11-16T22:29:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcd74f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9bc8164e10 0x7f9bc8062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44768/0 (socket says 172.21.15.181:44768) 2020-11-16T22:29:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcdf50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcd74f700 1 -- 172.21.15.181:0/3148964285 learned_addr learned my addr 172.21.15.181:0/3148964285 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 --2- 172.21.15.181:0/3148964285 >> v2:172.21.15.181:3300/0 conn(0x7f9bc8161860 0x7f9bc8062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 -- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 msgr2=0x7f9bc8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 -- 172.21.15.181:0/3148964285 >> v1:172.21.15.181:6789/0 conn(0x7f9bc8164e10 legacy=0x7f9bc8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bccf4e700 1 -- 172.21.15.181:0/3148964285 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146530606 0 0) 0x7f9bc80c8db0 con 0x7f9bc8164e10 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 -- 172.21.15.181:0/3148964285 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80c8870 con 0x7f9bc8161860 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcdf50700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bce751700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8161860 0x7f9bc8062960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f9bc000b9a0 tx=0x7f9bc0006df0).ready entity=mon.1 client_cookie=3379ac07a43448c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bccf4e700 1 -- 172.21.15.181:0/3148964285 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00097b0 con 0x7f9bc8161860 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bccf4e700 1 -- 172.21.15.181:0/3148964285 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0009910 con 0x7f9bc8161860 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bccf4e700 1 -- 172.21.15.181:0/3148964285 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0009c90 con 0x7f9bc8161860 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8161860 msgr2=0x7f9bc8062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8161860 0x7f9bc8062960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f9bc000b9a0 tx=0x7f9bc0006df0).stop 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- 172.21.15.181:0/3148964285 shutdown_connections 2020-11-16T22:29:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8161860 0x7f9bc8062960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:07.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3148964285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:07.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- 172.21.15.181:0/3148964285 >> 172.21.15.181:0/3148964285 conn(0x7f9bc80d2dc0 msgr2=0x7f9bc80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:07.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- 172.21.15.181:0/3148964285 shutdown_connections 2020-11-16T22:29:07.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- 172.21.15.181:0/3148964285 wait complete. 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 Processor -- start 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.985+0000 7f9bcef52700 1 -- start start 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8176c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc8161860 0x7f9bc81605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8149fd0 con 0x7f9bc8161860 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8174b50 con 0x7f9bc8177140 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8177b60 con 0x7f9bc8161480 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcdf50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8176c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcdf50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8176c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38310/0 (socket says 172.21.15.181:38310) 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58702/0 (socket says 172.21.15.181:58702) 2020-11-16T22:29:07.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcdf50700 1 -- 172.21.15.181:0/3466995321 learned_addr learned my addr 172.21.15.181:0/3466995321 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcd74f700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc8161860 0x7f9bc81605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 -- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 msgr2=0x7f9bc8176c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8176c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 -- 172.21.15.181:0/3466995321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc8161860 msgr2=0x7f9bc81605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc8161860 0x7f9bc81605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 -- 172.21.15.181:0/3466995321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8177620 con 0x7f9bc8177140 2020-11-16T22:29:07.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bce751700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f9bc0012000 tx=0x7f9bc0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0006b20 con 0x7f9bc8177140 2020-11-16T22:29:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0009e80 con 0x7f9bc8177140 2020-11-16T22:29:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc81778c0 con 0x7f9bc8177140 2020-11-16T22:29:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001d800 con 0x7f9bc8177140 2020-11-16T22:29:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.989+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc80b7130 con 0x7f9bc8177140 2020-11-16T22:29:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc80b6cd0 con 0x7f9bc8177140 2020-11-16T22:29:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9bc0025030 con 0x7f9bc8177140 2020-11-16T22:29:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bbdffb700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 0x7f9ba0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bcdf50700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 0x7f9ba0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9bc006f380 con 0x7f9bc8177140 2020-11-16T22:29:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:07.993+0000 7f9bcdf50700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 0x7f9ba0057520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9bb800ba80 tx=0x7f9bb8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:07.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.001+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9bc0045690 con 0x7f9bc8177140 2020-11-16T22:29:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.205+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bc8044f60 con 0x7f9bc8177140 2020-11-16T22:29:08.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.205+0000 7f9bbdffb700 1 -- 172.21.15.181:0/3466995321 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f9bc0023070 con 0x7f9bc8177140 2020-11-16T22:29:08.191 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:08.191 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:29:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 msgr2=0x7f9ba0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 0x7f9ba0057520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9bb800ba80 tx=0x7f9bb8006df0).stop 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 msgr2=0x7f9bc8160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f9bc0012000 tx=0x7f9bc0015000).stop 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 shutdown_connections 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba0055060 0x7f9ba0057520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8177140 0x7f9bc8160aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bc8161480 0x7f9bc8176c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 --2- 172.21.15.181:0/3466995321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bc8161860 0x7f9bc81605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 >> 172.21.15.181:0/3466995321 conn(0x7f9bc80d2dc0 msgr2=0x7f9bc8063220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 shutdown_connections 2020-11-16T22:29:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.209+0000 7f9bcef52700 1 -- 172.21.15.181:0/3466995321 wait complete. 2020-11-16T22:29:08.195 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:29:08.206 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:29:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 Processor -- start 2020-11-16T22:29:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 -- start start 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 0x7f51e40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f51e40615f0 0x7f51e40d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e4160e40 con 0x7f51e40622a0 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e41609a0 con 0x7f51e40615f0 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51ea6f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4160ba0 con 0x7f51e415f5b0 2020-11-16T22:29:08.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e8c94700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f51e40622a0 0x7f51e4062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44782/0 (socket says 172.21.15.181:44782) 2020-11-16T22:29:08.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e8c94700 1 -- 172.21.15.181:0/1602464964 learned_addr learned my addr 172.21.15.181:0/1602464964 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:08.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1602464964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 0x7f51e40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1602464964 >> v2:172.21.15.175:3300/0 conn(0x7f51e40615f0 0x7f51e40d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e2ffd700 1 -- 172.21.15.181:0/1602464964 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14507977 0 0) 0x7f51e4160e40 con 0x7f51e40622a0 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e2ffd700 1 -- 172.21.15.181:0/1602464964 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d0003240 con 0x7f51e40622a0 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e3fff700 1 -- 172.21.15.181:0/1602464964 >> v2:172.21.15.175:3300/0 conn(0x7f51e40615f0 msgr2=0x7f51e40d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1602464964 >> v2:172.21.15.175:3300/0 conn(0x7f51e40615f0 0x7f51e40d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e3fff700 1 -- 172.21.15.181:0/1602464964 >> v1:172.21.15.181:6789/0 conn(0x7f51e40622a0 legacy=0x7f51e4062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.349+0000 7f51e3fff700 1 -- 172.21.15.181:0/1602464964 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4060970 con 0x7f51e415f5b0 2020-11-16T22:29:08.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1602464964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 0x7f51e40d1340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f51d4006fd0 tx=0x7f51d40091a0).ready entity=mon.2 client_cookie=f43e3dc6739e3b8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51e2ffd700 1 -- 172.21.15.181:0/1602464964 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4005a50 con 0x7f51e415f5b0 2020-11-16T22:29:08.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51e2ffd700 1 -- 172.21.15.181:0/1602464964 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4005bb0 con 0x7f51e415f5b0 2020-11-16T22:29:08.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51e2ffd700 1 -- 172.21.15.181:0/1602464964 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400b9c0 con 0x7f51e415f5b0 2020-11-16T22:29:08.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 -- 172.21.15.181:0/1602464964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 msgr2=0x7f51e40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/1602464964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 0x7f51e40d1340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f51d4006fd0 tx=0x7f51d40091a0).stop 2020-11-16T22:29:08.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 -- 172.21.15.181:0/1602464964 shutdown_connections 2020-11-16T22:29:08.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/1602464964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e415f5b0 0x7f51e40d1340 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/1602464964 >> v2:172.21.15.175:3300/0 conn(0x7f51e40615f0 0x7f51e40d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 -- 172.21.15.181:0/1602464964 >> 172.21.15.181:0/1602464964 conn(0x7f51e40d69a0 msgr2=0x7f51e40d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 -- 172.21.15.181:0/1602464964 shutdown_connections 2020-11-16T22:29:08.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.353+0000 7f51ea6f7700 1 -- 172.21.15.181:0/1602464964 wait complete. 2020-11-16T22:29:08.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 Processor -- start 2020-11-16T22:29:08.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 -- start start 2020-11-16T22:29:08.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e40622a0 0x7f51e4161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e415f5b0 0x7f51e4161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4161c10 con 0x7f51e40622a0 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40b9ca0 con 0x7f51e415f5b0 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51ea6f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40b9f40 con 0x7f51e40615f0 2020-11-16T22:29:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51e8c94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e415f5b0 0x7f51e4161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38324/0 (socket says 172.21.15.181:38324) 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.357+0000 7f51e3fff700 1 -- 172.21.15.181:0/3674450934 learned_addr learned my addr 172.21.15.181:0/3674450934 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e40622a0 0x7f51e4161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 -- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e415f5b0 msgr2=0x7f51e4161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e415f5b0 0x7f51e4161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 -- 172.21.15.181:0/3674450934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e40622a0 msgr2=0x7f51e4161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e40622a0 0x7f51e4161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 -- 172.21.15.181:0/3674450934 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40ba1e0 con 0x7f51e40615f0 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e3fff700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f51d40091e0 tx=0x7f51d4013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 -- 172.21.15.181:0/3674450934 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40066a0 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 -- 172.21.15.181:0/3674450934 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4006800 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 -- 172.21.15.181:0/3674450934 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4011690 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e40ba480 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e40bae60 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 -- 172.21.15.181:0/3674450934 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51d401a040 con 0x7f51e40615f0 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 0x7f51bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51c3fff700 1 -- 172.21.15.181:0/3674450934 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51d406f570 con 0x7f51e40615f0 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.361+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 0x7f51bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 0x7f51bc057520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f51e41698d0 tx=0x7f51d8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 Processor -- start 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 -- start start 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b61530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b24470 0x2b7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x29b54b0 con 0x2b24470 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a69930 con 0x289dbb0 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x29296f0 con 0x290b7b0 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b61530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38332/0 (socket says 172.21.15.181:38332) 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 -- 172.21.15.181:0/2117677188 learned_addr learned my addr 172.21.15.181:0/2117677188 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e8c94700 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b24470 0x2b7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 -- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 msgr2=0x2b61530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b61530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 -- 172.21.15.181:0/2117677188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b24470 msgr2=0x2b7ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b24470 0x2b7ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 -- 172.21.15.181:0/2117677188 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29245a0 con 0x290b7b0 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51e3fff700 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f51d4006c10 tx=0x7f51d4010000).ready entity=mon.2 client_cookie=ce14e6ebc438a673 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51c17fa700 1 -- 172.21.15.181:0/2117677188 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4047830 con 0x290b7b0 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51c17fa700 1 -- 172.21.15.181:0/2117677188 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4048980 con 0x290b7b0 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.365+0000 7f51c17fa700 1 -- 172.21.15.181:0/2117677188 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d406f0d0 con 0x290b7b0 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 msgr2=0x2b22cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f51d4006c10 tx=0x7f51d4010000).stop 2020-11-16T22:29:08.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- 172.21.15.181:0/2117677188 shutdown_connections 2020-11-16T22:29:08.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b24470 0x2b7ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x290b7b0 0x2b22cd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 --2- 172.21.15.181:0/2117677188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b61530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- 172.21.15.181:0/2117677188 >> 172.21.15.181:0/2117677188 conn(0x2ad9e40 msgr2=0x2b7e580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- 172.21.15.181:0/2117677188 shutdown_connections 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- 172.21.15.181:0/2117677188 wait complete. 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 -- start start 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.369+0000 7f51f9188740 1 Processor -- start 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 0x2b299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a566e0 con 0x2b27b10 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a56820 con 0x2b24470 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a57a90 con 0x289dbb0 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 10 client.0 ll_register_callbacks cb 0x2a4cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58730/0 (socket says 172.21.15.181:58730) 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 -- 172.21.15.181:0/3028338541 learned_addr learned my addr 172.21.15.181:0/3028338541 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e8c94700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 0x2b299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 -- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 msgr2=0x2b263a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b263a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 -- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 msgr2=0x2b299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 0x2b299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 -- 172.21.15.181:0/3028338541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a47280 con 0x2b24470 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e8c94700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 0x2b299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e37fe700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f51e40d2ea0 tx=0x7f51d8014000).ready entity=mon.1 client_cookie=4ced00b503e028d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8012a30 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2b1ee10 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51d8012b90 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d8006020 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 10 client.5395 fetch_fsmap learned FSMap version 38 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x287cfd0 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2b1ee10 con 0x2b24470 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f51d802e020 con 0x2b24470 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 10 client.5395 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 10 client.5395 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 20 client.5395 populate_metadata read hostname 'smithi181' 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27f50 0x2b799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 4 client.5395 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2947c50 con 0x2b27f50 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e3fff700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27f50 0x2b799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51ba7fc700 10 client.5395 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.373+0000 7f51e3fff700 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27f50 0x2b799a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b1bfd49c0b420c9c server_cookie=1827abb94857ea8d in_seq=0 out_seq=0 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51ba7fc700 1 -- 172.21.15.181:0/3028338541 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2947c50 con 0x2b27f50 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51ba7fc700 10 client.5395 handle_command_reply: tid=0 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 client.5395 shutdown 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 msgr2=0x2b8a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f51e40d2ea0 tx=0x7f51d8014000).stop 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 shutdown_connections 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27f50 0x2b799a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b27b10 0x2b299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51ba7fc700 0 client.5395 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 --2- 172.21.15.181:0/3028338541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b8a420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 >> 172.21.15.181:0/3028338541 conn(0x2ad9e40 msgr2=0x2b61620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 shutdown_connections 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 1 -- 172.21.15.181:0/3028338541 wait complete. 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.377+0000 7f51f9188740 20 client.5395 trim_cache size 0 max 16384 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 Processor -- start 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- start start 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b26410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2b24470 0x2b99680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a2ee40 con 0x2ad9e40 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b8cec0 con 0x289dbb0 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b63230 con 0x2b24470 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58738/0 (socket says 172.21.15.181:58738) 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 -- 172.21.15.181:0/1238946758 learned_addr learned my addr 172.21.15.181:0/1238946758 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e8c94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2b24470 0x2b99680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b26410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 -- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2b24470 msgr2=0x2b99680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2b24470 0x2b99680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 -- 172.21.15.181:0/1238946758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 msgr2=0x2b26410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b26410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 -- 172.21.15.181:0/1238946758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2b28720 con 0x289dbb0 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f51d8005780 tx=0x7f51d8024000).ready entity=mon.1 client_cookie=84e65bb3cb85b040 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51c1ffb700 1 -- 172.21.15.181:0/1238946758 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8013510 con 0x289dbb0 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51c1ffb700 1 -- 172.21.15.181:0/1238946758 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d801f3f0 con 0x289dbb0 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51c1ffb700 1 -- 172.21.15.181:0/1238946758 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8013670 con 0x289dbb0 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 msgr2=0x2b23d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f51d8005780 tx=0x7f51d8024000).stop 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- 172.21.15.181:0/1238946758 shutdown_connections 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2b24470 0x2b99680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b26410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 --2- 172.21.15.181:0/1238946758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x289dbb0 0x2b23d00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- 172.21.15.181:0/1238946758 >> 172.21.15.181:0/1238946758 conn(0x7f51b0002320 msgr2=0x2b255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- 172.21.15.181:0/1238946758 shutdown_connections 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- 172.21.15.181:0/1238946758 wait complete. 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 -- start start 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.397+0000 7f51f9188740 1 Processor -- start 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b4ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b7dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b4d6d0 con 0x2ad9e40 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b4d970 con 0x2b24470 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2be4510 con 0x289dbb0 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 10 client.0 ll_register_callbacks cb 0x2b63900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e8c94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b7dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e8c94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b7dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58744/0 (socket says 172.21.15.181:58744) 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e8c94700 1 -- 172.21.15.181:0/1387709496 learned_addr learned my addr 172.21.15.181:0/1387709496 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b4ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 -- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 msgr2=0x2b7dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b7dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 -- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 msgr2=0x2b4ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b4ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 -- 172.21.15.181:0/1387709496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b00081d0 con 0x289dbb0 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e3fff700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b2b2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f51d40093f0 tx=0x7f51d406f000).ready entity=mon.2 client_cookie=469988ac79531661 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4042070 con 0x289dbb0 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51d4048980 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51d4047830 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d406f0d0 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 10 client.5132 fetch_fsmap learned FSMap version 38 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x28bb720 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4050020 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 38) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f51d4019070 con 0x289dbb0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 10 client.5132 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 10 client.5132 resolve_mds: resolved ID 'e' to GID 5010 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 20 client.5132 populate_metadata read hostname 'smithi181' 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27b10 0x2b5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 4 client.5132 mds_command: new command op to 5010 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 --> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x294ef50 con 0x2b27b10 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27b10 0x2b5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 10 client.5132 ms_handle_connect on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51e37fe700 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27b10 0x2b5fb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=14b402b11969b10d server_cookie=f537f2c4036a4653 in_seq=0 out_seq=0 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 1 -- 172.21.15.181:0/1387709496 <== mds.0 v2:172.21.15.175:6839/2161152860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x294ef50 con 0x2b27b10 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51ba7fc700 10 client.5132 handle_command_reply: tid=0 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.401+0000 7f51f9188740 1 client.5132 shutdown 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 msgr2=0x2b2b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b2b2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f51d40093f0 tx=0x7f51d406f000).stop 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 shutdown_connections 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:6839/2161152860,v1:172.21.15.175:6841/2161152860] conn(0x2b27b10 0x2b5fb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2b24470 0x2b7dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ba7fc700 0 client.5132 ms_handle_reset on v2:172.21.15.175:6839/2161152860 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x289dbb0 0x2b2b2c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 --2- 172.21.15.181:0/1387709496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2ad9e40 0x2b4ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 >> 172.21.15.181:0/1387709496 conn(0x7f51b0002320 msgr2=0x2b21ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 shutdown_connections 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 1 -- 172.21.15.181:0/1387709496 wait complete. 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51f9188740 20 client.5132 trim_cache size 0 max 16384 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 msgr2=0x7f51bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 0x7f51bc057520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f51e41698d0 tx=0x7f51d8006df0).stop 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 msgr2=0x7f51e414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f51d40091e0 tx=0x7f51d4013000).stop 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 shutdown_connections 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51bc055060 0x7f51bc057520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e415f5b0 0x7f51e4161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51e40615f0 0x7f51e414e4a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 --2- 172.21.15.181:0/3674450934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51e40622a0 0x7f51e4161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 >> 172.21.15.181:0/3674450934 conn(0x7f51e40d69a0 msgr2=0x7f51e4062720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 shutdown_connections 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.405+0000 7f51ea6f7700 1 -- 172.21.15.181:0/3674450934 wait complete. 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:29:08.790 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2020-11-16T22:29:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.837+0000 7f919a559c40 1 Processor -- start 2020-11-16T22:29:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.837+0000 7f919a559c40 1 -- start start 2020-11-16T22:29:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d17fcd0 0x55e61d22ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d19e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61d0f3190 con 0x55e61d17fcd0 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61d0f41f0 con 0x55e61d1800b0 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61d0c2be0 con 0x55e61d195690 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918c360700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58760/0 (socket says 172.21.15.181:58760) 2020-11-16T22:29:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918c360700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38368/0 (socket says 172.21.15.181:38368) 2020-11-16T22:29:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918bb5f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d17fcd0 0x55e61d22ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 -- 172.21.15.181:0/1196573008 learned_addr learned my addr 172.21.15.181:0/1196573008 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 -- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 msgr2=0x55e61d19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 -- 172.21.15.181:0/1196573008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d17fcd0 msgr2=0x55e61d22ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d17fcd0 0x55e61d22ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 -- 172.21.15.181:0/1196573008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61d0e8280 con 0x55e61d1800b0 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918b35e700 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x55e61d134170 tx=0x7f9174010000).ready entity=mon.1 client_cookie=63811941d33f7fbe server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918a35c700 1 -- 172.21.15.181:0/1196573008 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740087e0 con 0x55e61d1800b0 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918a35c700 1 -- 172.21.15.181:0/1196573008 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9174018040 con 0x55e61d1800b0 2020-11-16T22:29:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f918a35c700 1 -- 172.21.15.181:0/1196573008 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917401d440 con 0x55e61d1800b0 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 msgr2=0x55e61d22f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x55e61d134170 tx=0x7f9174010000).stop 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- 172.21.15.181:0/1196573008 shutdown_connections 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d17fcd0 0x55e61d22ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 --2- 172.21.15.181:0/1196573008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d1800b0 0x55e61d22f0a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- 172.21.15.181:0/1196573008 >> 172.21.15.181:0/1196573008 conn(0x55e61d19d1d0 msgr2=0x55e61d19c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- 172.21.15.181:0/1196573008 shutdown_connections 2020-11-16T22:29:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.841+0000 7f919a559c40 1 -- 172.21.15.181:0/1196573008 wait complete. 2020-11-16T22:29:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 Processor -- start 2020-11-16T22:29:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 -- start start 2020-11-16T22:29:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 0x55e61d185360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d1800b0 0x55e61d182230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d182720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61d198430 con 0x55e61d1800b0 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61d197c90 con 0x55e61d17fcd0 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f919a559c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61d197f30 con 0x55e61d195690 2020-11-16T22:29:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f918bb5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 0x55e61d185360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f918bb5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 0x55e61d185360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58762/0 (socket says 172.21.15.181:58762) 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f918bb5f700 1 -- 172.21.15.181:0/2653919894 learned_addr learned my addr 172.21.15.181:0/2653919894 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f918b35e700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d1800b0 0x55e61d182230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.845+0000 7f918c360700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d182720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 -- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 msgr2=0x55e61d182720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d182720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 -- 172.21.15.181:0/2653919894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d1800b0 msgr2=0x55e61d182230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e61d1800b0 0x55e61d182230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 -- 172.21.15.181:0/2653919894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61d0c6620 con 0x55e61d17fcd0 2020-11-16T22:29:08.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918c360700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e61d195690 0x55e61d182720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:08.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918bb5f700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 0x55e61d185360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f918400e7d0 tx=0x7f9184000f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 -- 172.21.15.181:0/2653919894 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184001720 con 0x55e61d17fcd0 2020-11-16T22:29:08.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e61d1981d0 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e61d183a00 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 -- 172.21.15.181:0/2653919894 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184010860 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 -- 172.21.15.181:0/2653919894 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91840109d0 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 -- 172.21.15.181:0/2653919894 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f918401a020 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9170053ce0 0x7f91700561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f9188b59700 1 -- 172.21.15.181:0/2653919894 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f918406cd50 con 0x55e61d17fcd0 2020-11-16T22:29:08.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918b35e700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9170053ce0 0x7f91700561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f919a559c40 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x55e61d2356b0 0x55e61d184650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- osd_op(unknown.0.0:1 5.2 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x55e61d196d50 con 0x55e61d2356b0 2020-11-16T22:29:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918c360700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x55e61d2356b0 0x55e61d184650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.849+0000 7f918b35e700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9170053ce0 0x7f91700561a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x55e61d2284f0 tx=0x7f9174019000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f918c360700 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x55e61d2356b0 0x55e61d184650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f918c360700 1 -- 172.21.15.181:0/2653919894 <== osd.0 v2:172.21.15.181:6808/15203 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f917c001c40 con 0x55e61d2356b0 2020-11-16T22:29:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x55e61d2356b0 msgr2=0x55e61d184650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x55e61d2356b0 0x55e61d184650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9170053ce0 msgr2=0x7f91700561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9170053ce0 0x7f91700561a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x55e61d2284f0 tx=0x7f9174019000).stop 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 msgr2=0x55e61d185360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 --2- 172.21.15.181:0/2653919894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61d17fcd0 0x55e61d185360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f918400e7d0 tx=0x7f9184000f00).stop 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f918c360700 1 -- 172.21.15.181:0/2653919894 reap_dead start 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 shutdown_connections 2020-11-16T22:29:08.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 >> 172.21.15.181:0/2653919894 conn(0x55e61d19d1d0 msgr2=0x55e61d234aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 shutdown_connections 2020-11-16T22:29:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.853+0000 7f919a559c40 1 -- 172.21.15.181:0/2653919894 wait complete. 2020-11-16T22:29:08.839 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:08.840 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2020-11-16T22:29:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 Processor -- start 2020-11-16T22:29:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 -- start start 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 0x564b19fb6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b19fb1fb0 0x564b19f633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b19edabe0 con 0x564b19fb1fb0 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1a0353b0 con 0x564b19fad5a0 2020-11-16T22:29:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.933+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1a042760 con 0x564b1a03ab80 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1376700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 0x564b19fb6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38392/0 (socket says 172.21.15.181:38392) 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1376700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 0x564b19fb6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58780/0 (socket says 172.21.15.181:58780) 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 -- 172.21.15.181:0/3279284720 learned_addr learned my addr 172.21.15.181:0/3279284720 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f0b75700 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b19fb1fb0 0x564b19f633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 -- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 msgr2=0x564b19fb6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 0x564b19fb6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 -- 172.21.15.181:0/3279284720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b19fb1fb0 msgr2=0x564b19f633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b19fb1fb0 0x564b19f633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 -- 172.21.15.181:0/3279284720 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b19f03480 con 0x564b1a03ab80 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32f1b77700 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f32e4005a40 tx=0x7f32e4015000).ready entity=mon.2 client_cookie=f39e8d438e222c0a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32eb7fe700 1 -- 172.21.15.181:0/3279284720 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4008c30 con 0x564b1a03ab80 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32eb7fe700 1 -- 172.21.15.181:0/3279284720 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e4008d90 con 0x564b1a03ab80 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32eb7fe700 1 -- 172.21.15.181:0/3279284720 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4012810 con 0x564b1a03ab80 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 msgr2=0x564b1a034ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f32e4005a40 tx=0x7f32e4015000).stop 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- 172.21.15.181:0/3279284720 shutdown_connections 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b1a03ab80 0x564b1a034ec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fad5a0 0x564b19fb6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/3279284720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b19fb1fb0 0x564b19f633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- 172.21.15.181:0/3279284720 >> 172.21.15.181:0/3279284720 conn(0x564b19fb40d0 msgr2=0x564b19f9cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- 172.21.15.181:0/3279284720 shutdown_connections 2020-11-16T22:29:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- 172.21.15.181:0/3279284720 wait complete. 2020-11-16T22:29:08.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 Processor -- start 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.937+0000 7f32ffd70c40 1 -- start start 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b19fad5a0 0x564b19f9d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 0x564b1a033430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b1a03ab80 0x564b19faf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b19fb5cc0 con 0x564b1a03ab80 2020-11-16T22:29:08.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b19fb1010 con 0x564b19fb1fb0 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b19fb12b0 con 0x564b19fad5a0 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f1376700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b19fad5a0 0x564b19f9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 0x564b1a033430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 0x564b1a033430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58788/0 (socket says 172.21.15.181:58788) 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f1376700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b19fad5a0 0x564b19f9d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38394/0 (socket says 172.21.15.181:38394) 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 -- 172.21.15.181:0/2390558999 learned_addr learned my addr 172.21.15.181:0/2390558999 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:08.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f1b77700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b1a03ab80 0x564b19faf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 -- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b19fad5a0 msgr2=0x564b19f9d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564b19fad5a0 0x564b19f9d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 -- 172.21.15.181:0/2390558999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b1a03ab80 msgr2=0x564b19faf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b1a03ab80 0x564b19faf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 -- 172.21.15.181:0/2390558999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b19f048a0 con 0x564b19fb1fb0 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32f0b75700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 0x564b1a033430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x564b19fb1970 tx=0x7f32dc00c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32e9ffb700 1 -- 172.21.15.181:0/2390558999 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc019070 con 0x564b19fb1fb0 2020-11-16T22:29:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32e9ffb700 1 -- 172.21.15.181:0/2390558999 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc008e10 con 0x564b19fb1fb0 2020-11-16T22:29:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564b19f00280 con 0x564b19fb1fb0 2020-11-16T22:29:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32e9ffb700 1 -- 172.21.15.181:0/2390558999 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00f040 con 0x564b19fb1fb0 2020-11-16T22:29:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.941+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b19ede620 con 0x564b19fb1fb0 2020-11-16T22:29:08.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32e9ffb700 1 -- 172.21.15.181:0/2390558999 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f32dc015470 con 0x564b19fb1fb0 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32e9ffb700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32d4053ce0 0x7f32d40561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32e9ffb700 1 -- 172.21.15.181:0/2390558999 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f32dc06d690 con 0x564b19fb1fb0 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1376700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32d4053ce0 0x7f32d40561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x564b19faf780 0x564b1a039e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- osd_op(unknown.0.0:1 5.2 5:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x564b1a033920 con 0x564b19faf780 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1376700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32d4053ce0 0x7f32d40561a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f32ec005fd0 tx=0x7f32ec007040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1b77700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x564b19faf780 0x564b1a039e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1b77700 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x564b19faf780 0x564b1a039e50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1b77700 1 -- 172.21.15.181:0/2390558999 <== osd.0 v2:172.21.15.181:6808/15203 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f32e4008750 con 0x564b19faf780 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x564b19faf780 msgr2=0x564b1a039e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x564b19faf780 0x564b1a039e50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32d4053ce0 msgr2=0x7f32d40561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32d4053ce0 0x7f32d40561a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f32ec005fd0 tx=0x7f32ec007040).stop 2020-11-16T22:29:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 msgr2=0x564b1a033430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 --2- 172.21.15.181:0/2390558999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b19fb1fb0 0x564b1a033430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x564b19fb1970 tx=0x7f32dc00c040).stop 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32f1b77700 1 -- 172.21.15.181:0/2390558999 reap_dead start 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 shutdown_connections 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 >> 172.21.15.181:0/2390558999 conn(0x564b19fb40d0 msgr2=0x564b19f9c560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 shutdown_connections 2020-11-16T22:29:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:08.945+0000 7f32ffd70c40 1 -- 172.21.15.181:0/2390558999 wait complete. 2020-11-16T22:29:08.934 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:08.934 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2020-11-16T22:29:09.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.021+0000 7fb504909c40 1 Processor -- start 2020-11-16T22:29:09.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.021+0000 7fb504909c40 1 -- start start 2020-11-16T22:29:09.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d5646da60 0x563d56468700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 0x563d56468bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d563c2190 con 0x563d5646da60 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d563c31f0 con 0x563d5646dec0 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d56391be0 con 0x563d564fa4a0 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38412/0 (socket says 172.21.15.181:38412) 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 -- 172.21.15.181:0/3568592430 learned_addr learned my addr 172.21.15.181:0/3568592430 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f570e700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 0x563d56468bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f5f0f700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d5646da60 0x563d56468700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 -- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 msgr2=0x563d56468bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 0x563d56468bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 -- 172.21.15.181:0/3568592430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d5646da60 msgr2=0x563d56468700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d5646da60 0x563d56468700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 -- 172.21.15.181:0/3568592430 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d563b7280 con 0x563d564fa4a0 2020-11-16T22:29:09.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f570e700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 0x563d56468bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:09.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4f6710700 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fb4f00044b0 tx=0x7fb4f0006930).ready entity=mon.2 client_cookie=da7e1231abc44910 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4e7fff700 1 -- 172.21.15.181:0/3568592430 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00062e0 con 0x563d564fa4a0 2020-11-16T22:29:09.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4e7fff700 1 -- 172.21.15.181:0/3568592430 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0004500 con 0x563d564fa4a0 2020-11-16T22:29:09.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb4e7fff700 1 -- 172.21.15.181:0/3568592430 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00048a0 con 0x563d564fa4a0 2020-11-16T22:29:09.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 msgr2=0x563d5646f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fb4f00044b0 tx=0x7fb4f0006930).stop 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- 172.21.15.181:0/3568592430 shutdown_connections 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d564fa4a0 0x563d5646f700 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d5646da60 0x563d56468700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- 172.21.15.181:0/3568592430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646dec0 0x563d56468bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- 172.21.15.181:0/3568592430 >> 172.21.15.181:0/3568592430 conn(0x563d5646e3e0 msgr2=0x563d56450980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- 172.21.15.181:0/3568592430 shutdown_connections 2020-11-16T22:29:09.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- 172.21.15.181:0/3568592430 wait complete. 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 Processor -- start 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- start start 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 0x563d56467d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d5646dec0 0x563d56454dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d564fa4a0 0x563d5641ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d564148a0 con 0x563d564fa4a0 2020-11-16T22:29:09.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d56468700 con 0x563d5646da60 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.025+0000 7fb504909c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d564f93e0 con 0x563d5646dec0 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 0x563d56467d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 0x563d56467d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58806/0 (socket says 172.21.15.181:58806) 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 -- 172.21.15.181:0/1687154836 learned_addr learned my addr 172.21.15.181:0/1687154836 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f570e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d5646dec0 0x563d56454dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f6710700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d564fa4a0 0x563d5641ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 -- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d5646dec0 msgr2=0x563d56454dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d5646dec0 0x563d56454dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 -- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d564fa4a0 msgr2=0x563d5641ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d564fa4a0 0x563d5641ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 -- 172.21.15.181:0/1687154836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d56395620 con 0x563d5646da60 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f6710700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d564fa4a0 0x563d5641ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4f5f0f700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 0x563d56467d50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x563d563aae70 tx=0x7fb4e8013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1687154836 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8012c70 con 0x563d5646da60 2020-11-16T22:29:09.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1687154836 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8012dd0 con 0x563d5646da60 2020-11-16T22:29:09.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563d56468240 con 0x563d5646da60 2020-11-16T22:29:09.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1687154836 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801d690 con 0x563d5646da60 2020-11-16T22:29:09.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d564140c0 con 0x563d5646da60 2020-11-16T22:29:09.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1687154836 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb4e801a040 con 0x563d5646da60 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.029+0000 7fb4e67fc700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4d8053ce0 0x7fb4d80561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f570e700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4d8053ce0 0x7fb4d80561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1687154836 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb4e806ee80 con 0x563d5646da60 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563d564171c0 0x563d5646fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f570e700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4d8053ce0 0x7fb4d80561a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x563d56400810 tx=0x7fb4ec003a90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- osd_op(unknown.0.0:1 5.5 5:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x563d5646c090 con 0x563d564171c0 2020-11-16T22:29:09.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f6710700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563d564171c0 0x563d5646fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f6710700 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563d564171c0 0x563d5646fdd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f6710700 1 -- 172.21.15.181:0/1687154836 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7fb4f00016c0 con 0x563d564171c0 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563d564171c0 msgr2=0x563d5646fdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563d564171c0 0x563d5646fdd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4d8053ce0 msgr2=0x7fb4d80561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4d8053ce0 0x7fb4d80561a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x563d56400810 tx=0x7fb4ec003a90).stop 2020-11-16T22:29:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 msgr2=0x563d56467d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 --2- 172.21.15.181:0/1687154836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d5646da60 0x563d56467d50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x563d563aae70 tx=0x7fb4e8013000).stop 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb4f6710700 1 -- 172.21.15.181:0/1687154836 reap_dead start 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 shutdown_connections 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 >> 172.21.15.181:0/1687154836 conn(0x563d5646e3e0 msgr2=0x563d56450980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 shutdown_connections 2020-11-16T22:29:09.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.033+0000 7fb504909c40 1 -- 172.21.15.181:0/1687154836 wait complete. 2020-11-16T22:29:09.022 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:09.022 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2020-11-16T22:29:09.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.113+0000 7ff6f5e60c40 1 Processor -- start 2020-11-16T22:29:09.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.113+0000 7ff6f5e60c40 1 -- start start 2020-11-16T22:29:09.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 0x5606f3e56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3dbc210 0x5606f3dbc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f3d151f0 con 0x5606f3dbc210 2020-11-16T22:29:09.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f3ce3be0 con 0x5606f3e32bb0 2020-11-16T22:29:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606f3da37f0 con 0x7ff6c8003070 2020-11-16T22:29:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e7466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 0x5606f3e56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e7466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38438/0 (socket says 172.21.15.181:38438) 2020-11-16T22:29:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e7466700 1 -- 172.21.15.181:0/2600472388 learned_addr learned my addr 172.21.15.181:0/2600472388 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e7c67700 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3dbc210 0x5606f3dbc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 -- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 msgr2=0x5606f3da4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 -- 172.21.15.181:0/2600472388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3dbc210 msgr2=0x5606f3dbc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3dbc210 0x5606f3dbc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 -- 172.21.15.181:0/2600472388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f3d0d8a0 con 0x5606f3e32bb0 2020-11-16T22:29:09.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e7466700 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 0x5606f3e56ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff6d000b1e0 tx=0x7ff6d0014000).ready entity=mon.1 client_cookie=93f4b0f3fb4a65d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e5c63700 1 -- 172.21.15.181:0/2600472388 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000b930 con 0x5606f3e32bb0 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e5c63700 1 -- 172.21.15.181:0/2600472388 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d00063a0 con 0x5606f3e32bb0 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6e5c63700 1 -- 172.21.15.181:0/2600472388 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004cd0 con 0x5606f3e32bb0 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 msgr2=0x5606f3e56ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 0x5606f3e56ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff6d000b1e0 tx=0x7ff6d0014000).stop 2020-11-16T22:29:09.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/2600472388 shutdown_connections 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3dbc210 0x5606f3dbc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c8003070 0x5606f3da4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/2600472388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5606f3e32bb0 0x5606f3e56ae0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/2600472388 >> 172.21.15.181:0/2600472388 conn(0x5606f3dbd0d0 msgr2=0x5606f3e324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/2600472388 shutdown_connections 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/2600472388 wait complete. 2020-11-16T22:29:09.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 Processor -- start 2020-11-16T22:29:09.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 -- start start 2020-11-16T22:29:09.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5606f3dbc210 0x5606f3da5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3e32bb0 0x5606f3da5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.117+0000 7ff6f5e60c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 0x5606f3e60160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f3e4cab0 con 0x5606f3e32bb0 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f3e5efe0 con 0x7ff6c8003070 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6f5e60c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606f3e5f240 con 0x5606f3dbc210 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 0x5606f3e60160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 0x5606f3e60160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58836/0 (socket says 172.21.15.181:58836) 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 -- 172.21.15.181:0/702681332 learned_addr learned my addr 172.21.15.181:0/702681332 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7466700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5606f3dbc210 0x5606f3da5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3e32bb0 0x5606f3da5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 -- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5606f3dbc210 msgr2=0x5606f3da5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5606f3dbc210 0x5606f3da5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 -- 172.21.15.181:0/702681332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3e32bb0 msgr2=0x5606f3da5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5606f3e32bb0 0x5606f3da5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 -- 172.21.15.181:0/702681332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f3d09280 con 0x7ff6c8003070 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6e7c67700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 0x5606f3e60160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x5606f3e33190 tx=0x7ff6e000c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6dffff700 1 -- 172.21.15.181:0/702681332 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000f070 con 0x7ff6c8003070 2020-11-16T22:29:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6dffff700 1 -- 172.21.15.181:0/702681332 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0010040 con 0x7ff6c8003070 2020-11-16T22:29:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606f3ce7620 con 0x7ff6c8003070 2020-11-16T22:29:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6dffff700 1 -- 172.21.15.181:0/702681332 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0007cd0 con 0x7ff6c8003070 2020-11-16T22:29:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606f3e5f780 con 0x7ff6c8003070 2020-11-16T22:29:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.121+0000 7ff6dffff700 1 -- 172.21.15.181:0/702681332 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff6e0016620 con 0x7ff6c8003070 2020-11-16T22:29:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6dffff700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6cc053ce0 0x7ff6cc0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6dffff700 1 -- 172.21.15.181:0/702681332 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff6e006d870 con 0x7ff6c8003070 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e7466700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6cc053ce0 0x7ff6cc0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x5606f3e60b40 0x5606f3e4bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 5.1 5:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x5606f3da0e80 con 0x5606f3e60b40 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e7466700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6cc053ce0 0x7ff6cc0561a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7ff6d80059e0 tx=0x7ff6d800c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x5606f3e60b40 0x5606f3e4bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e6c65700 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x5606f3e60b40 0x5606f3e4bf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e6c65700 1 -- 172.21.15.181:0/702681332 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7ff6d0000f00 con 0x5606f3e60b40 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x5606f3e60b40 msgr2=0x5606f3e4bf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x5606f3e60b40 0x5606f3e4bf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6cc053ce0 msgr2=0x7ff6cc0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6cc053ce0 0x7ff6cc0561a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7ff6d80059e0 tx=0x7ff6d800c040).stop 2020-11-16T22:29:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 msgr2=0x5606f3e60160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 --2- 172.21.15.181:0/702681332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c8003070 0x5606f3e60160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x5606f3e33190 tx=0x7ff6e000c040).stop 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6e7c67700 1 -- 172.21.15.181:0/702681332 reap_dead start 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 shutdown_connections 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 >> 172.21.15.181:0/702681332 conn(0x5606f3dbd0d0 msgr2=0x5606f3da39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 shutdown_connections 2020-11-16T22:29:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.125+0000 7ff6f5e60c40 1 -- 172.21.15.181:0/702681332 wait complete. 2020-11-16T22:29:09.114 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:09.114 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2020-11-16T22:29:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 Processor -- start 2020-11-16T22:29:09.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 -- start start 2020-11-16T22:29:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506cc4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506d54450 0x561506cc4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561506c1b1f0 con 0x561506d54450 2020-11-16T22:29:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561506be9be0 con 0x561506ca9050 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.201+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561506cc1ac0 con 0x561506d4a690 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5f58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5f58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58854/0 (socket says 172.21.15.181:58854) 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5f58700 1 -- 172.21.15.181:0/3373053503 learned_addr learned my addr 172.21.15.181:0/3373053503 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506cc4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db6759700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506d54450 0x561506cc4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 -- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 msgr2=0x561506d597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 -- 172.21.15.181:0/3373053503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506d54450 msgr2=0x561506cc4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506d54450 0x561506cc4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 -- 172.21.15.181:0/3373053503 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561506c138a0 con 0x561506d4a690 2020-11-16T22:29:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5f58700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4db5757700 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506cc4770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f4db000a900 tx=0x7f4db0015000).ready entity=mon.2 client_cookie=d773d3960cc458d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4daffff700 1 -- 172.21.15.181:0/3373053503 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0008910 con 0x561506d4a690 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4daffff700 1 -- 172.21.15.181:0/3373053503 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0008a70 con 0x561506d4a690 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4daffff700 1 -- 172.21.15.181:0/3373053503 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db001b6a0 con 0x561506d4a690 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 msgr2=0x561506cc4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506cc4770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f4db000a900 tx=0x7f4db0015000).stop 2020-11-16T22:29:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- 172.21.15.181:0/3373053503 shutdown_connections 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506d54450 0x561506cc4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506ca9050 0x561506d597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/3373053503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506cc4770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- 172.21.15.181:0/3373053503 >> 172.21.15.181:0/3373053503 conn(0x561506cc41d0 msgr2=0x561506cc52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- 172.21.15.181:0/3373053503 shutdown_connections 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- 172.21.15.181:0/3373053503 wait complete. 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 Processor -- start 2020-11-16T22:29:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- start start 2020-11-16T22:29:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506ca9050 0x561506d2e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506d30160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506d54450 0x561506d2c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x561506d2ca70 con 0x561506ca9050 2020-11-16T22:29:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561506d30820 con 0x561506d54450 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.205+0000 7f4dc4952c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561506d30ac0 con 0x561506d4a690 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db6759700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506d54450 0x561506d2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5757700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506d30160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db6759700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506d54450 0x561506d2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58860/0 (socket says 172.21.15.181:58860) 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db6759700 1 -- 172.21.15.181:0/1344675279 learned_addr learned my addr 172.21.15.181:0/1344675279 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506ca9050 0x561506d2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 -- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 msgr2=0x561506d30160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561506d4a690 0x561506d30160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 -- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506d54450 msgr2=0x561506d2c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561506d54450 0x561506d2c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 -- 172.21.15.181:0/1344675279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561506c0f280 con 0x561506ca9050 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4db5f58700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506ca9050 0x561506d2e040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f4da0008ca0 tx=0x7f4da000c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4dae7fc700 1 -- 172.21.15.181:0/1344675279 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da000f040 con 0x561506ca9050 2020-11-16T22:29:09.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561506bed620 con 0x561506ca9050 2020-11-16T22:29:09.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4dae7fc700 1 -- 172.21.15.181:0/1344675279 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0010a70 con 0x561506ca9050 2020-11-16T22:29:09.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561506d31000 con 0x561506ca9050 2020-11-16T22:29:09.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.209+0000 7f4dae7fc700 1 -- 172.21.15.181:0/1344675279 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0010df0 con 0x561506ca9050 2020-11-16T22:29:09.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4dae7fc700 1 -- 172.21.15.181:0/1344675279 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4da0015680 con 0x561506ca9050 2020-11-16T22:29:09.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4dae7fc700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d98053ce0 0x7f4d980561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4db5757700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d98053ce0 0x7f4d980561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4dae7fc700 1 -- 172.21.15.181:0/1344675279 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4da006d8f0 con 0x561506ca9050 2020-11-16T22:29:09.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x561506d2d3c0 0x561506cbd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- osd_op(unknown.0.0:1 5.5 5:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x561506cc3dd0 con 0x561506d2d3c0 2020-11-16T22:29:09.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4db5757700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d98053ce0 0x7f4d980561a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4db001c000 tx=0x7f4db001b000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4db6759700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x561506d2d3c0 0x561506cbd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.213+0000 7f4db6759700 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x561506d2d3c0 0x561506cbd3d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4db6759700 1 -- 172.21.15.181:0/1344675279 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f4da8009040 con 0x561506d2d3c0 2020-11-16T22:29:09.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x561506d2d3c0 msgr2=0x561506cbd3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x561506d2d3c0 0x561506cbd3d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d98053ce0 msgr2=0x7f4d980561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4d98053ce0 0x7f4d980561a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4db001c000 tx=0x7f4db001b000).stop 2020-11-16T22:29:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506ca9050 msgr2=0x561506d2e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 --2- 172.21.15.181:0/1344675279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561506ca9050 0x561506d2e040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f4da0008ca0 tx=0x7f4da000c040).stop 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4db6759700 1 -- 172.21.15.181:0/1344675279 reap_dead start 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 shutdown_connections 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 >> 172.21.15.181:0/1344675279 conn(0x561506cc41d0 msgr2=0x561506d4c350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 shutdown_connections 2020-11-16T22:29:09.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.217+0000 7f4dc4952c40 1 -- 172.21.15.181:0/1344675279 wait complete. 2020-11-16T22:29:09.404 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:09.405 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2020-11-16T22:29:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.493+0000 7f2286320c40 1 Processor -- start 2020-11-16T22:29:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.493+0000 7f2286320c40 1 -- start start 2020-11-16T22:29:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 0x563cf0f71110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 0x563cf0f90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf0f8f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf0ee61f0 con 0x563cf1014b80 2020-11-16T22:29:09.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf0eb4be0 con 0x563cf0f875a0 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cf0f8c390 con 0x563cf0f8bfb0 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277125700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 0x563cf0f90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 0x563cf0f71110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277125700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 0x563cf0f90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38486/0 (socket says 172.21.15.181:38486) 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277125700 1 -- 172.21.15.181:0/4280218713 learned_addr learned my addr 172.21.15.181:0/4280218713 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2278127700 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf0f8f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 -- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 msgr2=0x563cf0f90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 0x563cf0f90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 -- 172.21.15.181:0/4280218713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 msgr2=0x563cf0f8f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf0f8f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 -- 172.21.15.181:0/4280218713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cf0ede8a0 con 0x563cf0f875a0 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2277926700 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 0x563cf0f71110 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2268009bd0 tx=0x7f226800c040).ready entity=mon.1 client_cookie=c890781fe73e7778 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2276123700 1 -- 172.21.15.181:0/4280218713 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800f040 con 0x563cf0f875a0 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2276123700 1 -- 172.21.15.181:0/4280218713 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268008b20 con 0x563cf0f875a0 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2276123700 1 -- 172.21.15.181:0/4280218713 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268005e50 con 0x563cf0f875a0 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 msgr2=0x563cf0f71110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 0x563cf0f71110 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2268009bd0 tx=0x7f226800c040).stop 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- 172.21.15.181:0/4280218713 shutdown_connections 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf0f8f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f875a0 0x563cf0f71110 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 --2- 172.21.15.181:0/4280218713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f8bfb0 0x563cf0f90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- 172.21.15.181:0/4280218713 >> 172.21.15.181:0/4280218713 conn(0x563cf0f8e0d0 msgr2=0x563cf0f77010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- 172.21.15.181:0/4280218713 shutdown_connections 2020-11-16T22:29:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.497+0000 7f2286320c40 1 -- 172.21.15.181:0/4280218713 wait complete. 2020-11-16T22:29:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 Processor -- start 2020-11-16T22:29:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 -- start start 2020-11-16T22:29:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f875a0 0x563cf101e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 0x563cf0f8f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf102f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf0f90cf0 con 0x563cf1014b80 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf101f2e0 con 0x563cf0f8bfb0 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2286320c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cf101f580 con 0x563cf0f875a0 2020-11-16T22:29:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 0x563cf0f8f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 0x563cf0f8f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58878/0 (socket says 172.21.15.181:58878) 2020-11-16T22:29:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 -- 172.21.15.181:0/3557806776 learned_addr learned my addr 172.21.15.181:0/3557806776 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277926700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f875a0 0x563cf101e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2278127700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf102f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 -- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f875a0 msgr2=0x563cf101e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563cf0f875a0 0x563cf101e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 -- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 msgr2=0x563cf102f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf102f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 -- 172.21.15.181:0/3557806776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cf0eda280 con 0x563cf0f8bfb0 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2278127700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563cf1014b80 0x563cf102f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.501+0000 7f2277125700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 0x563cf0f8f230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f226c00c810 tx=0x7f226c0018b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 -- 172.21.15.181:0/3557806776 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c010710 con 0x563cf0f8bfb0 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 -- 172.21.15.181:0/3557806776 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c010870 con 0x563cf0f8bfb0 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563cf0eb8620 con 0x563cf0f8bfb0 2020-11-16T22:29:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 -- 172.21.15.181:0/3557806776 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c010bf0 con 0x563cf0f8bfb0 2020-11-16T22:29:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cf0f911c0 con 0x563cf0f8bfb0 2020-11-16T22:29:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 -- 172.21.15.181:0/3557806776 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f226c019030 con 0x563cf0f8bfb0 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2264053ce0 0x7f22640561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2274920700 1 -- 172.21.15.181:0/3557806776 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f226c00b030 con 0x563cf0f8bfb0 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2277926700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2264053ce0 0x7f22640561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2286320c40 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563cf0f8fd90 0x563cf101cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- osd_op(unknown.0.0:1 5.5 5:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e35) v8 -- 0x563cf101d420 con 0x563cf0f8fd90 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2277926700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2264053ce0 0x7f22640561a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f22680088e0 tx=0x7f2268018000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.505+0000 7f2278127700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563cf0f8fd90 0x563cf101cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2278127700 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563cf0f8fd90 0x563cf101cf30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2278127700 1 -- 172.21.15.181:0/3557806776 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f2270005830 con 0x563cf0f8fd90 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563cf0f8fd90 msgr2=0x563cf101cf30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x563cf0f8fd90 0x563cf101cf30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2264053ce0 msgr2=0x7f22640561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2264053ce0 0x7f22640561a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f22680088e0 tx=0x7f2268018000).stop 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 msgr2=0x563cf0f8f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 --2- 172.21.15.181:0/3557806776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563cf0f8bfb0 0x563cf0f8f230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f226c00c810 tx=0x7f226c0018b0).stop 2020-11-16T22:29:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2278127700 1 -- 172.21.15.181:0/3557806776 reap_dead start 2020-11-16T22:29:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 shutdown_connections 2020-11-16T22:29:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 >> 172.21.15.181:0/3557806776 conn(0x563cf0f8e0d0 msgr2=0x563cf0f90410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 shutdown_connections 2020-11-16T22:29:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.509+0000 7f2286320c40 1 -- 172.21.15.181:0/3557806776 wait complete. 2020-11-16T22:29:09.495 INFO:teuthology.orchestra.run.smithi181.stdout:tag123 2020-11-16T22:29:09.496 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:29:09.496 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:09.534 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:09.551 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:09.568 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:29:09.568 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:29:09.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:09.569 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:29:09.656 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:29:09.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:09.657 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:29:09.674 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:29:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.825+0000 7f3afffff700 1 Processor -- start 2020-11-16T22:29:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.825+0000 7f3afffff700 1 -- start start 2020-11-16T22:29:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 0x7f3b0015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3b0015ff80 0x7f3b00160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000ce460 con 0x7f3b000b2190 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000ce700 con 0x7f3b0015ff80 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000ce9a0 con 0x7f3b000b2570 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afe7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3b0015ff80 0x7f3b00160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afe7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38498/0 (socket says 172.21.15.181:38498) 2020-11-16T22:29:09.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afe7fc700 1 -- 172.21.15.181:0/3842195043 learned_addr learned my addr 172.21.15.181:0/3842195043 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afeffd700 1 --2- 172.21.15.181:0/3842195043 >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 0x7f3b0015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 -- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 msgr2=0x7f3b000cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 -- 172.21.15.181:0/3842195043 >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 msgr2=0x7f3b0015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 --2- 172.21.15.181:0/3842195043 >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 0x7f3b0015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 -- 172.21.15.181:0/3842195043 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000cec40 con 0x7f3b0015ff80 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afeffd700 1 --2- 172.21.15.181:0/3842195043 >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 0x7f3b0015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afe7fc700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3aff7fe700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b0015ff80 0x7f3b00160410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f3af8009c90 tx=0x7f3af80091d0).ready entity=mon.1 client_cookie=8e976168ae9f0cc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afdffb700 1 -- 172.21.15.181:0/3842195043 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800ba30 con 0x7f3b0015ff80 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afdffb700 1 -- 172.21.15.181:0/3842195043 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af800bb90 con 0x7f3b0015ff80 2020-11-16T22:29:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afdffb700 1 -- 172.21.15.181:0/3842195043 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011400 con 0x7f3b0015ff80 2020-11-16T22:29:09.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b0015ff80 msgr2=0x7f3b00160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:09.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b0015ff80 0x7f3b00160410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f3af8009c90 tx=0x7f3af80091d0).stop 2020-11-16T22:29:10.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- 172.21.15.181:0/3842195043 shutdown_connections 2020-11-16T22:29:10.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b0015ff80 0x7f3b00160410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- 172.21.15.181:0/3842195043 >> v2:172.21.15.175:3300/0 conn(0x7f3b000b2190 0x7f3b0015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 --2- 172.21.15.181:0/3842195043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b000b2570 0x7f3b000cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- 172.21.15.181:0/3842195043 >> 172.21.15.181:0/3842195043 conn(0x7f3b000cfe90 msgr2=0x7f3b000d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- 172.21.15.181:0/3842195043 shutdown_connections 2020-11-16T22:29:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.829+0000 7f3afffff700 1 -- 172.21.15.181:0/3842195043 wait complete. 2020-11-16T22:29:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 Processor -- start 2020-11-16T22:29:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- start start 2020-11-16T22:29:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b000b2570 0x7f3b000b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 0x7f3b000b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001650c0 con 0x7f3b000b2570 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00165360 con 0x7f3b000b2190 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00165600 con 0x7f3b0015ff80 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 0x7f3b000b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 0x7f3b000b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38506/0 (socket says 172.21.15.181:38506) 2020-11-16T22:29:10.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58894/0 (socket says 172.21.15.181:58894) 2020-11-16T22:29:10.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3aff7fe700 1 -- 172.21.15.181:0/2390014640 learned_addr learned my addr 172.21.15.181:0/2390014640 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afe7fc700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b000b2570 0x7f3b000b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 -- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 msgr2=0x7f3b000b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 0x7f3b000b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 -- 172.21.15.181:0/2390014640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b000b2570 msgr2=0x7f3b000b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b000b2570 0x7f3b000b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 -- 172.21.15.181:0/2390014640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b001658a0 con 0x7f3b000b2190 2020-11-16T22:29:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afeffd700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3af000b8b0 tx=0x7f3af0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af000f5e0 con 0x7f3b000b2190 2020-11-16T22:29:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af000f740 con 0x7f3b000b2190 2020-11-16T22:29:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af000fae0 con 0x7f3b000b2190 2020-11-16T22:29:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b00151950 con 0x7f3b000b2190 2020-11-16T22:29:10.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.833+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00164d90 con 0x7f3b000b2190 2020-11-16T22:29:10.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3af001c460 con 0x7f3b000b2190 2020-11-16T22:29:10.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00044f60 con 0x7f3b000b2190 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3af6ffd700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 0x7f3adc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3afe7fc700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 0x7f3adc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3af006dca0 con 0x7f3b000b2190 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.837+0000 7f3afe7fc700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 0x7f3adc057520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3b000cf100 tx=0x7f3ae8006eb0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.841+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3af0072070 con 0x7f3b000b2190 2020-11-16T22:29:10.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:09.993+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b0005c4a0 con 0x7f3b000b2190 2020-11-16T22:29:10.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3af6ffd700 1 -- 172.21.15.181:0/2390014640 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3af0043ea0 con 0x7f3b000b2190 2020-11-16T22:29:10.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 msgr2=0x7f3adc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 0x7f3adc057520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3b000cf100 tx=0x7f3ae8006eb0).stop 2020-11-16T22:29:10.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 msgr2=0x7f3b00164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3af000b8b0 tx=0x7f3af0008040).stop 2020-11-16T22:29:10.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 shutdown_connections 2020-11-16T22:29:10.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3adc055060 0x7f3adc057520 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b0015ff80 0x7f3b000b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b000b2190 0x7f3b00164680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 --2- 172.21.15.181:0/2390014640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b000b2570 0x7f3b000b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 >> 172.21.15.181:0/2390014640 conn(0x7f3b000cfe90 msgr2=0x7f3b000600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 shutdown_connections 2020-11-16T22:29:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.005+0000 7f3afffff700 1 -- 172.21.15.181:0/2390014640 wait complete. 2020-11-16T22:29:10.636 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2020-11-16T22:29:10.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.785+0000 7f918c940700 1 Processor -- start 2020-11-16T22:29:10.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.785+0000 7f918c940700 1 -- start start 2020-11-16T22:29:10.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f918c940700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9188145a30 0x7f9188145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f918c940700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f91880816e0 0x7f91880cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:10.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f918c940700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9188061b50 con 0x7f91881492c0 2020-11-16T22:29:10.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f918c940700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880618b0 con 0x7f91880816e0 2020-11-16T22:29:10.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f918c940700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880cb930 con 0x7f9188145a30 2020-11-16T22:29:10.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f91877fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9188145a30 0x7f9188145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f917ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f91881492c0 0x7f9188080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44974/0 (socket says 172.21.15.181:44974) 2020-11-16T22:29:10.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f917ffff700 1 -- 172.21.15.181:0/1830807407 learned_addr learned my addr 172.21.15.181:0/1830807407 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:10.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 --2- 172.21.15.181:0/1830807407 >> v2:172.21.15.175:3300/0 conn(0x7f91880816e0 0x7f91880cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:10.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 -- 172.21.15.181:0/1830807407 >> v2:172.21.15.181:3300/0 conn(0x7f9188145a30 msgr2=0x7f9188145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:10.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 --2- 172.21.15.181:0/1830807407 >> v2:172.21.15.181:3300/0 conn(0x7f9188145a30 0x7f9188145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 -- 172.21.15.181:0/1830807407 >> v1:172.21.15.181:6789/0 conn(0x7f91881492c0 legacy=0x7f9188080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 -- 172.21.15.181:0/1830807407 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880cbbd0 con 0x7f91880816e0 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9187fff700 1 --2- 172.21.15.181:0/1830807407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91880816e0 0x7f91880cc5b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f91880d43d0 tx=0x7f91800091d0).ready entity=mon.0 client_cookie=d9998478752eac5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9186ffd700 1 -- 172.21.15.181:0/1830807407 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000ba70 con 0x7f91880816e0 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9186ffd700 1 -- 172.21.15.181:0/1830807407 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918000bbd0 con 0x7f91880816e0 2020-11-16T22:29:10.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.789+0000 7f9186ffd700 1 -- 172.21.15.181:0/1830807407 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011400 con 0x7f91880816e0 2020-11-16T22:29:10.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- 172.21.15.181:0/1830807407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91880816e0 msgr2=0x7f91880cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:10.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 --2- 172.21.15.181:0/1830807407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91880816e0 0x7f91880cc5b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f91880d43d0 tx=0x7f91800091d0).stop 2020-11-16T22:29:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- 172.21.15.181:0/1830807407 shutdown_connections 2020-11-16T22:29:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 --2- 172.21.15.181:0/1830807407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91880816e0 0x7f91880cc5b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 --2- 172.21.15.181:0/1830807407 >> v2:172.21.15.181:3300/0 conn(0x7f9188145a30 0x7f9188145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- 172.21.15.181:0/1830807407 >> 172.21.15.181:0/1830807407 conn(0x7f91880d68c0 msgr2=0x7f91880d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- 172.21.15.181:0/1830807407 shutdown_connections 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- 172.21.15.181:0/1830807407 wait complete. 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 Processor -- start 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.793+0000 7f918c940700 1 -- start start 2020-11-16T22:29:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91880816e0 0x7f9188161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 0x7f9188147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880d0c50 con 0x7f91881492c0 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188161e20 con 0x7f9188145a30 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91881620c0 con 0x7f91880816e0 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f91877fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91880816e0 0x7f9188161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58910/0 (socket says 172.21.15.181:58910) 2020-11-16T22:29:11.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 -- 172.21.15.181:0/2286670268 learned_addr learned my addr 172.21.15.181:0/2286670268 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:11.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f9187fff700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 0x7f9188147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 -- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91880816e0 msgr2=0x7f9188161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:11.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91880816e0 0x7f9188161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 -- 172.21.15.181:0/2286670268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 msgr2=0x7f9188147cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 0x7f9188147cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 -- 172.21.15.181:0/2286670268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188162360 con 0x7f9188145a30 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f9187fff700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 0x7f9188147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917ffff700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f917000c710 tx=0x7f9170007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:11.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700072e0 con 0x7f9188145a30 2020-11-16T22:29:11.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9170007440 con 0x7f9188145a30 2020-11-16T22:29:11.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170005800 con 0x7f9188145a30 2020-11-16T22:29:11.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9188162600 con 0x7f9188145a30 2020-11-16T22:29:11.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880ce180 con 0x7f9188145a30 2020-11-16T22:29:11.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9188044f60 con 0x7f9188145a30 2020-11-16T22:29:11.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9170010070 con 0x7f9188145a30 2020-11-16T22:29:11.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f917f7fe700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 0x7f9164057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f917006d490 con 0x7f9188145a30 2020-11-16T22:29:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f91877fe700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 0x7f9164057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.801+0000 7f91877fe700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 0x7f9164057520 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f918815db70 tx=0x7f9178006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:10.805+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f91700436f0 con 0x7f9188145a30 2020-11-16T22:29:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.013+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7f9188068e30 con 0x7f9188145a30 2020-11-16T22:29:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.793+0000 7f917f7fe700 1 -- 172.21.15.181:0/2286670268 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v129) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9170046d10 con 0x7f9188145a30 2020-11-16T22:29:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 msgr2=0x7f9164057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 0x7f9164057520 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f918815db70 tx=0x7f9178006df0).stop 2020-11-16T22:29:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 msgr2=0x7f9188147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f917000c710 tx=0x7f9170007c00).stop 2020-11-16T22:29:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 shutdown_connections 2020-11-16T22:29:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9164055060 0x7f9164057520 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91881492c0 0x7f9188147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91880816e0 0x7f9188161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 --2- 172.21.15.181:0/2286670268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9188145a30 0x7f9188147800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 >> 172.21.15.181:0/2286670268 conn(0x7f91880d68c0 msgr2=0x7f918813edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:11.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 shutdown_connections 2020-11-16T22:29:11.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.797+0000 7f918c940700 1 -- 172.21.15.181:0/2286670268 wait complete. 2020-11-16T22:29:11.795 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:29:11.796 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:29:11.796 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2020-11-16T22:29:11.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 Processor -- start 2020-11-16T22:29:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- start start 2020-11-16T22:29:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 0x7f220815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f22080b2570 0x7f22080cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080ce460 con 0x7f22080b2570 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080ce700 con 0x7f22080b2190 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22080ce9a0 con 0x7f220815ff80 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220d24a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 0x7f220815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38528/0 (socket says 172.21.15.181:38528) 2020-11-16T22:29:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220d24a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 0x7f220815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58916/0 (socket says 172.21.15.181:58916) 2020-11-16T22:29:11.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 -- 172.21.15.181:0/588158108 learned_addr learned my addr 172.21.15.181:0/588158108 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:11.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220ca49700 1 --2- 172.21.15.181:0/588158108 >> v2:172.21.15.175:3300/0 conn(0x7f22080b2570 0x7f22080cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:11.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 -- 172.21.15.181:0/588158108 >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 msgr2=0x7f220815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:11.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 --2- 172.21.15.181:0/588158108 >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 0x7f220815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 -- 172.21.15.181:0/588158108 >> v2:172.21.15.175:3300/0 conn(0x7f22080b2570 msgr2=0x7f22080cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:11.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 --2- 172.21.15.181:0/588158108 >> v2:172.21.15.175:3300/0 conn(0x7f22080b2570 0x7f22080cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:11.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 -- 172.21.15.181:0/588158108 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080cec40 con 0x7f220815ff80 2020-11-16T22:29:11.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220da4b700 1 --2- 172.21.15.181:0/588158108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f22080622a0 tx=0x7f2200009290).ready entity=mon.2 client_cookie=b2d2ea1d1ec74800 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:11.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f21fffff700 1 -- 172.21.15.181:0/588158108 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000bb10 con 0x7f220815ff80 2020-11-16T22:29:12.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f21fffff700 1 -- 172.21.15.181:0/588158108 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220000bc70 con 0x7f220815ff80 2020-11-16T22:29:12.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f21fffff700 1 -- 172.21.15.181:0/588158108 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200011400 con 0x7f220815ff80 2020-11-16T22:29:12.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- 172.21.15.181:0/588158108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 msgr2=0x7f2208160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- 172.21.15.181:0/588158108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f22080622a0 tx=0x7f2200009290).stop 2020-11-16T22:29:12.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- 172.21.15.181:0/588158108 shutdown_connections 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- 172.21.15.181:0/588158108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f220815ff80 0x7f2208160410 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- 172.21.15.181:0/588158108 >> v2:172.21.15.181:3300/0 conn(0x7f22080b2190 0x7f220815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 --2- 172.21.15.181:0/588158108 >> v2:172.21.15.175:3300/0 conn(0x7f22080b2570 0x7f22080cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.941+0000 7f220e24c700 1 -- 172.21.15.181:0/588158108 >> 172.21.15.181:0/588158108 conn(0x7f22080cfe90 msgr2=0x7f22080d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.945+0000 7f220e24c700 1 -- 172.21.15.181:0/588158108 shutdown_connections 2020-11-16T22:29:12.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.945+0000 7f220e24c700 1 -- 172.21.15.181:0/588158108 wait complete. 2020-11-16T22:29:12.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.945+0000 7f220e24c700 1 Processor -- start 2020-11-16T22:29:12.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.945+0000 7f220e24c700 1 -- start start 2020-11-16T22:29:12.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22080b2570 0x7f2208160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f220815ff80 0x7f22080b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220815c5c0 con 0x7f220815ff80 2020-11-16T22:29:12.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220815c860 con 0x7f22080b2570 2020-11-16T22:29:12.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220815cb00 con 0x7f22080b2190 2020-11-16T22:29:12.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220ca49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22080b2570 0x7f2208160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38532/0 (socket says 172.21.15.181:38532) 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 -- 172.21.15.181:0/2498230472 learned_addr learned my addr 172.21.15.181:0/2498230472 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220da4b700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f220815ff80 0x7f22080b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 -- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22080b2570 msgr2=0x7f2208160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22080b2570 0x7f2208160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 -- 172.21.15.181:0/2498230472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f220815ff80 msgr2=0x7f22080b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f220815ff80 0x7f22080b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 -- 172.21.15.181:0/2498230472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220815cda0 con 0x7f22080b2190 2020-11-16T22:29:12.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220d24a700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f21f800ebd0 tx=0x7f21f8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:12.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800c2e0 con 0x7f22080b2190 2020-11-16T22:29:12.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f800b8d0 con 0x7f22080b2190 2020-11-16T22:29:12.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220815d040 con 0x7f22080b2190 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800bc90 con 0x7f22080b2190 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.949+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22080b5410 con 0x7f22080b2190 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220815f410 con 0x7f22080b2190 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f21f8012600 con 0x7f22080b2190 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f21fd7fa700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 0x7f21e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f21f806e0b0 con 0x7f22080b2190 2020-11-16T22:29:12.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f220ca49700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 0x7f21e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.953+0000 7f220ca49700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 0x7f21e0057520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2208061040 tx=0x7f2204006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:12.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:11.961+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f21f80442b0 con 0x7f22080b2190 2020-11-16T22:29:12.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.169+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7f2208044f60 con 0x7f22080b2190 2020-11-16T22:29:12.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.805+0000 7f21fd7fa700 1 -- 172.21.15.181:0/2498230472 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v130) v1 ==== 146+0+0 (secure 0 0 0) 0x7f21f80478d0 con 0x7f22080b2190 2020-11-16T22:29:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 msgr2=0x7f21e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 0x7f21e0057520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2208061040 tx=0x7f2204006df0).stop 2020-11-16T22:29:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 msgr2=0x7f2208160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f21f800ebd0 tx=0x7f21f8006df0).stop 2020-11-16T22:29:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 shutdown_connections 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21e0055060 0x7f21e0057520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f220815ff80 0x7f22080b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22080b2190 0x7f2208160530 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 --2- 172.21.15.181:0/2498230472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22080b2570 0x7f2208160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 >> 172.21.15.181:0/2498230472 conn(0x7f22080cfe90 msgr2=0x7f22080600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 shutdown_connections 2020-11-16T22:29:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.809+0000 7f220e24c700 1 -- 172.21.15.181:0/2498230472 wait complete. 2020-11-16T22:29:12.808 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:29:12.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.953+0000 7f5e04986700 1 Processor -- start 2020-11-16T22:29:12.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.953+0000 7f5e04986700 1 -- start start 2020-11-16T22:29:12.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e00142660 0x7f5e00142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 0x7f5e00063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:12.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e00062700 con 0x7f5e00160530 2020-11-16T22:29:12.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000629a0 con 0x7f5e00063620 2020-11-16T22:29:12.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e00062c40 con 0x7f5e00142660 2020-11-16T22:29:12.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e00142660 0x7f5e00142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e00142660 0x7f5e00142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58934/0 (socket says 172.21.15.181:58934) 2020-11-16T22:29:12.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 -- 172.21.15.181:0/4129006917 learned_addr learned my addr 172.21.15.181:0/4129006917 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:12.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dfeffd700 1 --2- 172.21.15.181:0/4129006917 >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 0x7f5e00063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:12.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 -- 172.21.15.181:0/4129006917 >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 msgr2=0x7f5e00063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 --2- 172.21.15.181:0/4129006917 >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 0x7f5e00063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 -- 172.21.15.181:0/4129006917 >> v1:172.21.15.175:6789/0 conn(0x7f5e00160530 legacy=0x7f5e00061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 -- 172.21.15.181:0/4129006917 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00062ee0 con 0x7f5e00142660 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dfeffd700 1 --2- 172.21.15.181:0/4129006917 >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 0x7f5e00063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dff7fe700 1 --2- 172.21.15.181:0/4129006917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00142660 0x7f5e00142a40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5e00066500 tx=0x7f5df0009290).ready entity=mon.1 client_cookie=5d435c9e6ddf4794 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:12.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dfe7fc700 1 -- 172.21.15.181:0/4129006917 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000bc50 con 0x7f5e00142660 2020-11-16T22:29:13.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dfe7fc700 1 -- 172.21.15.181:0/4129006917 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df000bdb0 con 0x7f5e00142660 2020-11-16T22:29:13.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5dfe7fc700 1 -- 172.21.15.181:0/4129006917 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0011610 con 0x7f5e00142660 2020-11-16T22:29:13.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- 172.21.15.181:0/4129006917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00142660 msgr2=0x7f5e00142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 --2- 172.21.15.181:0/4129006917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00142660 0x7f5e00142a40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5e00066500 tx=0x7f5df0009290).stop 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- 172.21.15.181:0/4129006917 shutdown_connections 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 --2- 172.21.15.181:0/4129006917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00142660 0x7f5e00142a40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 --2- 172.21.15.181:0/4129006917 >> v2:172.21.15.175:3300/0 conn(0x7f5e00063620 0x7f5e00063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- 172.21.15.181:0/4129006917 >> 172.21.15.181:0/4129006917 conn(0x7f5e000d2dc0 msgr2=0x7f5e000ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:13.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- 172.21.15.181:0/4129006917 shutdown_connections 2020-11-16T22:29:13.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- 172.21.15.181:0/4129006917 wait complete. 2020-11-16T22:29:13.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 Processor -- start 2020-11-16T22:29:13.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.957+0000 7f5e04986700 1 -- start start 2020-11-16T22:29:13.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e00063620 0x7f5e0013e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:13.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 0x7f5e0013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:13.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 0x7f5e000ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:13.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000d0d00 con 0x7f5e00063620 2020-11-16T22:29:13.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0013cc30 con 0x7f5e00160530 2020-11-16T22:29:13.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0013ced0 con 0x7f5e0015de40 2020-11-16T22:29:13.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 0x7f5e0013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:13.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 0x7f5e0013e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58938/0 (socket says 172.21.15.181:58938) 2020-11-16T22:29:13.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 0x7f5e000ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:13.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e00063620 0x7f5e0013e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfeffd700 1 -- 172.21.15.181:0/3861265692 learned_addr learned my addr 172.21.15.181:0/3861265692 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 -- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 msgr2=0x7f5e0013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 0x7f5e0013e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 -- 172.21.15.181:0/3861265692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e00063620 msgr2=0x7f5e0013e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e00063620 0x7f5e0013e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 -- 172.21.15.181:0/3861265692 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0013d170 con 0x7f5e0015de40 2020-11-16T22:29:13.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5dfffff700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 0x7f5e000ccba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f5df800c710 tx=0x7f5df8007a60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:13.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8013410 con 0x7f5e0015de40 2020-11-16T22:29:13.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8013570 con 0x7f5e0015de40 2020-11-16T22:29:13.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0013d410 con 0x7f5e0015de40 2020-11-16T22:29:13.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8013930 con 0x7f5e0015de40 2020-11-16T22:29:13.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.961+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0013c810 con 0x7f5e0015de40 2020-11-16T22:29:13.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0013ca90 con 0x7f5e0015de40 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5df8019020 con 0x7f5e0015de40 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5df77fe700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 0x7f5dd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5dff7fe700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 0x7f5dd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5df806d860 con 0x7f5e0015de40 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.965+0000 7f5dff7fe700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 0x7f5dd0057520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5e00045680 tx=0x7f5df0016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:13.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:12.973+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5e0013ca90 con 0x7f5e0015de40 2020-11-16T22:29:13.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.125+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f5e0013ca90 con 0x7f5e0015de40 2020-11-16T22:29:13.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5df77fe700 1 -- 172.21.15.181:0/3861265692 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e0013ca90 con 0x7f5e0015de40 2020-11-16T22:29:13.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 msgr2=0x7f5dd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:13.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 0x7f5dd0057520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5e00045680 tx=0x7f5df0016000).stop 2020-11-16T22:29:13.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 msgr2=0x7f5e000ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:13.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 0x7f5e000ccba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f5df800c710 tx=0x7f5df8007a60).stop 2020-11-16T22:29:13.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 shutdown_connections 2020-11-16T22:29:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5dd0055060 0x7f5dd0057520 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0015de40 0x7f5e000ccba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e00063620 0x7f5e0013e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 --2- 172.21.15.181:0/3861265692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e00160530 0x7f5e0013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 >> 172.21.15.181:0/3861265692 conn(0x7f5e000d2dc0 msgr2=0x7f5e00065200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:13.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 shutdown_connections 2020-11-16T22:29:13.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:13.137+0000 7f5e04986700 1 -- 172.21.15.181:0/3861265692 wait complete. 2020-11-16T22:29:13.764 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:29:13.765 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:13.801 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:13.818 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:29:13.834 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:29:13.834 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:29:13.835 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:14.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 Processor -- start 2020-11-16T22:29:14.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- start start 2020-11-16T22:29:14.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e4161350 con 0x7f65e40b21f0 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e41615f0 con 0x7f65e4162510 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4161890 con 0x7f65e4159b40 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38554/0 (socket says 172.21.15.181:38554) 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e97e8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f65e4162510 0x7f65e41629c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45020/0 (socket says 172.21.15.181:45020) 2020-11-16T22:29:14.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 -- 172.21.15.181:0/2753903279 learned_addr learned my addr 172.21.15.181:0/2753903279 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 -- 172.21.15.181:0/2753903279 >> v1:172.21.15.181:6789/0 conn(0x7f65e4162510 legacy=0x7f65e41629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e37fe700 1 -- 172.21.15.181:0/2753903279 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216893949 0 0) 0x7f65e41615f0 con 0x7f65e4162510 2020-11-16T22:29:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 -- 172.21.15.181:0/2753903279 >> v1:172.21.15.175:6789/0 conn(0x7f65e40b21f0 legacy=0x7f65e40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 -- 172.21.15.181:0/2753903279 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4161b30 con 0x7f65e4159b40 2020-11-16T22:29:14.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e8fe7700 1 --2- 172.21.15.181:0/2753903279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f65dc00ba20 tx=0x7f65dc006eb0).ready entity=mon.2 client_cookie=cd7d3a41fb060059 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e37fe700 1 -- 172.21.15.181:0/2753903279 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0098d0 con 0x7f65e4159b40 2020-11-16T22:29:14.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e37fe700 1 -- 172.21.15.181:0/2753903279 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc009a30 con 0x7f65e4159b40 2020-11-16T22:29:14.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e37fe700 1 -- 172.21.15.181:0/2753903279 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc009db0 con 0x7f65e4159b40 2020-11-16T22:29:14.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- 172.21.15.181:0/2753903279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 msgr2=0x7f65e4159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/2753903279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f65dc00ba20 tx=0x7f65dc006eb0).stop 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- 172.21.15.181:0/2753903279 shutdown_connections 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/2753903279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4159b40 0x7f65e4159f20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- 172.21.15.181:0/2753903279 >> 172.21.15.181:0/2753903279 conn(0x7f65e40d4570 msgr2=0x7f65e40d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- 172.21.15.181:0/2753903279 shutdown_connections 2020-11-16T22:29:14.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.025+0000 7f65e9fe9700 1 -- 172.21.15.181:0/2753903279 wait complete. 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 Processor -- start 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- start start 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e40b21f0 0x7f65e40d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 0x7f65e40613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b5930 con 0x7f65e40b21f0 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b53f0 con 0x7f65e4159b40 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b5690 con 0x7f65e4162510 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 0x7f65e40613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 0x7f65e40613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58950/0 (socket says 172.21.15.181:58950) 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38564/0 (socket says 172.21.15.181:38564) 2020-11-16T22:29:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e3fff700 1 -- 172.21.15.181:0/413830162 learned_addr learned my addr 172.21.15.181:0/413830162 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e8fe7700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e40b21f0 0x7f65e40d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 -- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 msgr2=0x7f65e40613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 0x7f65e40613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 -- 172.21.15.181:0/413830162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e40b21f0 msgr2=0x7f65e40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e40b21f0 0x7f65e40d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 -- 172.21.15.181:0/413830162 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40d06f0 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e97e8700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f65d400cb10 tx=0x7f65d4007c60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400ce20 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4005ac0 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40d0990 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4005e80 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.029+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e4170570 con 0x7f65e4162510 2020-11-16T22:29:14.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e40d0e30 con 0x7f65e4162510 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f65d4010030 con 0x7f65e4162510 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e0ff9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 0x7f65c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65d406dcd0 con 0x7f65e4162510 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e8fe7700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 0x7f65c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.033+0000 7f65e8fe7700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 0x7f65c0057520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f65dc00b1b0 tx=0x7f65dc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.041+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f65d4047e60 con 0x7f65e4162510 2020-11-16T22:29:14.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e4044f60 con 0x7f65e4162510 2020-11-16T22:29:14.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e0ff9700 1 -- 172.21.15.181:0/413830162 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f65e40d0e30 con 0x7f65e4162510 2020-11-16T22:29:14.224 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:14.224 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:29:14.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 msgr2=0x7f65c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 0x7f65c0057520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f65dc00b1b0 tx=0x7f65dc015000).stop 2020-11-16T22:29:14.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 msgr2=0x7f65e4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f65d400cb10 tx=0x7f65d4007c60).stop 2020-11-16T22:29:14.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 shutdown_connections 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f65c0055060 0x7f65c0057520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e4162510 0x7f65e4061890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e40b21f0 0x7f65e40d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 --2- 172.21.15.181:0/413830162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e4159b40 0x7f65e40613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 >> 172.21.15.181:0/413830162 conn(0x7f65e40d4570 msgr2=0x7f65e4061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 shutdown_connections 2020-11-16T22:29:14.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.241+0000 7f65e9fe9700 1 -- 172.21.15.181:0/413830162 wait complete. 2020-11-16T22:29:14.228 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:29:14.239 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:14.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.373+0000 7f32ce830700 1 Processor -- start 2020-11-16T22:29:14.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.373+0000 7f32ce830700 1 -- start start 2020-11-16T22:29:14.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32c815f5b0 0x7f32c80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f32c80d1830 0x7f32c80629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c8160e40 con 0x7f32c80615f0 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c81609a0 con 0x7f32c80d1830 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c8160ba0 con 0x7f32c815f5b0 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32c815f5b0 0x7f32c80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32c815f5b0 0x7f32c80d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58958/0 (socket says 172.21.15.181:58958) 2020-11-16T22:29:14.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 -- 172.21.15.181:0/819514452 learned_addr learned my addr 172.21.15.181:0/819514452 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ccdcd700 1 --2- 172.21.15.181:0/819514452 >> v2:172.21.15.175:3300/0 conn(0x7f32c80d1830 0x7f32c80629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 -- 172.21.15.181:0/819514452 >> v2:172.21.15.175:3300/0 conn(0x7f32c80d1830 msgr2=0x7f32c80629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:14.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 --2- 172.21.15.181:0/819514452 >> v2:172.21.15.175:3300/0 conn(0x7f32c80d1830 0x7f32c80629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 -- 172.21.15.181:0/819514452 >> v1:172.21.15.175:6789/0 conn(0x7f32c80615f0 legacy=0x7f32c80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 -- 172.21.15.181:0/819514452 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8060970 con 0x7f32c815f5b0 2020-11-16T22:29:14.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c7fff700 1 --2- 172.21.15.181:0/819514452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c815f5b0 0x7f32c80d1340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f32c8083ad0 tx=0x7f32b8006e50).ready entity=mon.1 client_cookie=f525059c4e968255 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c6ffd700 1 -- 172.21.15.181:0/819514452 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8006030 con 0x7f32c815f5b0 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c6ffd700 1 -- 172.21.15.181:0/819514452 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8006190 con 0x7f32c815f5b0 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32c6ffd700 1 -- 172.21.15.181:0/819514452 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8006510 con 0x7f32c815f5b0 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- 172.21.15.181:0/819514452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c815f5b0 msgr2=0x7f32c80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 --2- 172.21.15.181:0/819514452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c815f5b0 0x7f32c80d1340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f32c8083ad0 tx=0x7f32b8006e50).stop 2020-11-16T22:29:14.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- 172.21.15.181:0/819514452 shutdown_connections 2020-11-16T22:29:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 --2- 172.21.15.181:0/819514452 >> v2:172.21.15.175:3300/0 conn(0x7f32c80d1830 0x7f32c80629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 --2- 172.21.15.181:0/819514452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c815f5b0 0x7f32c80d1340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- 172.21.15.181:0/819514452 >> 172.21.15.181:0/819514452 conn(0x7f32c80d69a0 msgr2=0x7f32c80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- 172.21.15.181:0/819514452 shutdown_connections 2020-11-16T22:29:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.377+0000 7f32ce830700 1 -- 172.21.15.181:0/819514452 wait complete. 2020-11-16T22:29:14.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 Processor -- start 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- start start 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c80615f0 0x7f32c8081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c80d1830 0x7f32c8081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8173020 con 0x7f32c80d1830 2020-11-16T22:29:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81732c0 con 0x7f32c80615f0 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8173560 con 0x7f32c8163870 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c80615f0 0x7f32c8081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38576/0 (socket says 172.21.15.181:38576) 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 -- 172.21.15.181:0/2176075596 learned_addr learned my addr 172.21.15.181:0/2176075596 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32c77fe700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c80d1830 0x7f32c8081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 -- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c80615f0 msgr2=0x7f32c8081170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c80615f0 0x7f32c8081170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 -- 172.21.15.181:0/2176075596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c80d1830 msgr2=0x7f32c8081660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c80d1830 0x7f32c8081660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 -- 172.21.15.181:0/2176075596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8173800 con 0x7f32c8163870 2020-11-16T22:29:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ccdcd700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f32c000ef50 tx=0x7f32c00091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00096f0 con 0x7f32c8163870 2020-11-16T22:29:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000cb50 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00123f0 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c8173aa0 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.381+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80d2ea0 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80d66f0 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f32c0022070 con 0x7f32c8163870 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32bffff700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 0x7f32a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f32c006e5e0 con 0x7f32c8163870 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32c7fff700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 0x7f32a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.385+0000 7f32c7fff700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 0x7f32a4057520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f32c805d5c0 tx=0x7f32b8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.393+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f32c80d66f0 con 0x7f32c8163870 2020-11-16T22:29:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.537+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c80d66f0 con 0x7f32c8163870 2020-11-16T22:29:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32bffff700 1 -- 172.21.15.181:0/2176075596 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f32c80d66f0 con 0x7f32c8163870 2020-11-16T22:29:14.524 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:14.524 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:07.314280+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:14.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 msgr2=0x7f32a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 0x7f32a4057520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f32c805d5c0 tx=0x7f32b8015000).stop 2020-11-16T22:29:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 msgr2=0x7f32c8163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f32c000ef50 tx=0x7f32c00091d0).stop 2020-11-16T22:29:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 shutdown_connections 2020-11-16T22:29:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f32a4055060 0x7f32a4057520 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32c8163870 0x7f32c8163cf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c80615f0 0x7f32c8081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 --2- 172.21.15.181:0/2176075596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c80d1830 0x7f32c8081660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 >> 172.21.15.181:0/2176075596 conn(0x7f32c80d69a0 msgr2=0x7f32c8061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 shutdown_connections 2020-11-16T22:29:14.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.541+0000 7f32ce830700 1 -- 172.21.15.181:0/2176075596 wait complete. 2020-11-16T22:29:14.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:28:03.979538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:28:05.295145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:29:14.539 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 Processor -- start 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- start start 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680c8db0 con 0x7f0868164e10 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680c8330 con 0x7f0868161860 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c85d0 con 0x7f0868161480 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086d2eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38582/0 (socket says 172.21.15.181:38582) 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086d2eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58974/0 (socket says 172.21.15.181:58974) 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 -- 172.21.15.181:0/1698759093 learned_addr learned my addr 172.21.15.181:0/1698759093 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 -- 172.21.15.181:0/1698759093 >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 msgr2=0x7f0868062960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 --2- 172.21.15.181:0/1698759093 >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 -- 172.21.15.181:0/1698759093 >> v1:172.21.15.175:6789/0 conn(0x7f0868164e10 legacy=0x7f0868062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 -- 172.21.15.181:0/1698759093 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680c8870 con 0x7f0868161480 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086d2eb700 1 --2- 172.21.15.181:0/1698759093 >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086caea700 1 --2- 172.21.15.181:0/1698759093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f0858006fd0 tx=0x7f0858006eb0).ready entity=mon.2 client_cookie=dcd931dffb517dc6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f08677fe700 1 -- 172.21.15.181:0/1698759093 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858009810 con 0x7f0868161480 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f08677fe700 1 -- 172.21.15.181:0/1698759093 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858009970 con 0x7f0868161480 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f08677fe700 1 -- 172.21.15.181:0/1698759093 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858009cf0 con 0x7f0868161480 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- 172.21.15.181:0/1698759093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 msgr2=0x7f0868164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 --2- 172.21.15.181:0/1698759093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f0858006fd0 tx=0x7f0858006eb0).stop 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- 172.21.15.181:0/1698759093 shutdown_connections 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 --2- 172.21.15.181:0/1698759093 >> v2:172.21.15.181:3300/0 conn(0x7f0868161860 0x7f0868062960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 --2- 172.21.15.181:0/1698759093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0868161480 0x7f0868164920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- 172.21.15.181:0/1698759093 >> 172.21.15.181:0/1698759093 conn(0x7f08680d2dc0 msgr2=0x7f08680ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- 172.21.15.181:0/1698759093 shutdown_connections 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.685+0000 7f086daec700 1 -- 172.21.15.181:0/1698759093 wait complete. 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 Processor -- start 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 -- start start 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0868161480 0x7f08680ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 0x7f08680ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 0x7f08680b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b67b0 con 0x7f0868161480 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b6a50 con 0x7f0868161860 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b6cf0 con 0x7f08680b5a60 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086d2eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 0x7f08680b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 0x7f08680ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086d2eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 0x7f08680b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38590/0 (socket says 172.21.15.181:38590) 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086d2eb700 1 -- 172.21.15.181:0/42908462 learned_addr learned my addr 172.21.15.181:0/42908462 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086caea700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0868161480 0x7f08680ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 -- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 msgr2=0x7f08680b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 0x7f08680b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 -- 172.21.15.181:0/42908462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0868161480 msgr2=0x7f08680ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0868161480 0x7f08680ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 -- 172.21.15.181:0/42908462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680b6f90 con 0x7f0868161860 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0867fff700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 0x7f08680ce620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f0868061220 tx=0x7f085c007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c007020 con 0x7f0868161860 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c007180 con 0x7f0868161860 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.689+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680b7230 con 0x7f0868161860 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c0064f0 con 0x7f0868161860 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680b63d0 con 0x7f0868161860 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680b6620 con 0x7f0868161860 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f085c010030 con 0x7f0868161860 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f0864ff9700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 0x7f0840057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f085c06d590 con 0x7f0868161860 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f086caea700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 0x7f0840057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.693+0000 7f086caea700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 0x7f0840057520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f085800beb0 tx=0x7f0858015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.697+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f085c0437f0 con 0x7f0868161860 2020-11-16T22:29:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.849+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0868044f60 con 0x7f0868161860 2020-11-16T22:29:14.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.849+0000 7f0864ff9700 1 -- 172.21.15.181:0/42908462 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f085c046e10 con 0x7f0868161860 2020-11-16T22:29:14.834 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:29:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 msgr2=0x7f0840057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 0x7f0840057520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f085800beb0 tx=0x7f0858015000).stop 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 msgr2=0x7f08680ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 0x7f08680ce620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f0868061220 tx=0x7f085c007b00).stop 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 shutdown_connections 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0840055060 0x7f0840057520 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08680b5a60 0x7f08680b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0868161480 0x7f08680ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 --2- 172.21.15.181:0/42908462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0868161860 0x7f08680ce620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 >> 172.21.15.181:0/42908462 conn(0x7f08680d2dc0 msgr2=0x7f0868160320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 shutdown_connections 2020-11-16T22:29:14.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.853+0000 7f086daec700 1 -- 172.21.15.181:0/42908462 wait complete. 2020-11-16T22:29:14.848 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:29:14.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.993+0000 7f7061963700 1 Processor -- start 2020-11-16T22:29:14.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.993+0000 7f7061963700 1 -- start start 2020-11-16T22:29:14.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7061963700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 0x7f705c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7061963700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f705c0615f0 0x7f705c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7061963700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c160e40 con 0x7f705c0622a0 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7061963700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c1609a0 con 0x7f705c0615f0 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7061963700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c160ba0 con 0x7f705c15f5b0 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 0x7f705c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f705c0615f0 0x7f705c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f705c0615f0 0x7f705c0d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58988/0 (socket says 172.21.15.181:58988) 2020-11-16T22:29:14.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 0x7f705c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38596/0 (socket says 172.21.15.181:38596) 2020-11-16T22:29:14.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 -- 172.21.15.181:0/1925216325 learned_addr learned my addr 172.21.15.181:0/1925216325 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 -- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 msgr2=0x7f705c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:14.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 --2- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 0x7f705c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 -- 172.21.15.181:0/1925216325 >> v1:172.21.15.175:6789/0 conn(0x7f705c0622a0 legacy=0x7f705c062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:14.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 -- 172.21.15.181:0/1925216325 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c060970 con 0x7f705c0615f0 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f705a7fc700 1 --2- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c0d1830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f7054009d50 tx=0x7f7054009290).ready entity=mon.1 client_cookie=ee7d88d3ad518e67 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7059ffb700 1 -- 172.21.15.181:0/1925216325 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400bb80 con 0x7f705c0615f0 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7059ffb700 1 -- 172.21.15.181:0/1925216325 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400bce0 con 0x7f705c0615f0 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:14.997+0000 7f7059ffb700 1 -- 172.21.15.181:0/1925216325 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054011610 con 0x7f705c0615f0 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 -- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 msgr2=0x7f705c0d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 --2- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c0d1830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f7054009d50 tx=0x7f7054009290).stop 2020-11-16T22:29:14.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 -- 172.21.15.181:0/1925216325 shutdown_connections 2020-11-16T22:29:14.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 --2- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c15f5b0 0x7f705c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 --2- 172.21.15.181:0/1925216325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c0d1830 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 -- 172.21.15.181:0/1925216325 >> 172.21.15.181:0/1925216325 conn(0x7f705c0d69a0 msgr2=0x7f705c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:14.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 -- 172.21.15.181:0/1925216325 shutdown_connections 2020-11-16T22:29:14.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.001+0000 7f7061963700 1 -- 172.21.15.181:0/1925216325 wait complete. 2020-11-16T22:29:14.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 Processor -- start 2020-11-16T22:29:14.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 -- start start 2020-11-16T22:29:14.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 0x7f705c16f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 0x7f705c16fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c145400 con 0x7f705c15f5b0 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c1701a0 con 0x7f705c0615f0 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7061963700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0bc820 con 0x7f705c0622a0 2020-11-16T22:29:14.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 0x7f705c16f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 0x7f705c16f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38604/0 (socket says 172.21.15.181:38604) 2020-11-16T22:29:14.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705a7fc700 1 -- 172.21.15.181:0/2604216857 learned_addr learned my addr 172.21.15.181:0/2604216857 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:14.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705b7fe700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 0x7f705c16fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 -- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 msgr2=0x7f705c16f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 0x7f705c16f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 -- 172.21.15.181:0/2604216857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 msgr2=0x7f705c16fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 0x7f705c16fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 -- 172.21.15.181:0/2604216857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0d25b0 con 0x7f705c0615f0 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705b7fe700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 0x7f705c16fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f705affd700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c144f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f704c00eba0 tx=0x7f704c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:14.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.005+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00c820 con 0x7f705c0615f0 2020-11-16T22:29:14.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c00c980 con 0x7f705c0615f0 2020-11-16T22:29:14.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c0bbba0 con 0x7f705c0615f0 2020-11-16T22:29:14.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00cd20 con 0x7f705c0615f0 2020-11-16T22:29:14.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c0bc0e0 con 0x7f705c0615f0 2020-11-16T22:29:14.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c069eb0 con 0x7f705c0615f0 2020-11-16T22:29:14.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f704c012600 con 0x7f705c0615f0 2020-11-16T22:29:14.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7042ffd700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 0x7f7038057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:14.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f704c06e140 con 0x7f705c0615f0 2020-11-16T22:29:14.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f705a7fc700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 0x7f7038057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:14.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.009+0000 7f705a7fc700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 0x7f7038057520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f705c082a10 tx=0x7f7054016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.017+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f704c044290 con 0x7f705c0615f0 2020-11-16T22:29:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.165+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f705c1627b0 con 0x7f705c0615f0 2020-11-16T22:29:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.165+0000 7f7042ffd700 1 -- 172.21.15.181:0/2604216857 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f704c0478b0 con 0x7f705c0615f0 2020-11-16T22:29:15.149 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:15.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.165+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 msgr2=0x7f7038057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.165+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 0x7f7038057520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f705c082a10 tx=0x7f7054016000).stop 2020-11-16T22:29:15.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 msgr2=0x7f705c144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c144f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f704c00eba0 tx=0x7f704c008040).stop 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 shutdown_connections 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7038055060 0x7f7038057520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f705c15f5b0 0x7f705c16fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f705c0615f0 0x7f705c144f10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 --2- 172.21.15.181:0/2604216857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f705c0622a0 0x7f705c16f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 >> 172.21.15.181:0/2604216857 conn(0x7f705c0d69a0 msgr2=0x7f705c061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:15.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 shutdown_connections 2020-11-16T22:29:15.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.169+0000 7f7061963700 1 -- 172.21.15.181:0/2604216857 wait complete. 2020-11-16T22:29:15.164 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:29:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 Processor -- start 2020-11-16T22:29:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 -- start start 2020-11-16T22:29:15.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb7c015f5b0 0x7fb7c00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7c00615f0 0x7fb7c00d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c0160e40 con 0x7fb7c015f5b0 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c0160ba0 con 0x7fb7c00615f0 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7c5703700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0060970 con 0x7fb7c00622a0 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38612/0 (socket says 172.21.15.181:38612) 2020-11-16T22:29:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7be7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7c00615f0 0x7fb7c00d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 -- 172.21.15.181:0/2905744862 learned_addr learned my addr 172.21.15.181:0/2905744862 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7beffd700 1 --2- 172.21.15.181:0/2905744862 >> v2:172.21.15.175:3300/0 conn(0x7fb7c015f5b0 0x7fb7c00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 -- 172.21.15.181:0/2905744862 >> v2:172.21.15.181:3300/0 conn(0x7fb7c00615f0 msgr2=0x7fb7c00d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 --2- 172.21.15.181:0/2905744862 >> v2:172.21.15.181:3300/0 conn(0x7fb7c00615f0 0x7fb7c00d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 -- 172.21.15.181:0/2905744862 >> v2:172.21.15.175:3300/0 conn(0x7fb7c015f5b0 msgr2=0x7fb7c00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 --2- 172.21.15.181:0/2905744862 >> v2:172.21.15.175:3300/0 conn(0x7fb7c015f5b0 0x7fb7c00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 -- 172.21.15.181:0/2905744862 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0060c10 con 0x7fb7c00622a0 2020-11-16T22:29:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bf7fe700 1 --2- 172.21.15.181:0/2905744862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb7c00bcb90 tx=0x7fb7b8009290).ready entity=mon.2 client_cookie=6ebf2fa3d8a7304 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bdffb700 1 -- 172.21.15.181:0/2905744862 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800bb10 con 0x7fb7c00622a0 2020-11-16T22:29:15.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bdffb700 1 -- 172.21.15.181:0/2905744862 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b800bc70 con 0x7fb7c00622a0 2020-11-16T22:29:15.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.309+0000 7fb7bdffb700 1 -- 172.21.15.181:0/2905744862 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8011400 con 0x7fb7c00622a0 2020-11-16T22:29:15.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- 172.21.15.181:0/2905744862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 msgr2=0x7fb7c0062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- 172.21.15.181:0/2905744862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb7c00bcb90 tx=0x7fb7b8009290).stop 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- 172.21.15.181:0/2905744862 shutdown_connections 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- 172.21.15.181:0/2905744862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0062700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- 172.21.15.181:0/2905744862 >> v2:172.21.15.175:3300/0 conn(0x7fb7c015f5b0 0x7fb7c00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- 172.21.15.181:0/2905744862 >> v2:172.21.15.181:3300/0 conn(0x7fb7c00615f0 0x7fb7c00d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- 172.21.15.181:0/2905744862 >> 172.21.15.181:0/2905744862 conn(0x7fb7c00d69a0 msgr2=0x7fb7c00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- 172.21.15.181:0/2905744862 shutdown_connections 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- 172.21.15.181:0/2905744862 wait complete. 2020-11-16T22:29:15.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 Processor -- start 2020-11-16T22:29:15.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- start start 2020-11-16T22:29:15.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c00615f0 0x7fb7c0170860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0164d70 con 0x7fb7c00615f0 2020-11-16T22:29:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0165010 con 0x7fb7c015f5b0 2020-11-16T22:29:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7c5703700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c01652b0 con 0x7fb7c00622a0 2020-11-16T22:29:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38618/0 (socket says 172.21.15.181:38618) 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59010/0 (socket says 172.21.15.181:59010) 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 -- 172.21.15.181:0/1474914140 learned_addr learned my addr 172.21.15.181:0/1474914140 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7beffd700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c00615f0 0x7fb7c0170860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 -- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 msgr2=0x7fb7c01730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 -- 172.21.15.181:0/1474914140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c00615f0 msgr2=0x7fb7c0170860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c00615f0 0x7fb7c0170860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 -- 172.21.15.181:0/1474914140 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0165550 con 0x7fb7c00622a0 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7bf7fe700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.313+0000 7fb7be7fc700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fb7b400eba0 tx=0x7fb7b4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400b810 con 0x7fb7c00622a0 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c01657f0 con 0x7fb7c00622a0 2020-11-16T22:29:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c0166420 con 0x7fb7c00622a0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b400b970 con 0x7fb7c00622a0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400bcf0 con 0x7fb7c00622a0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0069eb0 con 0x7fb7c00622a0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb7b4012600 con 0x7fb7c00622a0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 0x7fb790057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7beffd700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 0x7fb790057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb7b406e050 con 0x7fb7c00622a0 2020-11-16T22:29:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.317+0000 7fb7beffd700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 0x7fb790057520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb7b0006fd0 tx=0x7fb7b0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.325+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb7b40442b0 con 0x7fb7c00622a0 2020-11-16T22:29:15.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7c01642f0 con 0x7fb7c00622a0 2020-11-16T22:29:15.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7a6ffd700 1 -- 172.21.15.181:0/1474914140 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb7b40478d0 con 0x7fb7c00622a0 2020-11-16T22:29:15.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 msgr2=0x7fb790057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 0x7fb790057520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb7b0006fd0 tx=0x7fb7b0008040).stop 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 msgr2=0x7fb7c0165f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.477+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fb7b400eba0 tx=0x7fb7b4006df0).stop 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 shutdown_connections 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb790055060 0x7fb790057520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c015f5b0 0x7fb7c01730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c00615f0 0x7fb7c0170860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 --2- 172.21.15.181:0/1474914140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c00622a0 0x7fb7c0165f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:15.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 >> 172.21.15.181:0/1474914140 conn(0x7fb7c00d69a0 msgr2=0x7fb7c00b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:15.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 shutdown_connections 2020-11-16T22:29:15.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:15.481+0000 7fb7c5703700 1 -- 172.21.15.181:0/1474914140 wait complete. 2020-11-16T22:29:15.916 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:15.917 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:29:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 Processor -- start 2020-11-16T22:29:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 -- start start 2020-11-16T22:29:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 0x7fe120140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1200cf390 con 0x7fe1200d07e0 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12015ebb0 con 0x7fe120140550 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200cf5d0 con 0x7fe1200cef10 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 0x7fe120140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe117fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe117fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38624/0 (socket says 172.21.15.181:38624) 2020-11-16T22:29:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe117fff700 1 -- 172.21.15.181:0/3145143715 learned_addr learned my addr 172.21.15.181:0/3145143715 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:16.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.277+0000 7fe11effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 0x7fe120140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59016/0 (socket says 172.21.15.181:59016) 2020-11-16T22:29:16.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe117fff700 1 -- 172.21.15.181:0/3145143715 >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 msgr2=0x7fe120140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:16.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe117fff700 1 --2- 172.21.15.181:0/3145143715 >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 0x7fe120140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe117fff700 1 -- 172.21.15.181:0/3145143715 >> v1:172.21.15.175:6789/0 conn(0x7fe1200d07e0 legacy=0x7fe120063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe117fff700 1 -- 172.21.15.181:0/3145143715 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200cf870 con 0x7fe1200cef10 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe117fff700 1 --2- 172.21.15.181:0/3145143715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe110006fd0 tx=0x7fe110006eb0).ready entity=mon.2 client_cookie=f4ed11601c6ef31e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11e7fc700 1 -- 172.21.15.181:0/3145143715 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110009710 con 0x7fe1200cef10 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11e7fc700 1 -- 172.21.15.181:0/3145143715 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110009870 con 0x7fe1200cef10 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11e7fc700 1 -- 172.21.15.181:0/3145143715 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110009bf0 con 0x7fe1200cef10 2020-11-16T22:29:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- 172.21.15.181:0/3145143715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 msgr2=0x7fe1200d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3145143715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe110006fd0 tx=0x7fe110006eb0).stop 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- 172.21.15.181:0/3145143715 shutdown_connections 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3145143715 >> v2:172.21.15.181:3300/0 conn(0x7fe120140550 0x7fe120140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3145143715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1200cef10 0x7fe1200d02f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- 172.21.15.181:0/3145143715 >> 172.21.15.181:0/3145143715 conn(0x7fe1200d4570 msgr2=0x7fe1200d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- 172.21.15.181:0/3145143715 shutdown_connections 2020-11-16T22:29:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- 172.21.15.181:0/3145143715 wait complete. 2020-11-16T22:29:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 Processor -- start 2020-11-16T22:29:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.281+0000 7fe11ffff700 1 -- start start 2020-11-16T22:29:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200cef10 0x7fe12015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe120140550 0x7fe120160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120162070 con 0x7fe1200cef10 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201712d0 con 0x7fe1200d07e0 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12015f100 con 0x7fe120140550 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59022/0 (socket says 172.21.15.181:59022) 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 -- 172.21.15.181:0/3984533057 learned_addr learned my addr 172.21.15.181:0/3984533057 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11f7fe700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe120140550 0x7fe120160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11effd700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200cef10 0x7fe12015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 -- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe120140550 msgr2=0x7fe120160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe120140550 0x7fe120160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 -- 172.21.15.181:0/3984533057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200cef10 msgr2=0x7fe12015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200cef10 0x7fe12015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 -- 172.21.15.181:0/3984533057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12015f3a0 con 0x7fe1200d07e0 2020-11-16T22:29:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe117fff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe110006ef0 tx=0x7fe110015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11001e460 con 0x7fe1200d07e0 2020-11-16T22:29:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11001e5c0 con 0x7fe1200d07e0 2020-11-16T22:29:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12015f640 con 0x7fe1200d07e0 2020-11-16T22:29:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11001e960 con 0x7fe1200d07e0 2020-11-16T22:29:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12015e380 con 0x7fe1200d07e0 2020-11-16T22:29:16.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.285+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120044f60 con 0x7fe1200d07e0 2020-11-16T22:29:16.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.289+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe110025030 con 0x7fe1200d07e0 2020-11-16T22:29:16.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.289+0000 7fe116ffd700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 0x7fe0f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.289+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe11006fa00 con 0x7fe1200d07e0 2020-11-16T22:29:16.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.289+0000 7fe11effd700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 0x7fe0f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.289+0000 7fe11effd700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 0x7fe0f0057520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fe108006fd0 tx=0x7fe108006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:16.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.297+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe110045c60 con 0x7fe1200d07e0 2020-11-16T22:29:16.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.449+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe12015e600 con 0x7fe1200d07e0 2020-11-16T22:29:16.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe116ffd700 1 -- 172.21.15.181:0/3984533057 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe110023070 con 0x7fe1200d07e0 2020-11-16T22:29:16.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 msgr2=0x7fe0f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 0x7fe0f0057520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fe108006fd0 tx=0x7fe108006df0).stop 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 msgr2=0x7fe120161b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe110006ef0 tx=0x7fe110015000).stop 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 shutdown_connections 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f0055060 0x7fe0f0057520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe120140550 0x7fe120160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200cef10 0x7fe12015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3984533057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1200d07e0 0x7fe120161b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 >> 172.21.15.181:0/3984533057 conn(0x7fe1200d4570 msgr2=0x7fe1200cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:16.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 shutdown_connections 2020-11-16T22:29:16.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.453+0000 7fe11ffff700 1 -- 172.21.15.181:0/3984533057 wait complete. 2020-11-16T22:29:16.684 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:16.685 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.825+0000 7f92bd004700 1 Processor -- start 2020-11-16T22:29:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.825+0000 7f92bd004700 1 -- start start 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92b80629b0 0x7f92b8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b8160e40 con 0x7f92b80615f0 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b8160ba0 con 0x7f92b815f5b0 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b81609a0 con 0x7f92b80629b0 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92b80629b0 0x7f92b8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f92b815f5b0 0x7f92b80d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45102/0 (socket says 172.21.15.181:45102) 2020-11-16T22:29:16.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92b80629b0 0x7f92b8080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59032/0 (socket says 172.21.15.181:59032) 2020-11-16T22:29:16.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b6d9d700 1 -- 172.21.15.181:0/199985857 learned_addr learned my addr 172.21.15.181:0/199985857 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:16.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 -- 172.21.15.181:0/199985857 >> v1:172.21.15.181:6789/0 conn(0x7f92b815f5b0 legacy=0x7f92b80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 -- 172.21.15.181:0/199985857 >> v1:172.21.15.175:6789/0 conn(0x7f92b80615f0 legacy=0x7f92b80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 -- 172.21.15.181:0/199985857 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8060970 con 0x7f92b80629b0 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b759e700 1 --2- 172.21.15.181:0/199985857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80629b0 0x7f92b8080fd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f92b0009c90 tx=0x7f92b00091d0).ready entity=mon.1 client_cookie=e6a6fedc20b3fe52 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b5d9b700 1 -- 172.21.15.181:0/199985857 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b000bbc0 con 0x7f92b80629b0 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b5d9b700 1 -- 172.21.15.181:0/199985857 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b000bd20 con 0x7f92b80629b0 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92b5d9b700 1 -- 172.21.15.181:0/199985857 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011610 con 0x7f92b80629b0 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- 172.21.15.181:0/199985857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80629b0 msgr2=0x7f92b8080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 --2- 172.21.15.181:0/199985857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80629b0 0x7f92b8080fd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f92b0009c90 tx=0x7f92b00091d0).stop 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- 172.21.15.181:0/199985857 shutdown_connections 2020-11-16T22:29:16.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 --2- 172.21.15.181:0/199985857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80629b0 0x7f92b8080fd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:16.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- 172.21.15.181:0/199985857 >> 172.21.15.181:0/199985857 conn(0x7f92b80d69a0 msgr2=0x7f92b80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:16.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- 172.21.15.181:0/199985857 shutdown_connections 2020-11-16T22:29:16.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.829+0000 7f92bd004700 1 -- 172.21.15.181:0/199985857 wait complete. 2020-11-16T22:29:16.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 Processor -- start 2020-11-16T22:29:16.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- start start 2020-11-16T22:29:16.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 0x7f92b80b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b80629b0 0x7f92b80b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:16.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8163010 con 0x7f92b80629b0 2020-11-16T22:29:16.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8161e50 con 0x7f92b80615f0 2020-11-16T22:29:16.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81620f0 con 0x7f92b815f5b0 2020-11-16T22:29:16.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 0x7f92b80b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38648/0 (socket says 172.21.15.181:38648) 2020-11-16T22:29:17.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 -- 172.21.15.181:0/289713223 learned_addr learned my addr 172.21.15.181:0/289713223 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b659c700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b80629b0 0x7f92b80b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 -- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 msgr2=0x7f92b80b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 0x7f92b80b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 -- 172.21.15.181:0/289713223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b80629b0 msgr2=0x7f92b80b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b80629b0 0x7f92b80b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 -- 172.21.15.181:0/289713223 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8162390 con 0x7f92b815f5b0 2020-11-16T22:29:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b6d9d700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 0x7f92b80b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92b759e700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f92b0009e70 tx=0x7f92b0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b000be90 con 0x7f92b815f5b0 2020-11-16T22:29:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0010ae0 con 0x7f92b815f5b0 2020-11-16T22:29:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b8162630 con 0x7f92b815f5b0 2020-11-16T22:29:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0010e60 con 0x7f92b815f5b0 2020-11-16T22:29:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.833+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81617b0 con 0x7f92b815f5b0 2020-11-16T22:29:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f92b0017070 con 0x7f92b815f5b0 2020-11-16T22:29:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b8069eb0 con 0x7f92b815f5b0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92aeffd700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 0x7f9294057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f92b00704d0 con 0x7f92b815f5b0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92b6d9d700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 0x7f9294057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.837+0000 7f92b6d9d700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 0x7f9294057520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f92a80010a0 tx=0x7f92a8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:16.841+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f92b0046730 con 0x7f92b815f5b0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92b81642f0 con 0x7f92b815f5b0 2020-11-16T22:29:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92aeffd700 1 -- 172.21.15.181:0/289713223 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f92b0015070 con 0x7f92b815f5b0 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 msgr2=0x7f9294057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 0x7f9294057520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f92a80010a0 tx=0x7f92a8009040).stop 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 msgr2=0x7f92b81612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f92b0009e70 tx=0x7f92b0016000).stop 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 shutdown_connections 2020-11-16T22:29:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9294055060 0x7f9294057520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b815f5b0 0x7f92b81612c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b80615f0 0x7f92b80b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 --2- 172.21.15.181:0/289713223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b80629b0 0x7f92b80b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 >> 172.21.15.181:0/289713223 conn(0x7f92b80d69a0 msgr2=0x7f92b80622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 shutdown_connections 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.045+0000 7f92bd004700 1 -- 172.21.15.181:0/289713223 wait complete. 2020-11-16T22:29:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 38 2020-11-16T22:29:17.232 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:17.232 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1374602437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1374602437},{"type":"v1","addr":"172.21.15.181:6839","nonce":1374602437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4802,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/77898437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":77898437},{"type":"v1","addr":"172.21.15.175:6835","nonce":77898437}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/3643151253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3643151253},{"type":"v1","addr":"172.21.15.181:6841","nonce":3643151253}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26},{"gid":4968,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/2982355145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2982355145},{"type":"v1","addr":"172.21.15.175:6833","nonce":2982355145}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4978,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2535786512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2535786512},{"type":"v1","addr":"172.21.15.181:6835","nonce":2535786512}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":25},{"gid":4989,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6837/1592469138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1592469138},{"type":"v1","addr":"172.21.15.175:6837","nonce":1592469138}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:28:06.310129+0000","modified":"2020-11-16T22:28:48.813225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5010,"mds_1":5007,"mds_2":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"a","rank":2,"incarnation":34,"state":"up:active","state_seq":9,"addr":"172.21.15.181:6837/4047172210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4047172210},{"type":"v1","addr":"172.21.15.181:6837","nonce":4047172210}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5007":{"gid":5007,"name":"f","rank":1,"incarnation":32,"state":"up:active","state_seq":9,"addr":"172.21.15.175:6840/2720169666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2720169666},{"type":"v1","addr":"172.21.15.175:6840","nonce":2720169666}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5010":{"gid":5010,"name":"e","rank":0,"incarnation":27,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6841/2161152860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":2161152860},{"type":"v1","addr":"172.21.15.175:6841","nonce":2161152860}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:29:17.434 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:17.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.581+0000 7f17727e2700 1 Processor -- start 2020-11-16T22:29:17.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.581+0000 7f17727e2700 1 -- start start 2020-11-16T22:29:17.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f176c0654c0 0x7f176c160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0c85e0 con 0x7f176c15d140 2020-11-16T22:29:17.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c15ff70 con 0x7f176c0654c0 2020-11-16T22:29:17.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0c8840 con 0x7f176c15d520 2020-11-16T22:29:17.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38654/0 (socket says 172.21.15.181:38654) 2020-11-16T22:29:17.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 -- 172.21.15.181:0/2221953131 learned_addr learned my addr 172.21.15.181:0/2221953131 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:17.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1771fe1700 1 --2- 172.21.15.181:0/2221953131 >> v2:172.21.15.175:3300/0 conn(0x7f176c0654c0 0x7f176c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f175bfff700 1 -- 172.21.15.181:0/2221953131 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955165180 0 0) 0x7f176c0c85e0 con 0x7f176c15d140 2020-11-16T22:29:17.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f175bfff700 1 -- 172.21.15.181:0/2221953131 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1754003240 con 0x7f176c15d140 2020-11-16T22:29:17.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 -- 172.21.15.181:0/2221953131 >> v2:172.21.15.175:3300/0 conn(0x7f176c0654c0 msgr2=0x7f176c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:17.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 --2- 172.21.15.181:0/2221953131 >> v2:172.21.15.175:3300/0 conn(0x7f176c0654c0 0x7f176c160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 -- 172.21.15.181:0/2221953131 >> v1:172.21.15.181:6789/0 conn(0x7f176c15d140 legacy=0x7f176c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 -- 172.21.15.181:0/2221953131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0c8ae0 con 0x7f176c15d520 2020-11-16T22:29:17.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f1770fdf700 1 --2- 172.21.15.181:0/2221953131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1768006fd0 tx=0x7f1768006eb0).ready entity=mon.2 client_cookie=8ba4c176aca1ea89 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f175bfff700 1 -- 172.21.15.181:0/2221953131 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768009720 con 0x7f176c15d520 2020-11-16T22:29:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f175bfff700 1 -- 172.21.15.181:0/2221953131 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768009880 con 0x7f176c15d520 2020-11-16T22:29:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f175bfff700 1 -- 172.21.15.181:0/2221953131 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768009c00 con 0x7f176c15d520 2020-11-16T22:29:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- 172.21.15.181:0/2221953131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 msgr2=0x7f176c064fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 --2- 172.21.15.181:0/2221953131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1768006fd0 tx=0x7f1768006eb0).stop 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- 172.21.15.181:0/2221953131 shutdown_connections 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 --2- 172.21.15.181:0/2221953131 >> v2:172.21.15.175:3300/0 conn(0x7f176c0654c0 0x7f176c160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 --2- 172.21.15.181:0/2221953131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c064fd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- 172.21.15.181:0/2221953131 >> 172.21.15.181:0/2221953131 conn(0x7f176c0d2dc0 msgr2=0x7f176c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- 172.21.15.181:0/2221953131 shutdown_connections 2020-11-16T22:29:17.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.585+0000 7f17727e2700 1 -- 172.21.15.181:0/2221953131 wait complete. 2020-11-16T22:29:17.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 Processor -- start 2020-11-16T22:29:17.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 -- start start 2020-11-16T22:29:17.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0654c0 0x7f176c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0d1040 con 0x7f176c0654c0 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cd090 con 0x7f176c15d140 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17727e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0d1230 con 0x7f176c15d520 2020-11-16T22:29:17.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1771fe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59052/0 (socket says 172.21.15.181:59052) 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f17717e0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0654c0 0x7f176c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1771fe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38660/0 (socket says 172.21.15.181:38660) 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 -- 172.21.15.181:0/348494977 learned_addr learned my addr 172.21.15.181:0/348494977 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:17.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 -- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 msgr2=0x7f176c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 -- 172.21.15.181:0/348494977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0654c0 msgr2=0x7f176c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:17.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0654c0 0x7f176c0d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:17.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 -- 172.21.15.181:0/348494977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0d14d0 con 0x7f176c15d140 2020-11-16T22:29:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.589+0000 7f1770fdf700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f176c0c8770 tx=0x7f1768015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768011b40 con 0x7f176c15d140 2020-11-16T22:29:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768011ca0 con 0x7f176c15d140 2020-11-16T22:29:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176801e690 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c0d1770 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c0d1f50 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c16d080 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1768025030 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f176806f4a0 con 0x7f176c15d140 2020-11-16T22:29:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.593+0000 7f17717e0700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:17.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.597+0000 7f17717e0700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f176c0d0730 tx=0x7f175c007970).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:17.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.601+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1768045670 con 0x7f176c15d140 2020-11-16T22:29:17.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.757+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f176c0d1d90 con 0x7f176c15d140 2020-11-16T22:29:18.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.757+0000 7f17597fa700 1 -- 172.21.15.181:0/348494977 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f1768023020 con 0x7f176c15d140 2020-11-16T22:29:18.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 msgr2=0x7f1748057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:18.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f176c0d0730 tx=0x7f175c007970).stop 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 msgr2=0x7f176c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f176c0c8770 tx=0x7f1768015000).stop 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 shutdown_connections 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c15d520 0x7f176c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0654c0 0x7f176c0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 --2- 172.21.15.181:0/348494977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c15d140 0x7f176c0ccba0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 >> 172.21.15.181:0/348494977 conn(0x7f176c0d2dc0 msgr2=0x7f176c160280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 shutdown_connections 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:17.761+0000 7f17727e2700 1 -- 172.21.15.181:0/348494977 wait complete. 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:18.188 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:28:07.314280+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:28:03.979538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:28:05.295145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:18.798 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:29:18.799 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:29:18.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 Processor -- start 2020-11-16T22:29:18.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 -- start start 2020-11-16T22:29:18.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b2900 0x7f50240b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:18.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50240cf6b0 0x7f502415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:18.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240b31d0 con 0x7f502415d2e0 2020-11-16T22:29:18.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240c8910 con 0x7f50240cf6b0 2020-11-16T22:29:18.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.949+0000 7f502914b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240b3460 con 0x7f50240b2900 2020-11-16T22:29:18.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f5023fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b2900 0x7f50240b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:18.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50240cf6b0 0x7f502415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:18.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502894a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f502415d2e0 0x7f50240c7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45132/0 (socket says 172.21.15.181:45132) 2020-11-16T22:29:18.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502894a700 1 -- 172.21.15.181:0/3561439208 learned_addr learned my addr 172.21.15.181:0/3561439208 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:18.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 -- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b2900 msgr2=0x7f50240b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:18.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 --2- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b2900 0x7f50240b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 -- 172.21.15.181:0/3561439208 >> v1:172.21.15.181:6789/0 conn(0x7f502415d2e0 legacy=0x7f50240c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:18.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 -- 172.21.15.181:0/3561439208 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240c8130 con 0x7f50240cf6b0 2020-11-16T22:29:18.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f50237fe700 1 --2- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f502415cdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f500c00ba00 tx=0x7f500c006df0).ready entity=mon.1 client_cookie=a9e2ce302b14e553 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f5022ffd700 1 -- 172.21.15.181:0/3561439208 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c009730 con 0x7f50240cf6b0 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f5022ffd700 1 -- 172.21.15.181:0/3561439208 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c009890 con 0x7f50240cf6b0 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f5022ffd700 1 -- 172.21.15.181:0/3561439208 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c009c10 con 0x7f50240cf6b0 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 -- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 msgr2=0x7f502415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 --2- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f502415cdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f500c00ba00 tx=0x7f500c006df0).stop 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 -- 172.21.15.181:0/3561439208 shutdown_connections 2020-11-16T22:29:18.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 --2- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b2900 0x7f50240b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 --2- 172.21.15.181:0/3561439208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f502415cdf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 -- 172.21.15.181:0/3561439208 >> 172.21.15.181:0/3561439208 conn(0x7f50240d2dc0 msgr2=0x7f50240ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 -- 172.21.15.181:0/3561439208 shutdown_connections 2020-11-16T22:29:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.953+0000 7f502914b700 1 -- 172.21.15.181:0/3561439208 wait complete. 2020-11-16T22:29:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 Processor -- start 2020-11-16T22:29:18.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- start start 2020-11-16T22:29:18.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 0x7f50241733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:18.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502415d2e0 0x7f50240d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50241738c0 con 0x7f50240b2900 2020-11-16T22:29:19.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240d0ce0 con 0x7f50240cf6b0 2020-11-16T22:29:19.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240d0f80 con 0x7f502415d2e0 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59064/0 (socket says 172.21.15.181:59064) 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502894a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502415d2e0 0x7f50240d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 -- 172.21.15.181:0/1466324901 learned_addr learned my addr 172.21.15.181:0/1466324901 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f5023fff700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 0x7f50241733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 -- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502415d2e0 msgr2=0x7f50240d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502415d2e0 0x7f50240d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 -- 172.21.15.181:0/1466324901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 msgr2=0x7f50241733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 0x7f50241733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 -- 172.21.15.181:0/1466324901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240d1220 con 0x7f50240cf6b0 2020-11-16T22:29:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f5023fff700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 0x7f50241733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f50237fe700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f500c006e30 tx=0x7f500c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c011d70 con 0x7f50240cf6b0 2020-11-16T22:29:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240d14c0 con 0x7f50240cf6b0 2020-11-16T22:29:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c01e460 con 0x7f50240cf6b0 2020-11-16T22:29:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5024062770 con 0x7f50240cf6b0 2020-11-16T22:29:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.957+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c01e7e0 con 0x7f50240cf6b0 2020-11-16T22:29:19.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024066c90 con 0x7f50240cf6b0 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f500c01a040 con 0x7f50240cf6b0 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f501bfff700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 0x7f5000057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f5023fff700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 0x7f5000057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f500c06f160 con 0x7f50240cf6b0 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.961+0000 7f5023fff700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 0x7f5000057550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5014006fd0 tx=0x7f5014008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:18.965+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5024066c90 con 0x7f50240cf6b0 2020-11-16T22:29:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.173+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5024066c70 con 0x7f50240cf6b0 2020-11-16T22:29:19.810 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:29:19.828+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:29:19.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.837+0000 7f501bfff700 1 -- 172.21.15.181:0/1466324901 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f500c023090 con 0x7f50240cf6b0 2020-11-16T22:29:19.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 msgr2=0x7f5000057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 0x7f5000057550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5014006fd0 tx=0x7f5014008040).stop 2020-11-16T22:29:19.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 msgr2=0x7f5024062280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f500c006e30 tx=0x7f500c015000).stop 2020-11-16T22:29:19.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 shutdown_connections 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5000055090 0x7f5000057550 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502415d2e0 0x7f50240d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50240b2900 0x7f50241733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 --2- 172.21.15.181:0/1466324901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50240cf6b0 0x7f5024062280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 >> 172.21.15.181:0/1466324901 conn(0x7f50240d2dc0 msgr2=0x7f5024151810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 shutdown_connections 2020-11-16T22:29:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.841+0000 7f502914b700 1 -- 172.21.15.181:0/1466324901 wait complete. 2020-11-16T22:29:19.827 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:29:19.842 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:29:19.847 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:29:19.848 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:29:19.854 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:29:19.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.985+0000 7fb24a95f700 1 Processor -- start 2020-11-16T22:29:19.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.985+0000 7fb24a95f700 1 -- start start 2020-11-16T22:29:19.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a95f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb24413ec50 0x7fb244061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a95f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 0x7fb24415e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a95f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24415d180 con 0x7fb24413e870 2020-11-16T22:29:19.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a95f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24415d810 con 0x7fb24413ec50 2020-11-16T22:29:19.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a95f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24415dab0 con 0x7fb2440615f0 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24995d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb24413e870 0x7fb24415fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45154/0 (socket says 172.21.15.181:45154) 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24995d700 1 -- 172.21.15.181:0/734909857 learned_addr learned my addr 172.21.15.181:0/734909857 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 --2- 172.21.15.181:0/734909857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 0x7fb24415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24915c700 1 --2- 172.21.15.181:0/734909857 >> v2:172.21.15.175:3300/0 conn(0x7fb24413ec50 0x7fb244061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24895b700 1 -- 172.21.15.181:0/734909857 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689399012 0 0) 0x7fb24415d180 con 0x7fb24413e870 2020-11-16T22:29:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24895b700 1 -- 172.21.15.181:0/734909857 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb22c003240 con 0x7fb24413e870 2020-11-16T22:29:19.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 -- 172.21.15.181:0/734909857 >> v2:172.21.15.175:3300/0 conn(0x7fb24413ec50 msgr2=0x7fb244061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:19.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 --2- 172.21.15.181:0/734909857 >> v2:172.21.15.175:3300/0 conn(0x7fb24413ec50 0x7fb244061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 -- 172.21.15.181:0/734909857 >> v1:172.21.15.181:6789/0 conn(0x7fb24413e870 legacy=0x7fb24415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 -- 172.21.15.181:0/734909857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24415dd50 con 0x7fb2440615f0 2020-11-16T22:29:19.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24a15e700 1 --2- 172.21.15.181:0/734909857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 0x7fb24415e730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb23c009c90 tx=0x7fb23c0091d0).ready entity=mon.2 client_cookie=f31bed1f906d751c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24895b700 1 -- 172.21.15.181:0/734909857 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00ba70 con 0x7fb2440615f0 2020-11-16T22:29:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24895b700 1 -- 172.21.15.181:0/734909857 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c00bbd0 con 0x7fb2440615f0 2020-11-16T22:29:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.989+0000 7fb24895b700 1 -- 172.21.15.181:0/734909857 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c011400 con 0x7fb2440615f0 2020-11-16T22:29:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- 172.21.15.181:0/734909857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 msgr2=0x7fb24415e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 --2- 172.21.15.181:0/734909857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 0x7fb24415e730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb23c009c90 tx=0x7fb23c0091d0).stop 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- 172.21.15.181:0/734909857 shutdown_connections 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 --2- 172.21.15.181:0/734909857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2440615f0 0x7fb24415e730 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 --2- 172.21.15.181:0/734909857 >> v2:172.21.15.175:3300/0 conn(0x7fb24413ec50 0x7fb244061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- 172.21.15.181:0/734909857 >> 172.21.15.181:0/734909857 conn(0x7fb2440cf630 msgr2=0x7fb2440cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- 172.21.15.181:0/734909857 shutdown_connections 2020-11-16T22:29:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- 172.21.15.181:0/734909857 wait complete. 2020-11-16T22:29:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 Processor -- start 2020-11-16T22:29:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.993+0000 7fb24a95f700 1 -- start start 2020-11-16T22:29:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb24413e870 0x7fb244159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24416d230 con 0x7fb24413e870 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24416d4d0 con 0x7fb2440615f0 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24416d770 con 0x7fb24413ec50 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24995d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38702/0 (socket says 172.21.15.181:38702) 2020-11-16T22:29:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24995d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59088/0 (socket says 172.21.15.181:59088) 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 -- 172.21.15.181:0/1560657065 learned_addr learned my addr 172.21.15.181:0/1560657065 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24915c700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb24413e870 0x7fb244159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 -- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 msgr2=0x7fb2440b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 -- 172.21.15.181:0/1560657065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb24413e870 msgr2=0x7fb244159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb24413e870 0x7fb244159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:19.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 -- 172.21.15.181:0/1560657065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24416da10 con 0x7fb24413ec50 2020-11-16T22:29:19.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24995d700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:19.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a15e700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb24415dbf0 tx=0x7fb23c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c006ea0 con 0x7fb24413ec50 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c01c6c0 con 0x7fb24413ec50 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24416dcb0 con 0x7fb24413ec50 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c01ca40 con 0x7fb24413ec50 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:19.997+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24416ce20 con 0x7fb24413ec50 2020-11-16T22:29:19.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2440ce920 con 0x7fb24413ec50 2020-11-16T22:29:19.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb23c017070 con 0x7fb24413ec50 2020-11-16T22:29:19.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb231ffb700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 0x7fb21c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:19.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb24995d700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 0x7fb21c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:19.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fb23c0701f0 con 0x7fb24413ec50 2020-11-16T22:29:19.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.001+0000 7fb24995d700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 0x7fb21c057520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb2400055a0 tx=0x7fb240007ab0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:19.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.005+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb2440ce920 con 0x7fb24413ec50 2020-11-16T22:29:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.201+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2440ce920 con 0x7fb24413ec50 2020-11-16T22:29:20.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.853+0000 7fb231ffb700 1 -- 172.21.15.181:0/1560657065 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2440ce920 con 0x7fb24413ec50 2020-11-16T22:29:20.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 msgr2=0x7fb21c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:20.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 0x7fb21c057520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb2400055a0 tx=0x7fb240007ab0).stop 2020-11-16T22:29:20.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 msgr2=0x7fb24415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:20.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fb24415dbf0 tx=0x7fb23c016000).stop 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 shutdown_connections 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb21c055060 0x7fb21c057520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb24413ec50 0x7fb24415a060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2440615f0 0x7fb2440b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 --2- 172.21.15.181:0/1560657065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb24413e870 0x7fb244159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:20.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 >> 172.21.15.181:0/1560657065 conn(0x7fb2440cf630 msgr2=0x7fb2440ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:20.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 shutdown_connections 2020-11-16T22:29:20.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:20.857+0000 7fb24a95f700 1 -- 172.21.15.181:0/1560657065 wait complete. 2020-11-16T22:29:20.856 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:29:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.049+0000 7fb0c71be700 1 Processor -- start 2020-11-16T22:29:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.049+0000 7fb0c71be700 1 -- start start 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c71be700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0c0162e80 0x7fb0c00b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c71be700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0c00b28b0 0x7fb0c00600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c71be700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c00b2d00 con 0x7fb0c0162aa0 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c71be700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00b14a0 con 0x7fb0c00b28b0 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c71be700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00b1740 con 0x7fb0c0162e80 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c4f5a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb0c0162aa0 0x7fb0c0163ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45168/0 (socket says 172.21.15.181:45168) 2020-11-16T22:29:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c4f5a700 1 -- 172.21.15.181:0/2178766720 learned_addr learned my addr 172.21.15.181:0/2178766720 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 --2- 172.21.15.181:0/2178766720 >> v2:172.21.15.181:3300/0 conn(0x7fb0c0162e80 0x7fb0c00b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0c575b700 1 --2- 172.21.15.181:0/2178766720 >> v2:172.21.15.175:3300/0 conn(0x7fb0c00b28b0 0x7fb0c00600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155636736 0 0) 0x7fb0c00b2d00 con 0x7fb0c0162aa0 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ac003240 con 0x7fb0c0162aa0 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 -- 172.21.15.181:0/2178766720 >> v2:172.21.15.175:3300/0 conn(0x7fb0c00b28b0 msgr2=0x7fb0c00600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 --2- 172.21.15.181:0/2178766720 >> v2:172.21.15.175:3300/0 conn(0x7fb0c00b28b0 0x7fb0c00600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 -- 172.21.15.181:0/2178766720 >> v1:172.21.15.181:6789/0 conn(0x7fb0c0162aa0 legacy=0x7fb0c0163ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1487667358 0 0) 0x7fb0ac003240 con 0x7fb0c0162aa0 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 -- 172.21.15.181:0/2178766720 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c00b19e0 con 0x7fb0c0162e80 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bffff700 1 --2- 172.21.15.181:0/2178766720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c0162e80 0x7fb0c00b23c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fb0c0149690 tx=0x7fb0b4006df0).ready entity=mon.1 client_cookie=a763f66c8a0179c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0001860 con 0x7fb0c0162e80 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4011400 con 0x7fb0c0162e80 2020-11-16T22:29:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.053+0000 7fb0bf7fe700 1 -- 172.21.15.181:0/2178766720 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011780 con 0x7fb0c0162e80 2020-11-16T22:29:21.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 -- 172.21.15.181:0/2178766720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c0162e80 msgr2=0x7fb0c00b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2178766720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c0162e80 0x7fb0c00b23c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fb0c0149690 tx=0x7fb0b4006df0).stop 2020-11-16T22:29:21.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 -- 172.21.15.181:0/2178766720 shutdown_connections 2020-11-16T22:29:21.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2178766720 >> v2:172.21.15.175:3300/0 conn(0x7fb0c00b28b0 0x7fb0c00600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2178766720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c0162e80 0x7fb0c00b23c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 -- 172.21.15.181:0/2178766720 >> 172.21.15.181:0/2178766720 conn(0x7fb0c00d35d0 msgr2=0x7fb0c00d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:21.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 -- 172.21.15.181:0/2178766720 shutdown_connections 2020-11-16T22:29:21.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.057+0000 7fb0c71be700 1 -- 172.21.15.181:0/2178766720 wait complete. 2020-11-16T22:29:21.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 Processor -- start 2020-11-16T22:29:21.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 -- start start 2020-11-16T22:29:21.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0162aa0 0x7fb0c00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 0x7fb0c0141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c016f6e0 con 0x7fb0c0162e80 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c016f980 con 0x7fb0c00b28b0 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c71be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c016fc20 con 0x7fb0c0162aa0 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59102/0 (socket says 172.21.15.181:59102) 2020-11-16T22:29:21.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 -- 172.21.15.181:0/2500054271 learned_addr learned my addr 172.21.15.181:0/2500054271 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:21.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0162aa0 0x7fb0c00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c575b700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 0x7fb0c0141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 -- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0162aa0 msgr2=0x7fb0c00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0162aa0 0x7fb0c00cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 -- 172.21.15.181:0/2500054271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 msgr2=0x7fb0c0141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 0x7fb0c0141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 -- 172.21.15.181:0/2500054271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c016fec0 con 0x7fb0c00b28b0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c575b700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 0x7fb0c0141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.061+0000 7fb0c4f5a700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fb0b000ce30 tx=0x7fb0b0001600).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000aac0 con 0x7fb0c00b28b0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b000ac20 con 0x7fb0c00b28b0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c0170160 con 0x7fb0c00b28b0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0007170 con 0x7fb0c00b28b0 2020-11-16T22:29:21.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c016f330 con 0x7fb0c00b28b0 2020-11-16T22:29:21.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c0069eb0 con 0x7fb0c00b28b0 2020-11-16T22:29:21.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb0b001b060 con 0x7fb0c00b28b0 2020-11-16T22:29:21.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 0x7fb098057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bffff700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 0x7fb098057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fb0b006de50 con 0x7fb0c00b28b0 2020-11-16T22:29:21.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.065+0000 7fb0bffff700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 0x7fb098057520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fb0b4006e30 tx=0x7fb0b4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:21.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.069+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb0b0043fb0 con 0x7fb0c00b28b0 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.225+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0c0062210 con 0x7fb0c00b28b0 2020-11-16T22:29:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.861+0000 7fb0bcff9700 1 -- 172.21.15.181:0/2500054271 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0b0019070 con 0x7fb0c00b28b0 2020-11-16T22:29:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 msgr2=0x7fb098057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 0x7fb098057520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fb0b4006e30 tx=0x7fb0b4015000).stop 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 msgr2=0x7fb0c00cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fb0b000ce30 tx=0x7fb0b0001600).stop 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 shutdown_connections 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb098055060 0x7fb098057520 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0162e80 0x7fb0c0141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b28b0 0x7fb0c00cd000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 --2- 172.21.15.181:0/2500054271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0162aa0 0x7fb0c00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 >> 172.21.15.181:0/2500054271 conn(0x7fb0c00d35d0 msgr2=0x7fb0c0142d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 shutdown_connections 2020-11-16T22:29:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:21.865+0000 7fb0c71be700 1 -- 172.21.15.181:0/2500054271 wait complete. 2020-11-16T22:29:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:29:21.863 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:29:21.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 Processor -- start 2020-11-16T22:29:21.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- start start 2020-11-16T22:29:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03ec0b50d0 0x7f03ec0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec0b4800 con 0x7f03ec0b54b0 2020-11-16T22:29:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec15d0f0 con 0x7f03ec0b50d0 2020-11-16T22:29:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15d390 con 0x7f03ec15c440 2020-11-16T22:29:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03ebfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03ec0b50d0 0x7f03ec0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38724/0 (socket says 172.21.15.181:38724) 2020-11-16T22:29:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 -- 172.21.15.181:0/921223538 learned_addr learned my addr 172.21.15.181:0/921223538 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 -- 172.21.15.181:0/921223538 >> v2:172.21.15.181:3300/0 conn(0x7f03ec0b50d0 msgr2=0x7f03ec0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 --2- 172.21.15.181:0/921223538 >> v2:172.21.15.181:3300/0 conn(0x7f03ec0b50d0 0x7f03ec0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 -- 172.21.15.181:0/921223538 >> v1:172.21.15.175:6789/0 conn(0x7f03ec0b54b0 legacy=0x7f03ec0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 -- 172.21.15.181:0/921223538 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0cd950 con 0x7f03ec15c440 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f0831700 1 --2- 172.21.15.181:0/921223538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f03e4009d50 tx=0x7f03e4009290).ready entity=mon.2 client_cookie=2b893bc5cf42078b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03eaffd700 1 -- 172.21.15.181:0/921223538 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e400bbc0 con 0x7f03ec15c440 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03eaffd700 1 -- 172.21.15.181:0/921223538 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e400bd20 con 0x7f03ec15c440 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03eaffd700 1 -- 172.21.15.181:0/921223538 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4011610 con 0x7f03ec15c440 2020-11-16T22:29:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- 172.21.15.181:0/921223538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 msgr2=0x7f03ec15c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 --2- 172.21.15.181:0/921223538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f03e4009d50 tx=0x7f03e4009290).stop 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- 172.21.15.181:0/921223538 shutdown_connections 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 --2- 172.21.15.181:0/921223538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec15c440 0x7f03ec15c8d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 --2- 172.21.15.181:0/921223538 >> v2:172.21.15.181:3300/0 conn(0x7f03ec0b50d0 0x7f03ec0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.013+0000 7f03f2294700 1 -- 172.21.15.181:0/921223538 >> 172.21.15.181:0/921223538 conn(0x7f03ec0cf770 msgr2=0x7f03ec0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.017+0000 7f03f2294700 1 -- 172.21.15.181:0/921223538 shutdown_connections 2020-11-16T22:29:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.017+0000 7f03f2294700 1 -- 172.21.15.181:0/921223538 wait complete. 2020-11-16T22:29:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.017+0000 7f03f2294700 1 Processor -- start 2020-11-16T22:29:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.017+0000 7f03f2294700 1 -- start start 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03ec0b50d0 0x7f03ec16e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec15c440 0x7f03ec0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec16f2e0 con 0x7f03ec0b50d0 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec16f580 con 0x7f03ec15c440 2020-11-16T22:29:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0d0d50 con 0x7f03ec0b54b0 2020-11-16T22:29:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38726/0 (socket says 172.21.15.181:38726) 2020-11-16T22:29:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 -- 172.21.15.181:0/3196600522 learned_addr learned my addr 172.21.15.181:0/3196600522 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f0831700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec15c440 0x7f03ec0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03ebfff700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03ec0b50d0 0x7f03ec16e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 -- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec15c440 msgr2=0x7f03ec0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec15c440 0x7f03ec0c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 -- 172.21.15.181:0/3196600522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03ec0b50d0 msgr2=0x7f03ec16e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03ec0b50d0 0x7f03ec16e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 -- 172.21.15.181:0/3196600522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0d0ff0 con 0x7f03ec0b54b0 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03eb7fe700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f03e000c6e0 tx=0x7f03e0007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0007020 con 0x7f03ec0b54b0 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0007180 con 0x7f03ec0b54b0 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec0d1290 con 0x7f03ec0b54b0 2020-11-16T22:29:22.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00064f0 con 0x7f03ec0b54b0 2020-11-16T22:29:22.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.021+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec16ef30 con 0x7f03ec0b54b0 2020-11-16T22:29:22.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec0642c0 con 0x7f03ec0b54b0 2020-11-16T22:29:22.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f03e0010070 con 0x7f03ec0b54b0 2020-11-16T22:29:22.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03cbfff700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 0x7f03c4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:22.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f03e006cda0 con 0x7f03ec0b54b0 2020-11-16T22:29:22.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03ebfff700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 0x7f03c4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:22.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.025+0000 7f03ebfff700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 0x7f03c4057550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f03dc006fd0 tx=0x7f03dc006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:22.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.029+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f03ec0642c0 con 0x7f03ec0b54b0 2020-11-16T22:29:22.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.181+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03ec0b4800 con 0x7f03ec0b54b0 2020-11-16T22:29:22.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.869+0000 7f03cbfff700 1 -- 172.21.15.181:0/3196600522 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f03ec0b4800 con 0x7f03ec0b54b0 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 msgr2=0x7f03c4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 0x7f03c4057550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f03dc006fd0 tx=0x7f03dc006df0).stop 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 msgr2=0x7f03ec0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f03e000c6e0 tx=0x7f03e0007c50).stop 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 shutdown_connections 2020-11-16T22:29:22.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f03c4055090 0x7f03c4057550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec15c440 0x7f03ec0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03ec0b50d0 0x7f03ec16e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 --2- 172.21.15.181:0/3196600522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03ec0b54b0 0x7f03ec0c6910 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 >> 172.21.15.181:0/3196600522 conn(0x7f03ec0cf770 msgr2=0x7f03ec0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 shutdown_connections 2020-11-16T22:29:22.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:22.873+0000 7f03f2294700 1 -- 172.21.15.181:0/3196600522 wait complete. 2020-11-16T22:29:22.860 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:29:22.875 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:29:22.875 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:29:22.875 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:29:22.875 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.876 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:29:22.876 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:29:22.876 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:29:22.876 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.877 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:29:22.877 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:29:22.877 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:29:22.877 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.877 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:29:22.878 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:29:22.878 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:29:22.878 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.878 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:29:22.878 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:29:22.879 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:29:22.879 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.879 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:29:22.879 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:29:22.879 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:29:22.880 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.880 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:29:22.880 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:29:22.880 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:29:22.880 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.881 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:29:22.881 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:29:22.881 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:29:22.881 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.882 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:29:22.882 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:29:22.882 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:29:22.882 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:29:22.890 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:29:22.890 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:29:22.891 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:29:22.891 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:29:22.892 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:29:22.892 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:29:22.893 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:29:22.893 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:29:22.895 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:29:22.895 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:29:22.895 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:29:22.896 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:29:22.896 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:29:22.896 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:29:22.897 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:29:22.897 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:29:22.898 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:29:22.899 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:29:22.935 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:29:22.936 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.937 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:29:22.937 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:29:22.937 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:29:22.938 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.938 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.938 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.943 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:29:22.943 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:29:22.943 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:29:22.944 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:29:22.944 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.945 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.945 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.945 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.946 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:29:22.946 DEBUG:teuthology.parallel:result is None 2020-11-16T22:29:22.946 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:29:23.000 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:29:23.003 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:29:23.004 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:29:23.008 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:29:23.010 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:29:23.019 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:29:23.023 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:29:23.025 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:29:23.030 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:29:23.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 Processor -- start 2020-11-16T22:29:23.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 -- start start 2020-11-16T22:29:23.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56d8161450 0x7f56d80c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d8142ee0 con 0x7f56d80b2ff0 2020-11-16T22:29:23.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d80b3830 con 0x7f56d8161450 2020-11-16T22:29:23.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56dc80e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8161a00 con 0x7f56d80b33d0 2020-11-16T22:29:23.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56d6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56cffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56d8161450 0x7f56d80c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56d6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8163550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38786/0 (socket says 172.21.15.181:38786) 2020-11-16T22:29:23.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.113+0000 7f56d6d9d700 1 -- 172.21.15.181:0/2124627335 learned_addr learned my addr 172.21.15.181:0/2124627335 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:23.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56cffff700 1 -- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 msgr2=0x7f56d8163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:23.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56cffff700 1 --2- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8163550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56cffff700 1 -- 172.21.15.181:0/2124627335 >> v1:172.21.15.175:6789/0 conn(0x7f56d80b2ff0 legacy=0x7f56d8162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56cffff700 1 -- 172.21.15.181:0/2124627335 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8161ca0 con 0x7f56d8161450 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56cffff700 1 --2- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80c74e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f56c000b9a0 tx=0x7f56c0006df0).ready entity=mon.1 client_cookie=c0c35e6a31ccdedb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56d5d9b700 1 -- 172.21.15.181:0/2124627335 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c00097b0 con 0x7f56d8161450 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56d5d9b700 1 -- 172.21.15.181:0/2124627335 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0009910 con 0x7f56d8161450 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56d5d9b700 1 -- 172.21.15.181:0/2124627335 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0009c90 con 0x7f56d8161450 2020-11-16T22:29:23.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 msgr2=0x7f56d80c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 --2- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80c74e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f56c000b9a0 tx=0x7f56c0006df0).stop 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- 172.21.15.181:0/2124627335 shutdown_connections 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 --2- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8163550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 --2- 172.21.15.181:0/2124627335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80c74e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- 172.21.15.181:0/2124627335 >> 172.21.15.181:0/2124627335 conn(0x7f56d8060b70 msgr2=0x7f56d80d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- 172.21.15.181:0/2124627335 shutdown_connections 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- 172.21.15.181:0/2124627335 wait complete. 2020-11-16T22:29:23.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2020-11-16T22:29:23.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i is failed for ~0s 2020-11-16T22:29:23.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 Processor -- start 2020-11-16T22:29:23.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.117+0000 7f56dc80e700 1 -- start start 2020-11-16T22:29:23.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 0x7f56d8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8172ad0 con 0x7f56d80b2ff0 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8172d70 con 0x7f56d8161450 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8173010 con 0x7f56d80b33d0 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59182/0 (socket says 172.21.15.181:59182) 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 -- 172.21.15.181:0/1010728241 learned_addr learned my addr 172.21.15.181:0/1010728241 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:23.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d659c700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 0x7f56d8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 -- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 msgr2=0x7f56d8145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 -- 172.21.15.181:0/1010728241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 msgr2=0x7f56d8157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 0x7f56d8157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 -- 172.21.15.181:0/1010728241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d81732b0 con 0x7f56d8161450 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d659c700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 0x7f56d8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:23.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56d6d9d700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f56d0009360 tx=0x7f56d0009290).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:23.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000ca60 con 0x7f56d8161450 2020-11-16T22:29:23.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d000cbc0 con 0x7f56d8161450 2020-11-16T22:29:23.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d8173550 con 0x7f56d8161450 2020-11-16T22:29:23.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00123f0 con 0x7f56d8161450 2020-11-16T22:29:23.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.121+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d8158080 con 0x7f56d8161450 2020-11-16T22:29:23.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d8069eb0 con 0x7f56d8161450 2020-11-16T22:29:23.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f56d0022070 con 0x7f56d8161450 2020-11-16T22:29:23.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56ce7fc700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 0x7f56b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:23.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f56d006e260 con 0x7f56d8161450 2020-11-16T22:29:23.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56d659c700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 0x7f56b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:23.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.125+0000 7f56d659c700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 0x7f56b4057520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f56c80010d0 tx=0x7f56c8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:23.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.133+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f56d00446f0 con 0x7f56d8161450 2020-11-16T22:29:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:23.297+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f56d80d3360 con 0x7f56d8161450 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.077+0000 7f56ce7fc700 1 -- 172.21.15.181:0/1010728241 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f56d0047d10 con 0x7f56d8161450 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 msgr2=0x7f56b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 0x7f56b4057520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f56c80010d0 tx=0x7f56c8009040).stop 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 msgr2=0x7f56d80cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f56d0009360 tx=0x7f56d0009290).stop 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 shutdown_connections 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f56b4055060 0x7f56b4057520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56d8161450 0x7f56d80cf300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56d80b2ff0 0x7f56d8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 --2- 172.21.15.181:0/1010728241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56d80b33d0 0x7f56d8145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 >> 172.21.15.181:0/1010728241 conn(0x7f56d8060b70 msgr2=0x7f56d8142140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 shutdown_connections 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.081+0000 7f56dc80e700 1 -- 172.21.15.181:0/1010728241 wait complete. 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 Processor -- start 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 -- start start 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5348061680 0x7f53480cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f534815fa10 0x7f534815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534815cda0 con 0x7f53480612a0 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534815d040 con 0x7f534815fa10 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f5350574700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534815d2e0 con 0x7f5348061680 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534eb11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f534815fa10 0x7f534815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534e310700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f53480612a0 0x7f53480ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45262/0 (socket says 172.21.15.181:45262) 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534db0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5348061680 0x7f53480cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534e310700 1 -- 172.21.15.181:0/1818583392 learned_addr learned my addr 172.21.15.181:0/1818583392 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534d30e700 1 -- 172.21.15.181:0/1818583392 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706793868 0 0) 0x7f534815cda0 con 0x7f53480612a0 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534d30e700 1 -- 172.21.15.181:0/1818583392 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5330003240 con 0x7f53480612a0 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534eb11700 1 -- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5348061680 msgr2=0x7f53480cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534eb11700 1 --2- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5348061680 0x7f53480cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534eb11700 1 -- 172.21.15.181:0/1818583392 >> v1:172.21.15.181:6789/0 conn(0x7f53480612a0 legacy=0x7f53480ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.217+0000 7f534eb11700 1 -- 172.21.15.181:0/1818583392 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480cdd70 con 0x7f534815fa10 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f534eb11700 1 --2- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534815fa10 0x7f534815fea0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f5340009cc0 tx=0x7f53400091d0).ready entity=mon.1 client_cookie=b2e38426a266ab50 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f534d30e700 1 -- 172.21.15.181:0/1818583392 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000b9e0 con 0x7f534815fa10 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f534d30e700 1 -- 172.21.15.181:0/1818583392 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534000bb40 con 0x7f534815fa10 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f534d30e700 1 -- 172.21.15.181:0/1818583392 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340011400 con 0x7f534815fa10 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534815fa10 msgr2=0x7f534815fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 --2- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534815fa10 0x7f534815fea0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f5340009cc0 tx=0x7f53400091d0).stop 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- 172.21.15.181:0/1818583392 shutdown_connections 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 --2- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534815fa10 0x7f534815fea0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 --2- 172.21.15.181:0/1818583392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5348061680 0x7f53480cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- 172.21.15.181:0/1818583392 >> 172.21.15.181:0/1818583392 conn(0x7f53480cf890 msgr2=0x7f53480cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- 172.21.15.181:0/1818583392 shutdown_connections 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- 172.21.15.181:0/1818583392 wait complete. 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 Processor -- start 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.221+0000 7f5350574700 1 -- start start 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 0x7f534815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 0x7f53480d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c7df0 con 0x7f534815fa10 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c8090 con 0x7f5348061680 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c8330 con 0x7f53480612a0 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534e310700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 0x7f534815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59198/0 (socket says 172.21.15.181:59198) 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534e310700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 0x7f534815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38806/0 (socket says 172.21.15.181:38806) 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 -- 172.21.15.181:0/2212168738 learned_addr learned my addr 172.21.15.181:0/2212168738 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534eb11700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 0x7f53480d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 -- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 msgr2=0x7f534815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 0x7f534815a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 -- 172.21.15.181:0/2212168738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 msgr2=0x7f53480d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 0x7f53480d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 -- 172.21.15.181:0/2212168738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480c85d0 con 0x7f5348061680 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534eb11700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 0x7f53480d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f534db0f700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f534400fbd0 tx=0x7f5344009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f5348061680 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53440072e0 con 0x7f5348061680 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53480c8870 con 0x7f5348061680 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400c360 con 0x7f5348061680 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534815a620 con 0x7f5348061680 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534815a8a0 con 0x7f5348061680 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.225+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f534400d9b0 con 0x7f5348061680 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.229+0000 7f533a7fc700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 0x7f5320057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.229+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f534406db60 con 0x7f5348061680 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.229+0000 7f534e310700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 0x7f5320057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.229+0000 7f534e310700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 0x7f5320057520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f533c0055a0 tx=0x7f533c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.237+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5344043fe0 con 0x7f5348061680 2020-11-16T22:29:24.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.441+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f53480cdf50 con 0x7f5348061680 2020-11-16T22:29:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f533a7fc700 1 -- 172.21.15.181:0/2212168738 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5978 (secure 0 0 0) 0x7f5344047600 con 0x7f5348061680 2020-11-16T22:29:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:24.427 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:29:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:29:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:29:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:29:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:29:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:29:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:29:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:29:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:29:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:29:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:29:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 msgr2=0x7f5320057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 0x7f5320057520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f533c0055a0 tx=0x7f533c009040).stop 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 msgr2=0x7f53480d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f534400fbd0 tx=0x7f5344009040).stop 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 shutdown_connections 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5320055060 0x7f5320057520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f534815fa10 0x7f53480d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53480612a0 0x7f534815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 --2- 172.21.15.181:0/2212168738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5348061680 0x7f53480d19d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 >> 172.21.15.181:0/2212168738 conn(0x7f53480cf890 msgr2=0x7f534805bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 shutdown_connections 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:24.445+0000 7f5350574700 1 -- 172.21.15.181:0/2212168738 wait complete. 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:25.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 Processor -- start 2020-11-16T22:29:25.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 -- start start 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0530063620 0x7f0530061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0530160c40 con 0x7f0530142660 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0530062700 con 0x7f0530142a40 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0536bde700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300629a0 con 0x7f0530063620 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0530063620 0x7f0530061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:25.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0530063620 0x7f0530061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59206/0 (socket says 172.21.15.181:59206) 2020-11-16T22:29:25.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 -- 172.21.15.181:0/2385214364 learned_addr learned my addr 172.21.15.181:0/2385214364 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:25.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 -- 172.21.15.181:0/2385214364 >> v1:172.21.15.181:6789/0 conn(0x7f0530142a40 legacy=0x7f0530160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0534bda700 1 -- 172.21.15.181:0/2385214364 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713740334 0 0) 0x7f0530062700 con 0x7f0530142a40 2020-11-16T22:29:25.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 -- 172.21.15.181:0/2385214364 >> v1:172.21.15.175:6789/0 conn(0x7f0530142660 legacy=0x7f0530063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 -- 172.21.15.181:0/2385214364 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530062c40 con 0x7f0530063620 2020-11-16T22:29:25.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f05363dd700 1 --2- 172.21.15.181:0/2385214364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530063620 0x7f0530061810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f0528009d50 tx=0x7f0528009290).ready entity=mon.1 client_cookie=13ef4d2d37aaedc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0534bda700 1 -- 172.21.15.181:0/2385214364 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800bc80 con 0x7f0530063620 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0534bda700 1 -- 172.21.15.181:0/2385214364 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052800bde0 con 0x7f0530063620 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.017+0000 7f0534bda700 1 -- 172.21.15.181:0/2385214364 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528011610 con 0x7f0530063620 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/2385214364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530063620 msgr2=0x7f0530061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 --2- 172.21.15.181:0/2385214364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530063620 0x7f0530061810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f0528009d50 tx=0x7f0528009290).stop 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/2385214364 shutdown_connections 2020-11-16T22:29:25.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 --2- 172.21.15.181:0/2385214364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530063620 0x7f0530061810 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/2385214364 >> 172.21.15.181:0/2385214364 conn(0x7f05300d2dc0 msgr2=0x7f05300ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:25.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/2385214364 shutdown_connections 2020-11-16T22:29:25.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/2385214364 wait complete. 2020-11-16T22:29:25.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 Processor -- start 2020-11-16T22:29:25.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- start start 2020-11-16T22:29:25.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0530063620 0x7f0530060290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:25.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 0x7f0530060780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:25.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:25.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05301408c0 con 0x7f0530063620 2020-11-16T22:29:25.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05301400e0 con 0x7f0530142a40 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0530140380 con 0x7f0530142660 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05353db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 0x7f0530060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05353db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 0x7f0530060780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38822/0 (socket says 172.21.15.181:38822) 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59216/0 (socket says 172.21.15.181:59216) 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05353db700 1 -- 172.21.15.181:0/1557363760 learned_addr learned my addr 172.21.15.181:0/1557363760 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0535bdc700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0530063620 0x7f0530060290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 -- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 msgr2=0x7f0530060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 0x7f0530060780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 -- 172.21.15.181:0/1557363760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0530063620 msgr2=0x7f0530060290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0530063620 0x7f0530060290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 -- 172.21.15.181:0/1557363760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530140620 con 0x7f0530142a40 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f05363dd700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f0528013000 tx=0x7f0528016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800bc80 con 0x7f0530142a40 2020-11-16T22:29:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0528009e90 con 0x7f0530142a40 2020-11-16T22:29:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05300cccb0 con 0x7f0530142a40 2020-11-16T22:29:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528010cb0 con 0x7f0530142a40 2020-11-16T22:29:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.021+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053016fa90 con 0x7f0530142a40 2020-11-16T22:29:25.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530063360 con 0x7f0530142a40 2020-11-16T22:29:25.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0528017070 con 0x7f0530142a40 2020-11-16T22:29:25.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f052dffb700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 0x7f050c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:25.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f0535bdc700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 0x7f050c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:25.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f052806f780 con 0x7f0530142a40 2020-11-16T22:29:25.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.025+0000 7f0535bdc700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 0x7f050c057550 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f0520003300 tx=0x7f0520007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.033+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0530063360 con 0x7f0530142a40 2020-11-16T22:29:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.185+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0530063360 con 0x7f0530142a40 2020-11-16T22:29:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f052dffb700 1 -- 172.21.15.181:0/1557363760 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f0530063360 con 0x7f0530142a40 2020-11-16T22:29:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 msgr2=0x7f050c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 0x7f050c057550 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f0520003300 tx=0x7f0520007b90).stop 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 msgr2=0x7f053016f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f0528013000 tx=0x7f0528016000).stop 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 shutdown_connections 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f050c055090 0x7f050c057550 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530142a40 0x7f053016f5a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0530063620 0x7f0530060290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 --2- 172.21.15.181:0/1557363760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0530142660 0x7f0530060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 >> 172.21.15.181:0/1557363760 conn(0x7f05300d2dc0 msgr2=0x7f053015d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:25.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 shutdown_connections 2020-11-16T22:29:25.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:25.189+0000 7f0536bde700 1 -- 172.21.15.181:0/1557363760 wait complete. 2020-11-16T22:29:25.626 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:25.626 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":42,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:24.070073+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:26.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:29:26.035 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:29:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.189+0000 7f43150b3700 1 Processor -- start 2020-11-16T22:29:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.189+0000 7f43150b3700 1 -- start start 2020-11-16T22:29:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.189+0000 7f43150b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4310061810 0x7f431015e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.189+0000 7f43150b3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431015d180 con 0x7f431013ec50 2020-11-16T22:29:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.189+0000 7f43150b3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431015d810 con 0x7f431013e870 2020-11-16T22:29:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431015dab0 con 0x7f4310061810 2020-11-16T22:29:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f431013e870 0x7f431015fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45292/0 (socket says 172.21.15.181:45292) 2020-11-16T22:29:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430ffff700 1 -- 172.21.15.181:0/2436154039 learned_addr learned my addr 172.21.15.181:0/2436154039 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43148b2700 1 --2- 172.21.15.181:0/2436154039 >> v2:172.21.15.181:3300/0 conn(0x7f4310061810 0x7f431015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430f7fe700 1 -- 172.21.15.181:0/2436154039 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412718545 0 0) 0x7f431015d810 con 0x7f431013e870 2020-11-16T22:29:26.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430f7fe700 1 -- 172.21.15.181:0/2436154039 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f0004350 con 0x7f431013e870 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43148b2700 1 -- 172.21.15.181:0/2436154039 >> v1:172.21.15.181:6789/0 conn(0x7f431013e870 legacy=0x7f431015fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43148b2700 1 -- 172.21.15.181:0/2436154039 >> v1:172.21.15.175:6789/0 conn(0x7f431013ec50 legacy=0x7f4310061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43148b2700 1 -- 172.21.15.181:0/2436154039 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431015dd50 con 0x7f4310061810 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43148b2700 1 --2- 172.21.15.181:0/2436154039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f431015e730 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4308009c90 tx=0x7f43080091d0).ready entity=mon.1 client_cookie=81c289878e9e6850 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430f7fe700 1 -- 172.21.15.181:0/2436154039 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430800ba70 con 0x7f4310061810 2020-11-16T22:29:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430f7fe700 1 -- 172.21.15.181:0/2436154039 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430800bbd0 con 0x7f4310061810 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f430f7fe700 1 -- 172.21.15.181:0/2436154039 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011400 con 0x7f4310061810 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- 172.21.15.181:0/2436154039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 msgr2=0x7f431015e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 --2- 172.21.15.181:0/2436154039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f431015e730 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4308009c90 tx=0x7f43080091d0).stop 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- 172.21.15.181:0/2436154039 shutdown_connections 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 --2- 172.21.15.181:0/2436154039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f431015e730 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- 172.21.15.181:0/2436154039 >> 172.21.15.181:0/2436154039 conn(0x7f43100cf630 msgr2=0x7f43100cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- 172.21.15.181:0/2436154039 shutdown_connections 2020-11-16T22:29:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.193+0000 7f43150b3700 1 -- 172.21.15.181:0/2436154039 wait complete. 2020-11-16T22:29:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 Processor -- start 2020-11-16T22:29:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- start start 2020-11-16T22:29:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431013e870 0x7f43100b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 0x7f431015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310065dd0 con 0x7f431013e870 2020-11-16T22:29:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100649d0 con 0x7f4310061810 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4310064c10 con 0x7f431013ec50 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43148b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 0x7f431015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59226/0 (socket says 172.21.15.181:59226) 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 -- 172.21.15.181:0/1096255972 learned_addr learned my addr 172.21.15.181:0/1096255972 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43077fe700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431013e870 0x7f43100b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 -- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 msgr2=0x7f431015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 0x7f431015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 -- 172.21.15.181:0/1096255972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431013e870 msgr2=0x7f43100b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431013e870 0x7f43100b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 -- 172.21.15.181:0/1096255972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310064eb0 con 0x7f4310061810 2020-11-16T22:29:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43148b2700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 0x7f431015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430ffff700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f430000c710 tx=0x7f4300007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000072e0 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300007440 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4310065150 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300005800 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.197+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4310065690 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43100c9b00 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f430001a060 con 0x7f4310061810 2020-11-16T22:29:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f430cff9700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 0x7f42ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f43077fe700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 0x7f42ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f430006d220 con 0x7f4310061810 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.201+0000 7f43077fe700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 0x7f42ec057520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f431015fc90 tx=0x7f42f8007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.205+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f43100c9b00 con 0x7f4310061810 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f43100c9b00 con 0x7f4310061810 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f430cff9700 1 -- 172.21.15.181:0/1096255972 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v42) v1 ==== 110+0+10 (secure 0 0 0) 0x7f43100c9b00 con 0x7f4310061810 2020-11-16T22:29:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 msgr2=0x7f42ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 0x7f42ec057520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f431015fc90 tx=0x7f42f8007b90).stop 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 msgr2=0x7f43100644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f430000c710 tx=0x7f4300007ad0).stop 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 shutdown_connections 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f42ec055060 0x7f42ec057520 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f431013ec50 0x7f431015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4310061810 0x7f43100644e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 --2- 172.21.15.181:0/1096255972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431013e870 0x7f43100b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 >> 172.21.15.181:0/1096255972 conn(0x7f43100cf630 msgr2=0x7f43100cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:26.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 shutdown_connections 2020-11-16T22:29:26.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.365+0000 7f43150b3700 1 -- 172.21.15.181:0/1096255972 wait complete. 2020-11-16T22:29:26.798 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:29:26.799 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:29:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 Processor -- start 2020-11-16T22:29:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 -- start start 2020-11-16T22:29:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16100816e0 0x7f1610145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1610061b50 con 0x7f1610145a30 2020-11-16T22:29:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100618b0 con 0x7f16101492c0 2020-11-16T22:29:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161687e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16100cb930 con 0x7f16100816e0 2020-11-16T22:29:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161507b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16100816e0 0x7f1610145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161607d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f16101492c0 0x7f16100cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45310/0 (socket says 172.21.15.181:45310) 2020-11-16T22:29:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.953+0000 7f161607d700 1 -- 172.21.15.181:0/3994864833 learned_addr learned my addr 172.21.15.181:0/3994864833 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803557985 0 0) 0x7f16100618b0 con 0x7f16101492c0 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f8003240 con 0x7f16101492c0 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431660740 0 0) 0x7f1610061b50 con 0x7f1610145a30 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16100618b0 con 0x7f1610145a30 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3937966262 0 0) 0x7f15f8003240 con 0x7f16101492c0 2020-11-16T22:29:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 >> v2:172.21.15.181:3300/0 conn(0x7f16100816e0 msgr2=0x7f1610145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 --2- 172.21.15.181:0/3994864833 >> v2:172.21.15.181:3300/0 conn(0x7f16100816e0 0x7f1610145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 >> v1:172.21.15.175:6789/0 conn(0x7f1610145a30 legacy=0x7f1610080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100cbbd0 con 0x7f16101492c0 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f1608001860 con 0x7f16101492c0 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 >> v1:172.21.15.181:6789/0 conn(0x7f16101492c0 legacy=0x7f16100cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15f80038c0 0x7f15f8003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 0x7f15f8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f8004e50 0x7f15f800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100cb930 con 0x7f15f80038c0 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80053b0 con 0x7f15f80044e0 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f800bca0 con 0x7f15f8004e50 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 --2- 172.21.15.181:0/3994864833 >> v2:172.21.15.181:3300/0 conn(0x7f16100816e0 0x7f1610145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161607d700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f8004e50 0x7f15f800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 0x7f15f8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161587c700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15f80038c0 0x7f15f8003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 -- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f8004e50 msgr2=0x7f15f800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f8004e50 0x7f15f800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 -- 172.21.15.181:0/3994864833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15f80038c0 msgr2=0x7f15f8003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15f80038c0 0x7f15f8003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 -- 172.21.15.181:0/3994864833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8003240 con 0x7f15f80044e0 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161607d700 1 -- 172.21.15.181:0/3994864833 reap_dead start 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161507b700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 0x7f15f8004960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f160000be50 tx=0x7f1600006df0).ready entity=mon.1 client_cookie=7987bad295d77d40 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000b580 con 0x7f15f80044e0 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160000ec30 con 0x7f15f80044e0 2020-11-16T22:29:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161487a700 1 -- 172.21.15.181:0/3994864833 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000123f0 con 0x7f15f80044e0 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 -- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 msgr2=0x7f15f8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 0x7f15f8004960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f160000be50 tx=0x7f1600006df0).stop 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 -- 172.21.15.181:0/3994864833 shutdown_connections 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 --2- 172.21.15.181:0/3994864833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15f80044e0 0x7f15f8004960 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 -- 172.21.15.181:0/3994864833 >> 172.21.15.181:0/3994864833 conn(0x7f16100d68c0 msgr2=0x7f16100d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 -- 172.21.15.181:0/3994864833 shutdown_connections 2020-11-16T22:29:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.957+0000 7f161687e700 1 -- 172.21.15.181:0/3994864833 wait complete. 2020-11-16T22:29:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 Processor -- start 2020-11-16T22:29:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- start start 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 0x7f16101722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 0x7f1610170740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1610170c30 0x7f16100cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610160b80 con 0x7f1610170c30 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610171110 con 0x7f1610140a40 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16101713b0 con 0x7f15f80044e0 2020-11-16T22:29:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161507b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 0x7f1610170740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161507b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 0x7f1610170740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59246/0 (socket says 172.21.15.181:59246) 2020-11-16T22:29:27.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161507b700 1 -- 172.21.15.181:0/423717306 learned_addr learned my addr 172.21.15.181:0/423717306 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:27.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 0x7f16101722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161607d700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1610170c30 0x7f16100cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 -- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 msgr2=0x7f1610170740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 0x7f1610170740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 -- 172.21.15.181:0/423717306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1610170c30 msgr2=0x7f16100cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1610170c30 0x7f16100cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 -- 172.21.15.181:0/423717306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610171650 con 0x7f15f80044e0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161587c700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 0x7f16101722d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f160c002ac0 tx=0x7f160c007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c007d70 con 0x7f15f80044e0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c007240 con 0x7f15f80044e0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16101718f0 con 0x7f15f80044e0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c00f980 con 0x7f15f80044e0 2020-11-16T22:29:27.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.961+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1610171e30 con 0x7f15f80044e0 2020-11-16T22:29:27.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610171ff0 con 0x7f15f80044e0 2020-11-16T22:29:27.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f160c01a030 con 0x7f15f80044e0 2020-11-16T22:29:27.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f15fdffb700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 0x7f15e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f160c06d650 con 0x7f15f80044e0 2020-11-16T22:29:27.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f161507b700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 0x7f15e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.965+0000 7f161507b700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 0x7f15e8057520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1610164610 tx=0x7f1600019000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:27.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:26.969+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f160c043ad0 con 0x7f15f80044e0 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.133+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1610044f60 con 0x7f15f80044e0 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.133+0000 7f15fdffb700 1 -- 172.21.15.181:0/423717306 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f160c0470f0 con 0x7f15f80044e0 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 msgr2=0x7f15e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 0x7f15e8057520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1610164610 tx=0x7f1600019000).stop 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 msgr2=0x7f16101722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 0x7f16101722d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f160c002ac0 tx=0x7f160c007b70).stop 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 shutdown_connections 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f15e8055060 0x7f15e8057520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1610170c30 0x7f16100cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15f80044e0 0x7f16101722d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 --2- 172.21.15.181:0/423717306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1610140a40 0x7f1610170740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 >> 172.21.15.181:0/423717306 conn(0x7f16100d68c0 msgr2=0x7f1610080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:27.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 shutdown_connections 2020-11-16T22:29:27.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.137+0000 7f161687e700 1 -- 172.21.15.181:0/423717306 wait complete. 2020-11-16T22:29:27.561 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:27.561 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:29:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.709+0000 7eff449e3700 1 Processor -- start 2020-11-16T22:29:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.709+0000 7eff449e3700 1 -- start start 2020-11-16T22:29:27.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7eff400b2190 0x7eff4015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff400b2570 0x7eff400cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff400ce460 con 0x7eff4015ff80 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400ce700 con 0x7eff400b2190 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400ce9a0 con 0x7eff400b2570 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff400b2570 0x7eff400cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff400b2570 0x7eff400cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59254/0 (socket says 172.21.15.181:59254) 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 -- 172.21.15.181:0/4253061427 learned_addr learned my addr 172.21.15.181:0/4253061427 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:27.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7eff4015ff80 0x7eff40160410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45330/0 (socket says 172.21.15.181:45330) 2020-11-16T22:29:27.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3f7fe700 1 --2- 172.21.15.181:0/4253061427 >> v2:172.21.15.175:3300/0 conn(0x7eff400b2190 0x7eff4015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 -- 172.21.15.181:0/4253061427 >> v2:172.21.15.175:3300/0 conn(0x7eff400b2190 msgr2=0x7eff4015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:27.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 --2- 172.21.15.181:0/4253061427 >> v2:172.21.15.175:3300/0 conn(0x7eff400b2190 0x7eff4015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 -- 172.21.15.181:0/4253061427 >> v1:172.21.15.181:6789/0 conn(0x7eff4015ff80 legacy=0x7eff40160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 -- 172.21.15.181:0/4253061427 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400cec40 con 0x7eff400b2570 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3effd700 1 --2- 172.21.15.181:0/4253061427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2570 0x7eff400cf620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7eff40064d90 tx=0x7eff30006eb0).ready entity=mon.1 client_cookie=dda7c6b4ed34970 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3e7fc700 1 -- 172.21.15.181:0/4253061427 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30009810 con 0x7eff400b2570 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3e7fc700 1 -- 172.21.15.181:0/4253061427 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30009970 con 0x7eff400b2570 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff3e7fc700 1 -- 172.21.15.181:0/4253061427 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30009cf0 con 0x7eff400b2570 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- 172.21.15.181:0/4253061427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2570 msgr2=0x7eff400cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 --2- 172.21.15.181:0/4253061427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2570 0x7eff400cf620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7eff40064d90 tx=0x7eff30006eb0).stop 2020-11-16T22:29:27.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- 172.21.15.181:0/4253061427 shutdown_connections 2020-11-16T22:29:27.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 --2- 172.21.15.181:0/4253061427 >> v2:172.21.15.175:3300/0 conn(0x7eff400b2190 0x7eff4015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 --2- 172.21.15.181:0/4253061427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2570 0x7eff400cf620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- 172.21.15.181:0/4253061427 >> 172.21.15.181:0/4253061427 conn(0x7eff400cfe90 msgr2=0x7eff400d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- 172.21.15.181:0/4253061427 shutdown_connections 2020-11-16T22:29:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.713+0000 7eff449e3700 1 -- 172.21.15.181:0/4253061427 wait complete. 2020-11-16T22:29:27.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 Processor -- start 2020-11-16T22:29:27.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 -- start start 2020-11-16T22:29:27.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2190 0x7eff401513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 0x7eff40151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff401518d0 con 0x7eff400b2570 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400b1d60 con 0x7eff400b2190 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff449e3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff40172b40 con 0x7eff4015ff80 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 0x7eff40151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38874/0 (socket says 172.21.15.181:38874) 2020-11-16T22:29:27.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2190 0x7eff401513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:27.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 -- 172.21.15.181:0/3857608466 learned_addr learned my addr 172.21.15.181:0/3857608466 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:28.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 -- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2190 msgr2=0x7eff401513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2190 0x7eff401513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 -- 172.21.15.181:0/3857608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 msgr2=0x7eff40151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 0x7eff40151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 -- 172.21.15.181:0/3857608466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40172de0 con 0x7eff4015ff80 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3effd700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 0x7eff40151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.717+0000 7eff3ffff700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7eff2800c710 tx=0x7eff28007af0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280072e0 con 0x7eff4015ff80 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28005530 con 0x7eff4015ff80 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280058b0 con 0x7eff4015ff80 2020-11-16T22:29:28.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff40173080 con 0x7eff4015ff80 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401735c0 con 0x7eff4015ff80 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff40044f60 con 0x7eff4015ff80 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7eff28010070 con 0x7eff4015ff80 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 0x7eff1c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5253+0+0 (secure 0 0 0) 0x7eff2806d010 con 0x7eff4015ff80 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff3f7fe700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 0x7eff1c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.721+0000 7eff3f7fe700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 0x7eff1c057520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7eff38009960 tx=0x7eff380091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.729+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7eff280435a0 con 0x7eff4015ff80 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.885+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff400cf2e0 con 0x7eff4015ff80 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.885+0000 7eff377fe700 1 -- 172.21.15.181:0/3857608466 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+143 (secure 0 0 0) 0x7eff28046bc0 con 0x7eff4015ff80 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 msgr2=0x7eff1c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 0x7eff1c057520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7eff38009960 tx=0x7eff380091d0).stop 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 msgr2=0x7eff400b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7eff2800c710 tx=0x7eff28007af0).stop 2020-11-16T22:29:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 shutdown_connections 2020-11-16T22:29:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7eff1c055060 0x7eff1c057520 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4015ff80 0x7eff400b1320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff400b2190 0x7eff401513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 --2- 172.21.15.181:0/3857608466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff400b2570 0x7eff40151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 >> 172.21.15.181:0/3857608466 conn(0x7eff400cfe90 msgr2=0x7eff40160410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 shutdown_connections 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:27.889+0000 7eff449e3700 1 -- 172.21.15.181:0/3857608466 wait complete. 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:28.319 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:29:28.320 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:29:28.321 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:29:28.321 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:29:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 Processor -- start 2020-11-16T22:29:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 -- start start 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b44145b50 0x7f0b440d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b440d2810 0x7f0b441612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b44081f20 con 0x7f0b44145770 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b44145fb0 con 0x7f0b44145b50 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b48e53700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b44060470 con 0x7f0b440d2810 2020-11-16T22:29:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b43fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b440d2810 0x7f0b441612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b43fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b440d2810 0x7f0b441612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59274/0 (socket says 172.21.15.181:59274) 2020-11-16T22:29:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b43fff700 1 -- 172.21.15.181:0/1116320056 learned_addr learned my addr 172.21.15.181:0/1116320056 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b42ffd700 1 --2- 172.21.15.181:0/1116320056 >> v2:172.21.15.175:3300/0 conn(0x7f0b44145b50 0x7f0b440d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b43fff700 1 -- 172.21.15.181:0/1116320056 >> v2:172.21.15.175:3300/0 conn(0x7f0b44145b50 msgr2=0x7f0b440d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.477+0000 7f0b43fff700 1 --2- 172.21.15.181:0/1116320056 >> v2:172.21.15.175:3300/0 conn(0x7f0b44145b50 0x7f0b440d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b43fff700 1 -- 172.21.15.181:0/1116320056 >> v1:172.21.15.175:6789/0 conn(0x7f0b44145770 legacy=0x7f0b44160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b43fff700 1 -- 172.21.15.181:0/1116320056 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44060710 con 0x7f0b440d2810 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b43fff700 1 --2- 172.21.15.181:0/1116320056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b440d2810 0x7f0b441612b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0b44062ae0 tx=0x7f0b3c009290).ready entity=mon.1 client_cookie=a11c98093413400d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b427fc700 1 -- 172.21.15.181:0/1116320056 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00bbc0 con 0x7f0b440d2810 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b427fc700 1 -- 172.21.15.181:0/1116320056 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c00bd20 con 0x7f0b440d2810 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b427fc700 1 -- 172.21.15.181:0/1116320056 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c011610 con 0x7f0b440d2810 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- 172.21.15.181:0/1116320056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b440d2810 msgr2=0x7f0b441612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 --2- 172.21.15.181:0/1116320056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b440d2810 0x7f0b441612b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0b44062ae0 tx=0x7f0b3c009290).stop 2020-11-16T22:29:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- 172.21.15.181:0/1116320056 shutdown_connections 2020-11-16T22:29:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 --2- 172.21.15.181:0/1116320056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b440d2810 0x7f0b441612b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 --2- 172.21.15.181:0/1116320056 >> v2:172.21.15.175:3300/0 conn(0x7f0b44145b50 0x7f0b440d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- 172.21.15.181:0/1116320056 >> 172.21.15.181:0/1116320056 conn(0x7f0b440d6760 msgr2=0x7f0b440d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- 172.21.15.181:0/1116320056 shutdown_connections 2020-11-16T22:29:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- 172.21.15.181:0/1116320056 wait complete. 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 Processor -- start 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 -- start start 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.481+0000 7f0b48e53700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 0x7f0b44159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b44145770 0x7f0b44159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 0x7f0b4413d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4413e500 con 0x7f0b44145770 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4413e7a0 con 0x7f0b44145b50 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4413ea40 con 0x7f0b440d2810 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 0x7f0b4413d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 0x7f0b44159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 0x7f0b4413d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59282/0 (socket says 172.21.15.181:59282) 2020-11-16T22:29:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b43fff700 1 -- 172.21.15.181:0/2165056572 learned_addr learned my addr 172.21.15.181:0/2165056572 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b42ffd700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b44145770 0x7f0b44159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 -- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 msgr2=0x7f0b4413d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 0x7f0b4413d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 -- 172.21.15.181:0/2165056572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b44145770 msgr2=0x7f0b44159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b44145770 0x7f0b44159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 -- 172.21.15.181:0/2165056572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44164d50 con 0x7f0b440d2810 2020-11-16T22:29:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b437fe700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 0x7f0b44159990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0b3400c710 tx=0x7f0b34007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400e490 con 0x7f0b440d2810 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400e5f0 con 0x7f0b440d2810 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b44164ff0 con 0x7f0b440d2810 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400e970 con 0x7f0b440d2810 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4413de90 con 0x7f0b440d2810 2020-11-16T22:29:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0b34010030 con 0x7f0b440d2810 2020-11-16T22:29:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b44044f60 con 0x7f0b440d2810 2020-11-16T22:29:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 0x7f0b20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b42ffd700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 0x7f0b20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0b3406d8d0 con 0x7f0b440d2810 2020-11-16T22:29:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.485+0000 7f0b42ffd700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 0x7f0b20057520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0b44082690 tx=0x7f0b2c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:28.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.489+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0b34043d50 con 0x7f0b440d2810 2020-11-16T22:29:28.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.641+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f0b440d7560 con 0x7f0b440d2810 2020-11-16T22:29:28.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.925+0000 7f0b3b7fe700 1 -- 172.21.15.181:0/2165056572 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v46) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0b34047370 con 0x7f0b440d2810 2020-11-16T22:29:28.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 msgr2=0x7f0b20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 0x7f0b20057520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0b44082690 tx=0x7f0b2c008040).stop 2020-11-16T22:29:28.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 msgr2=0x7f0b44159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:28.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 0x7f0b44159990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0b3400c710 tx=0x7f0b34007ac0).stop 2020-11-16T22:29:28.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 shutdown_connections 2020-11-16T22:29:29.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0b20055060 0x7f0b20057520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b44145b50 0x7f0b4413d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b440d2810 0x7f0b44159990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 --2- 172.21.15.181:0/2165056572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b44145770 0x7f0b44159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 >> 172.21.15.181:0/2165056572 conn(0x7f0b440d6760 msgr2=0x7f0b440d4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:29.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 shutdown_connections 2020-11-16T22:29:29.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:28.929+0000 7f0b48e53700 1 -- 172.21.15.181:0/2165056572 wait complete. 2020-11-16T22:29:29.116 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:29.116 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:29:29.117 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:29:29.123 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:29.125 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:29.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.273+0000 7fa0d2383700 1 Processor -- start 2020-11-16T22:29:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.273+0000 7fa0d2383700 1 -- start start 2020-11-16T22:29:29.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.273+0000 7fa0d2383700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 0x7fa0cc080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc061b50 con 0x7fa0cc0814c0 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0618b0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0cb930 con 0x7fa0cc145a30 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38896/0 (socket says 172.21.15.181:38896) 2020-11-16T22:29:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 -- 172.21.15.181:0/3149154073 learned_addr learned my addr 172.21.15.181:0/3149154073 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d0b80700 1 --2- 172.21.15.181:0/3149154073 >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 0x7fa0cc080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 -- 172.21.15.181:0/3149154073 >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 msgr2=0x7fa0cc080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 --2- 172.21.15.181:0/3149154073 >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 0x7fa0cc080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 -- 172.21.15.181:0/3149154073 >> v1:172.21.15.175:6789/0 conn(0x7fa0cc0814c0 legacy=0x7fa0cc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 -- 172.21.15.181:0/3149154073 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0cbbd0 con 0x7fa0cc145a30 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d0b80700 1 --2- 172.21.15.181:0/3149154073 >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 0x7fa0cc080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d1381700 1 --2- 172.21.15.181:0/3149154073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa0c8006fd0 tx=0x7fa0c8006eb0).ready entity=mon.2 client_cookie=d5eb1e25a6675341 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0bffff700 1 -- 172.21.15.181:0/3149154073 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80098d0 con 0x7fa0cc145a30 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0bffff700 1 -- 172.21.15.181:0/3149154073 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8009a30 con 0x7fa0cc145a30 2020-11-16T22:29:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0bffff700 1 -- 172.21.15.181:0/3149154073 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8009db0 con 0x7fa0cc145a30 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- 172.21.15.181:0/3149154073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 msgr2=0x7fa0cc145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3149154073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa0c8006fd0 tx=0x7fa0c8006eb0).stop 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- 172.21.15.181:0/3149154073 shutdown_connections 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3149154073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc145a30 0x7fa0cc145e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3149154073 >> v2:172.21.15.175:3300/0 conn(0x7fa0cc1492c0 0x7fa0cc080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- 172.21.15.181:0/3149154073 >> 172.21.15.181:0/3149154073 conn(0x7fa0cc0d68c0 msgr2=0x7fa0cc0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- 172.21.15.181:0/3149154073 shutdown_connections 2020-11-16T22:29:29.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 -- 172.21.15.181:0/3149154073 wait complete. 2020-11-16T22:29:29.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.277+0000 7fa0d2383700 1 Processor -- start 2020-11-16T22:29:29.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- start start 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0cc0814c0 0x7fa0cc1708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 0x7fa0cc162870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc161410 con 0x7fa0cc0814c0 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1616b0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc161950 con 0x7fa0cc162d60 2020-11-16T22:29:29.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d1b82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 0x7fa0cc162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d1b82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38906/0 (socket says 172.21.15.181:38906) 2020-11-16T22:29:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d1b82700 1 -- 172.21.15.181:0/3611485389 learned_addr learned my addr 172.21.15.181:0/3611485389 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d1381700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0cc0814c0 0x7fa0cc1708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 -- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 msgr2=0x7fa0cc0d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 -- 172.21.15.181:0/3611485389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0cc0814c0 msgr2=0x7fa0cc1708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0cc0814c0 0x7fa0cc1708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 -- 172.21.15.181:0/3611485389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc161bf0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d1b82700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d0b80700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 0x7fa0cc162870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa0c000d900 tx=0x7fa0c0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000f5e0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c000f740 con 0x7fa0cc1492c0 2020-11-16T22:29:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc161e90 con 0x7fa0cc1492c0 2020-11-16T22:29:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000fac0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.281+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1623d0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa0c001c460 con 0x7fa0cc1492c0 2020-11-16T22:29:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc044f60 con 0x7fa0cc1492c0 2020-11-16T22:29:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0bd7fa700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa0c006dcc0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0d1381700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.285+0000 7fa0d1381700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 0x7fa0a4057520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fa0cc0d43d0 tx=0x7fa0c8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.293+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa0c0044090 con 0x7fa0cc1492c0 2020-11-16T22:29:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.449+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa0cc144f20 con 0x7fa0cc1492c0 2020-11-16T22:29:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.925+0000 7fa0bd7fa700 1 -- 172.21.15.181:0/3611485389 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa0c00476b0 con 0x7fa0cc1492c0 2020-11-16T22:29:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 msgr2=0x7fa0a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 0x7fa0a4057520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fa0cc0d43d0 tx=0x7fa0c8015000).stop 2020-11-16T22:29:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 msgr2=0x7fa0cc162870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 0x7fa0cc162870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa0c000d900 tx=0x7fa0c0008040).stop 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 shutdown_connections 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0cc162d60 0x7fa0cc0d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0cc0814c0 0x7fa0cc1708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 --2- 172.21.15.181:0/3611485389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0cc1492c0 0x7fa0cc162870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.933+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 >> 172.21.15.181:0/3611485389 conn(0x7fa0cc0d68c0 msgr2=0x7fa0cc060480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:29.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.937+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 shutdown_connections 2020-11-16T22:29:29.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:29.937+0000 7fa0d2383700 1 -- 172.21.15.181:0/3611485389 wait complete. 2020-11-16T22:29:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:29:29.933 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:29:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 Processor -- start 2020-11-16T22:29:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- start start 2020-11-16T22:29:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc8c166e00 0x7efc8c1671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c1657b0 con 0x7efc8c0d2960 2020-11-16T22:29:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c1676d0 con 0x7efc8c166e00 2020-11-16T22:29:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0ccc90 con 0x7efc8c0c9690 2020-11-16T22:29:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc8c166e00 0x7efc8c1671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38914/0 (socket says 172.21.15.181:38914) 2020-11-16T22:29:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 -- 172.21.15.181:0/4091300922 learned_addr learned my addr 172.21.15.181:0/4091300922 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 -- 172.21.15.181:0/4091300922 >> v2:172.21.15.181:3300/0 conn(0x7efc8c166e00 msgr2=0x7efc8c1671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 --2- 172.21.15.181:0/4091300922 >> v2:172.21.15.181:3300/0 conn(0x7efc8c166e00 0x7efc8c1671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 -- 172.21.15.181:0/4091300922 >> v1:172.21.15.181:6789/0 conn(0x7efc8c0d2960 legacy=0x7efc8c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 -- 172.21.15.181:0/4091300922 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0ccf30 con 0x7efc8c0c9690 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8bfff700 1 --2- 172.21.15.181:0/4091300922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7efc74006fd0 tx=0x7efc74006eb0).ready entity=mon.2 client_cookie=2db570cd8f2c7616 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8a7fc700 1 -- 172.21.15.181:0/4091300922 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74009810 con 0x7efc8c0c9690 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8a7fc700 1 -- 172.21.15.181:0/4091300922 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc74009970 con 0x7efc8c0c9690 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc8a7fc700 1 -- 172.21.15.181:0/4091300922 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74009cf0 con 0x7efc8c0c9690 2020-11-16T22:29:30.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- 172.21.15.181:0/4091300922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 msgr2=0x7efc8c0c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 --2- 172.21.15.181:0/4091300922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7efc74006fd0 tx=0x7efc74006eb0).stop 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- 172.21.15.181:0/4091300922 shutdown_connections 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 --2- 172.21.15.181:0/4091300922 >> v2:172.21.15.181:3300/0 conn(0x7efc8c166e00 0x7efc8c1671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 --2- 172.21.15.181:0/4091300922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0c71b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- 172.21.15.181:0/4091300922 >> 172.21.15.181:0/4091300922 conn(0x7efc8c060bc0 msgr2=0x7efc8c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- 172.21.15.181:0/4091300922 shutdown_connections 2020-11-16T22:29:30.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.129+0000 7efc90fa5700 1 -- 172.21.15.181:0/4091300922 wait complete. 2020-11-16T22:29:30.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 Processor -- start 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 -- start start 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 0x7efc8c14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8c166e00 0x7efc8c0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0b6a80 con 0x7efc8c0d2960 2020-11-16T22:29:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0b6d20 con 0x7efc8c166e00 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0b6fc0 con 0x7efc8c0c9690 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8c166e00 0x7efc8c0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38916/0 (socket says 172.21.15.181:38916) 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 -- 172.21.15.181:0/638624358 learned_addr learned my addr 172.21.15.181:0/638624358 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8affd700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 0x7efc8c14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 -- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8c166e00 msgr2=0x7efc8c0c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8c166e00 0x7efc8c0c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 -- 172.21.15.181:0/638624358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 msgr2=0x7efc8c14ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 0x7efc8c14ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 -- 172.21.15.181:0/638624358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0b7260 con 0x7efc8c0c9690 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8affd700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 0x7efc8c14ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:29:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc8b7fe700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7efc8400eba0 tx=0x7efc84006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8400c9b0 con 0x7efc8c0c9690 2020-11-16T22:29:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8400cb10 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c0b7500 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.133+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8400ce90 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c174d70 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c044f60 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efc84012600 con 0x7efc8c0c9690 2020-11-16T22:29:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc837fe700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 0x7efc5c057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5957+0+0 (secure 0 0 0) 0x7efc8406e0e0 con 0x7efc8c0c9690 2020-11-16T22:29:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc8affd700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 0x7efc5c057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.137+0000 7efc8affd700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 0x7efc5c057570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7efc7c007fd0 tx=0x7efc7c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.141+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efc840443d0 con 0x7efc8c0c9690 2020-11-16T22:29:30.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.349+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7efc8c05d4e0 con 0x7efc8c0c9690 2020-11-16T22:29:30.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc837fe700 1 -- 172.21.15.181:0/638624358 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v43) v1 ==== 187+0+0 (secure 0 0 0) 0x7efc840479f0 con 0x7efc8c0c9690 2020-11-16T22:29:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 msgr2=0x7efc5c057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 0x7efc5c057570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7efc7c007fd0 tx=0x7efc7c009040).stop 2020-11-16T22:29:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 msgr2=0x7efc8c0b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7efc8400eba0 tx=0x7efc84006df0).stop 2020-11-16T22:29:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 shutdown_connections 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efc5c0550b0 0x7efc5c057570 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc8c166e00 0x7efc8c0c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc8c0c9690 0x7efc8c0b52e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 --2- 172.21.15.181:0/638624358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc8c0d2960 0x7efc8c14ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 >> 172.21.15.181:0/638624358 conn(0x7efc8c060bc0 msgr2=0x7efc8c0c71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 shutdown_connections 2020-11-16T22:29:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.373+0000 7efc90fa5700 1 -- 172.21.15.181:0/638624358 wait complete. 2020-11-16T22:29:30.360 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 7 and data pool 8 2020-11-16T22:29:30.372 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:30.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.501+0000 7fd2710ad700 1 Processor -- start 2020-11-16T22:29:30.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.501+0000 7fd2710ad700 1 -- start start 2020-11-16T22:29:30.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 0x7fd26c061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 0x7fd26c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c15d180 con 0x7fd26c13e870 2020-11-16T22:29:30.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c15d810 con 0x7fd26c13ec50 2020-11-16T22:29:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c15dab0 con 0x7fd26c0615f0 2020-11-16T22:29:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd26c13e870 0x7fd26c15fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45386/0 (socket says 172.21.15.181:45386) 2020-11-16T22:29:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26bfff700 1 -- 172.21.15.181:0/4248425279 learned_addr learned my addr 172.21.15.181:0/4248425279 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 --2- 172.21.15.181:0/4248425279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 0x7fd26c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26b7fe700 1 --2- 172.21.15.181:0/4248425279 >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 0x7fd26c061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26affd700 1 -- 172.21.15.181:0/4248425279 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861086424 0 0) 0x7fd26c15d180 con 0x7fd26c13e870 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26affd700 1 -- 172.21.15.181:0/4248425279 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd258003240 con 0x7fd26c13e870 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 -- 172.21.15.181:0/4248425279 >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 msgr2=0x7fd26c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 --2- 172.21.15.181:0/4248425279 >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 0x7fd26c061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 -- 172.21.15.181:0/4248425279 >> v1:172.21.15.181:6789/0 conn(0x7fd26c13e870 legacy=0x7fd26c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 -- 172.21.15.181:0/4248425279 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c15dd50 con 0x7fd26c0615f0 2020-11-16T22:29:30.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26b7fe700 1 --2- 172.21.15.181:0/4248425279 >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 0x7fd26c061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2708ac700 1 --2- 172.21.15.181:0/4248425279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 0x7fd26c15e730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fd25c00b9a0 tx=0x7fd25c006df0).ready entity=mon.2 client_cookie=c45a3ca9ef23e10a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26affd700 1 -- 172.21.15.181:0/4248425279 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0097b0 con 0x7fd26c0615f0 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26affd700 1 -- 172.21.15.181:0/4248425279 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c009910 con 0x7fd26c0615f0 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd26affd700 1 -- 172.21.15.181:0/4248425279 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c009c90 con 0x7fd26c0615f0 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- 172.21.15.181:0/4248425279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 msgr2=0x7fd26c15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 --2- 172.21.15.181:0/4248425279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 0x7fd26c15e730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fd25c00b9a0 tx=0x7fd25c006df0).stop 2020-11-16T22:29:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- 172.21.15.181:0/4248425279 shutdown_connections 2020-11-16T22:29:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 --2- 172.21.15.181:0/4248425279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c0615f0 0x7fd26c15e730 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 --2- 172.21.15.181:0/4248425279 >> v2:172.21.15.175:3300/0 conn(0x7fd26c13ec50 0x7fd26c061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- 172.21.15.181:0/4248425279 >> 172.21.15.181:0/4248425279 conn(0x7fd26c0cf630 msgr2=0x7fd26c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- 172.21.15.181:0/4248425279 shutdown_connections 2020-11-16T22:29:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.505+0000 7fd2710ad700 1 -- 172.21.15.181:0/4248425279 wait complete. 2020-11-16T22:29:30.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 Processor -- start 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- start start 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd26c0615f0 0x7fd26c060bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26c13ec50 0x7fd26c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0610b0 con 0x7fd26c0615f0 2020-11-16T22:29:30.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c139d10 con 0x7fd26c13ec50 2020-11-16T22:29:30.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c138d50 con 0x7fd26c13e870 2020-11-16T22:29:30.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2708ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26c13ec50 0x7fd26c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38932/0 (socket says 172.21.15.181:38932) 2020-11-16T22:29:30.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 -- 172.21.15.181:0/3683957423 learned_addr learned my addr 172.21.15.181:0/3683957423 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:30.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26bfff700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd26c0615f0 0x7fd26c060bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 -- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26c13ec50 msgr2=0x7fd26c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26c13ec50 0x7fd26c0d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 -- 172.21.15.181:0/3683957423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd26c0615f0 msgr2=0x7fd26c060bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd26c0615f0 0x7fd26c060bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 -- 172.21.15.181:0/3683957423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c138ff0 con 0x7fd26c13e870 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd26b7fe700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd26000dbc0 tx=0x7fd260008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600097d0 con 0x7fd26c13e870 2020-11-16T22:29:30.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd260009930 con 0x7fd26c13e870 2020-11-16T22:29:30.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c139290 con 0x7fd26c13e870 2020-11-16T22:29:30.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260009cd0 con 0x7fd26c13e870 2020-11-16T22:29:30.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c139a70 con 0x7fd26c13e870 2020-11-16T22:29:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.509+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c13e480 con 0x7fd26c13e870 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.513+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd260013690 con 0x7fd26c13e870 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.513+0000 7fd24bfff700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 0x7fd244057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.513+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd26006e0e0 con 0x7fd26c13e870 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.513+0000 7fd26bfff700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 0x7fd244057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.513+0000 7fd26bfff700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 0x7fd244057520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd264005920 tx=0x7fd264009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.521+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd2600442e0 con 0x7fd26c13e870 2020-11-16T22:29:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.673+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd26c044f60 con 0x7fd26c13e870 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd24bfff700 1 -- 172.21.15.181:0/3683957423 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16887 (secure 0 0 0) 0x7fd260047900 con 0x7fd26c13e870 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 msgr2=0x7fd244057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 0x7fd244057520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd264005920 tx=0x7fd264009040).stop 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 msgr2=0x7fd26c0cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd26000dbc0 tx=0x7fd260008040).stop 2020-11-16T22:29:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 shutdown_connections 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd244055060 0x7fd244057520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26c13ec50 0x7fd26c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd26c0615f0 0x7fd26c060bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 --2- 172.21.15.181:0/3683957423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd26c13e870 0x7fd26c0cfbb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 >> 172.21.15.181:0/3683957423 conn(0x7fd26c0cf630 msgr2=0x7fd26c15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 shutdown_connections 2020-11-16T22:29:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:30.677+0000 7fd2710ad700 1 -- 172.21.15.181:0/3683957423 wait complete. 2020-11-16T22:29:30.912 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:30.912 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":48,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:30.354386+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:31.523 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:31.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 Processor -- start 2020-11-16T22:29:31.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 -- start start 2020-11-16T22:29:31.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f524c163c60 0x7f524c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c161870 con 0x7f524c162f20 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c1633a0 con 0x7f524c163c60 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f5251ee6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c161b10 con 0x7f524c1428e0 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f524c162f20 0x7f524c163550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45400/0 (socket says 172.21.15.181:45400) 2020-11-16T22:29:31.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524affd700 1 -- 172.21.15.181:0/1551548684 learned_addr learned my addr 172.21.15.181:0/1551548684 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 --2- 172.21.15.181:0/1551548684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524bfff700 1 --2- 172.21.15.181:0/1551548684 >> v2:172.21.15.175:3300/0 conn(0x7f524c163c60 0x7f524c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524a7fc700 1 -- 172.21.15.181:0/1551548684 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395051999 0 0) 0x7f524c161870 con 0x7f524c162f20 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524a7fc700 1 -- 172.21.15.181:0/1551548684 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522c003240 con 0x7f524c162f20 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 -- 172.21.15.181:0/1551548684 >> v2:172.21.15.175:3300/0 conn(0x7f524c163c60 msgr2=0x7f524c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 --2- 172.21.15.181:0/1551548684 >> v2:172.21.15.175:3300/0 conn(0x7f524c163c60 0x7f524c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 -- 172.21.15.181:0/1551548684 >> v1:172.21.15.181:6789/0 conn(0x7f524c162f20 legacy=0x7f524c163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 -- 172.21.15.181:0/1551548684 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c161db0 con 0x7f524c1428e0 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524b7fe700 1 --2- 172.21.15.181:0/1551548684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c162a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f523c00b9a0 tx=0x7f523c006df0).ready entity=mon.2 client_cookie=ee41fc21a31bccb6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524a7fc700 1 -- 172.21.15.181:0/1551548684 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c009710 con 0x7f524c1428e0 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.665+0000 7f524a7fc700 1 -- 172.21.15.181:0/1551548684 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c009870 con 0x7f524c1428e0 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524a7fc700 1 -- 172.21.15.181:0/1551548684 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c009bf0 con 0x7f524c1428e0 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- 172.21.15.181:0/1551548684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 msgr2=0x7f524c162a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1551548684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c162a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f523c00b9a0 tx=0x7f523c006df0).stop 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- 172.21.15.181:0/1551548684 shutdown_connections 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1551548684 >> v2:172.21.15.175:3300/0 conn(0x7f524c163c60 0x7f524c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1551548684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c162a30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- 172.21.15.181:0/1551548684 >> 172.21.15.181:0/1551548684 conn(0x7f524c060bc0 msgr2=0x7f524c0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- 172.21.15.181:0/1551548684 shutdown_connections 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- 172.21.15.181:0/1551548684 wait complete. 2020-11-16T22:29:31.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 Processor -- start 2020-11-16T22:29:31.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- start start 2020-11-16T22:29:31.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c162f20 0x7f524c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 0x7f524c0b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b5510 con 0x7f524c162f20 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c159020 con 0x7f524c163c60 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f5251ee6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b5740 con 0x7f524c1428e0 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38944/0 (socket says 172.21.15.181:38944) 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524b7fe700 1 -- 172.21.15.181:0/1544140768 learned_addr learned my addr 172.21.15.181:0/1544140768 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:31.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524bfff700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 0x7f524c0b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.669+0000 7f524affd700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c162f20 0x7f524c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 -- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 msgr2=0x7f524c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 -- 172.21.15.181:0/1544140768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c162f20 msgr2=0x7f524c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c162f20 0x7f524c157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 -- 172.21.15.181:0/1544140768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0b59e0 con 0x7f524c163c60 2020-11-16T22:29:31.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524bfff700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 0x7f524c0b63c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f524400ed00 tx=0x7f52440091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:31.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400eec0 con 0x7f524c163c60 2020-11-16T22:29:31.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524400cb00 con 0x7f524c163c60 2020-11-16T22:29:31.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c0b5c80 con 0x7f524c163c60 2020-11-16T22:29:31.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400ce80 con 0x7f524c163c60 2020-11-16T22:29:31.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c164460 con 0x7f524c163c60 2020-11-16T22:29:31.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5244022070 con 0x7f524c163c60 2020-11-16T22:29:31.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 0x7f5228057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:31.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f524406e520 con 0x7f524c163c60 2020-11-16T22:29:31.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524b7fe700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 0x7f5228057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c069eb0 con 0x7f524c163c60 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.673+0000 7f524b7fe700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 0x7f5228057520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f523c002fc0 tx=0x7f523c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.681+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5244016020 con 0x7f524c163c60 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.829+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f524c0ccc30 con 0x7f524c163c60 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.829+0000 7f52437fe700 1 -- 172.21.15.181:0/1544140768 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f5244016020 con 0x7f524c163c60 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 msgr2=0x7f5228057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 0x7f5228057520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f523c002fc0 tx=0x7f523c015000).stop 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 msgr2=0x7f524c0b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 0x7f524c0b63c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f524400ed00 tx=0x7f52440091d0).stop 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 shutdown_connections 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5228055060 0x7f5228057520 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f524c163c60 0x7f524c0b63c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f524c1428e0 0x7f524c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 --2- 172.21.15.181:0/1544140768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c162f20 0x7f524c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:31.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 >> 172.21.15.181:0/1544140768 conn(0x7f524c060bc0 msgr2=0x7f524c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:31.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 shutdown_connections 2020-11-16T22:29:31.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:31.833+0000 7f5251ee6700 1 -- 172.21.15.181:0/1544140768 wait complete. 2020-11-16T22:29:31.868 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:31.868 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:31.359585+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:32.070 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:29:32.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2020-11-16T22:29:32.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i has been restored 2020-11-16T22:29:32.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.221+0000 7f7a8c7b4700 1 Processor -- start 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.221+0000 7f7a8c7b4700 1 -- start start 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a840615f0 0x7f7a840622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a84160e40 con 0x7f7a8415f5b0 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a841609a0 con 0x7f7a840615f0 2020-11-16T22:29:32.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84160ba0 con 0x7f7a84062790 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38954/0 (socket says 172.21.15.181:38954) 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 -- 172.21.15.181:0/2614982268 learned_addr learned my addr 172.21.15.181:0/2614982268 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a89d4f700 1 --2- 172.21.15.181:0/2614982268 >> v2:172.21.15.181:3300/0 conn(0x7f7a840615f0 0x7f7a840622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8954e700 1 -- 172.21.15.181:0/2614982268 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596155255 0 0) 0x7f7a84160e40 con 0x7f7a8415f5b0 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8954e700 1 -- 172.21.15.181:0/2614982268 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c003240 con 0x7f7a8415f5b0 2020-11-16T22:29:32.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 -- 172.21.15.181:0/2614982268 >> v2:172.21.15.181:3300/0 conn(0x7f7a840615f0 msgr2=0x7f7a840622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/2614982268 >> v2:172.21.15.181:3300/0 conn(0x7f7a840615f0 0x7f7a840622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 -- 172.21.15.181:0/2614982268 >> v1:172.21.15.181:6789/0 conn(0x7f7a8415f5b0 legacy=0x7f7a840d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 -- 172.21.15.181:0/2614982268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84060970 con 0x7f7a84062790 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/2614982268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f7a7c009d50 tx=0x7f7a7c009290).ready entity=mon.2 client_cookie=bdec7293e4fdaac9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8954e700 1 -- 172.21.15.181:0/2614982268 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00bad0 con 0x7f7a84062790 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8954e700 1 -- 172.21.15.181:0/2614982268 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00bc30 con 0x7f7a84062790 2020-11-16T22:29:32.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8954e700 1 -- 172.21.15.181:0/2614982268 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c011400 con 0x7f7a84062790 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/2614982268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 msgr2=0x7f7a84080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/2614982268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f7a7c009d50 tx=0x7f7a7c009290).stop 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/2614982268 shutdown_connections 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/2614982268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84062790 0x7f7a84080fd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/2614982268 >> v2:172.21.15.181:3300/0 conn(0x7f7a840615f0 0x7f7a840622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/2614982268 >> 172.21.15.181:0/2614982268 conn(0x7f7a840d69a0 msgr2=0x7f7a840d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/2614982268 shutdown_connections 2020-11-16T22:29:32.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.225+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/2614982268 wait complete. 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 Processor -- start 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- start start 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 0x7f7a841450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 0x7f7a841455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 0x7f7a8415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8415d780 con 0x7f7a84062790 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8415d4e0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84170670 con 0x7f7a840615f0 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8a550700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 0x7f7a841450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8a550700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 0x7f7a841450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38958/0 (socket says 172.21.15.181:38958) 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8a550700 1 -- 172.21.15.181:0/1219693872 learned_addr learned my addr 172.21.15.181:0/1219693872 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 0x7f7a8415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a89d4f700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 0x7f7a841455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 -- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 msgr2=0x7f7a841450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 0x7f7a841450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 -- 172.21.15.181:0/1219693872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 msgr2=0x7f7a841455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 0x7f7a841455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 -- 172.21.15.181:0/1219693872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84170910 con 0x7f7a8415f5b0 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a89d4f700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 0x7f7a841455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:32.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8ad51700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 0x7f7a8415c800 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f7a7c009d50 tx=0x7f7a7c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01b6b0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c01b810 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01bb90 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a84170bb0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.229+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8415ccf0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84069eb0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7a7c017070 con 0x7f7a8415f5b0 2020-11-16T22:29:32.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a767fc700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 0x7f7a5c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:32.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a8a550700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 0x7f7a5c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:32.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a7c06fde0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.233+0000 7f7a8a550700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 0x7f7a5c057520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f7a78007fd0 tx=0x7f7a780079a0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.241+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7a7c0460c0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:32.441+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7a841642f0 con 0x7f7a8415f5b0 2020-11-16T22:29:32.896 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:29:32.912+0000 7f6432437700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:29:33.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.377+0000 7f7a767fc700 1 -- 172.21.15.181:0/1219693872 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7a7c015030 con 0x7f7a8415f5b0 2020-11-16T22:29:33.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 msgr2=0x7f7a5c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 0x7f7a5c057520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f7a78007fd0 tx=0x7f7a780079a0).stop 2020-11-16T22:29:33.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 msgr2=0x7f7a8415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 0x7f7a8415c800 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f7a7c009d50 tx=0x7f7a7c016000).stop 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 shutdown_connections 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7a5c055060 0x7f7a5c057520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8415f5b0 0x7f7a8415c800 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a840615f0 0x7f7a841450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 --2- 172.21.15.181:0/1219693872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a84062790 0x7f7a841455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 >> 172.21.15.181:0/1219693872 conn(0x7f7a840d69a0 msgr2=0x7f7a840b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:33.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 shutdown_connections 2020-11-16T22:29:33.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.381+0000 7f7a8c7b4700 1 -- 172.21.15.181:0/1219693872 wait complete. 2020-11-16T22:29:33.380 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:33.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 Processor -- start 2020-11-16T22:29:33.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- start start 2020-11-16T22:29:33.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 0x7f52fc162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52fc162f20 0x7f52fc163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f52fc163a40 0x7f52fc15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc1633a0 con 0x7f52fc163a40 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc161870 con 0x7f52fc162f20 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc161b10 con 0x7f52fc1428e0 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52fa59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 0x7f52fc162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52fc162f20 0x7f52fc163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52fa59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 0x7f52fc162a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38966/0 (socket says 172.21.15.181:38966) 2020-11-16T22:29:33.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52fa59c700 1 -- 172.21.15.181:0/2047749370 learned_addr learned my addr 172.21.15.181:0/2047749370 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:33.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52fad9d700 1 --2- 172.21.15.181:0/2047749370 >> v2:172.21.15.175:3300/0 conn(0x7f52fc163a40 0x7f52fc15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 -- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 msgr2=0x7f52fc162a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 0x7f52fc162a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 -- 172.21.15.181:0/2047749370 >> v2:172.21.15.175:3300/0 conn(0x7f52fc163a40 msgr2=0x7f52fc15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/2047749370 >> v2:172.21.15.175:3300/0 conn(0x7f52fc163a40 0x7f52fc15ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 -- 172.21.15.181:0/2047749370 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc161db0 con 0x7f52fc162f20 2020-11-16T22:29:33.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc162f20 0x7f52fc163550 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f52fc05ce10 tx=0x7f52ec006df0).ready entity=mon.1 client_cookie=f5b8de01a10ebf1e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f959a700 1 -- 172.21.15.181:0/2047749370 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0096d0 con 0x7f52fc162f20 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f959a700 1 -- 172.21.15.181:0/2047749370 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec009830 con 0x7f52fc162f20 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f52f959a700 1 -- 172.21.15.181:0/2047749370 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec009bb0 con 0x7f52fc162f20 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc162f20 msgr2=0x7f52fc163550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc162f20 0x7f52fc163550 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f52fc05ce10 tx=0x7f52ec006df0).stop 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- 172.21.15.181:0/2047749370 shutdown_connections 2020-11-16T22:29:33.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- 172.21.15.181:0/2047749370 >> v2:172.21.15.175:3300/0 conn(0x7f52fc163a40 0x7f52fc15ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc1428e0 0x7f52fc162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 --2- 172.21.15.181:0/2047749370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc162f20 0x7f52fc163550 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- 172.21.15.181:0/2047749370 >> 172.21.15.181:0/2047749370 conn(0x7f52fc060bc0 msgr2=0x7f52fc0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:33.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- 172.21.15.181:0/2047749370 shutdown_connections 2020-11-16T22:29:33.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.525+0000 7f5300b80700 1 -- 172.21.15.181:0/2047749370 wait complete. 2020-11-16T22:29:33.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 Processor -- start 2020-11-16T22:29:33.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 -- start start 2020-11-16T22:29:33.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 0x7f52fc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc162f20 0x7f52fc157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc173000 con 0x7f52fc162f20 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1732a0 con 0x7f52fc1428e0 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f5300b80700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc173540 con 0x7f52fc163a40 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f52fad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f52fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 0x7f52fc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f52fad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38972/0 (socket says 172.21.15.181:38972) 2020-11-16T22:29:33.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f52fad9d700 1 -- 172.21.15.181:0/4256990482 learned_addr learned my addr 172.21.15.181:0/4256990482 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.529+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc162f20 0x7f52fc157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 -- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 msgr2=0x7f52fc16fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 -- 172.21.15.181:0/4256990482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc162f20 msgr2=0x7f52fc157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc162f20 0x7f52fc157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 -- 172.21.15.181:0/4256990482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc1737e0 con 0x7f52fc1428e0 2020-11-16T22:29:33.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fad9d700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52fa59c700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 0x7f52fc0ce370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f52f4009fd0 tx=0x7f52f4009290).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400c9b0 con 0x7f52fc1428e0 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f400cb10 con 0x7f52fc1428e0 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc173a80 con 0x7f52fc1428e0 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400ce90 con 0x7f52fc1428e0 2020-11-16T22:29:33.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0b6d50 con 0x7f52fc1428e0 2020-11-16T22:29:33.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc069eb0 con 0x7f52fc1428e0 2020-11-16T22:29:33.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f52f4022070 con 0x7f52fc1428e0 2020-11-16T22:29:33.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 0x7f52d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52f406e430 con 0x7f52fc1428e0 2020-11-16T22:29:33.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.533+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 0x7f52d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.537+0000 7f52f9d9b700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 0x7f52d8057520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f52fc05ce10 tx=0x7f52ec015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.541+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f52f4044690 con 0x7f52fc1428e0 2020-11-16T22:29:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.741+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52fc15afc0 con 0x7f52fc1428e0 2020-11-16T22:29:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.741+0000 7f52f27fc700 1 -- 172.21.15.181:0/4256990482 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4629 (secure 0 0 0) 0x7f52f4016070 con 0x7f52fc1428e0 2020-11-16T22:29:33.725 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:33.725 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":46,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:33.365462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:33.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 msgr2=0x7f52d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 0x7f52d8057520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f52fc05ce10 tx=0x7f52ec015000).stop 2020-11-16T22:29:33.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 msgr2=0x7f52fc0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 0x7f52fc0ce370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f52f4009fd0 tx=0x7f52f4009290).stop 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 shutdown_connections 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f52d8055060 0x7f52d8057520 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52fc163a40 0x7f52fc16fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52fc1428e0 0x7f52fc0ce370 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 --2- 172.21.15.181:0/4256990482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52fc162f20 0x7f52fc157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 >> 172.21.15.181:0/4256990482 conn(0x7f52fc060bc0 msgr2=0x7f52fc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 shutdown_connections 2020-11-16T22:29:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.745+0000 7f5300b80700 1 -- 172.21.15.181:0/4256990482 wait complete. 2020-11-16T22:29:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 46 2020-11-16T22:29:33.740 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:33.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 Processor -- start 2020-11-16T22:29:33.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 -- start start 2020-11-16T22:29:33.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1b881405f0 0x7f1b881409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b880c94f0 con 0x7f1b88066a20 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b880c9790 con 0x7f1b881405f0 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8ea85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880c9a30 con 0x7f1b880647a0 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38984/0 (socket says 172.21.15.181:38984) 2020-11-16T22:29:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 -- 172.21.15.181:0/1803690715 learned_addr learned my addr 172.21.15.181:0/1803690715 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:33.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8da83700 1 --2- 172.21.15.181:0/1803690715 >> v2:172.21.15.175:3300/0 conn(0x7f1b881405f0 0x7f1b881409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 -- 172.21.15.181:0/1803690715 >> v2:172.21.15.175:3300/0 conn(0x7f1b881405f0 msgr2=0x7f1b881409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:33.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 --2- 172.21.15.181:0/1803690715 >> v2:172.21.15.175:3300/0 conn(0x7f1b881405f0 0x7f1b881409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.877+0000 7f1b8d282700 1 -- 172.21.15.181:0/1803690715 >> v1:172.21.15.175:6789/0 conn(0x7f1b88066a20 legacy=0x7f1b8815f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8d282700 1 -- 172.21.15.181:0/1803690715 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880c9cd0 con 0x7f1b880647a0 2020-11-16T22:29:33.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8d282700 1 --2- 172.21.15.181:0/1803690715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f1b78006fd0 tx=0x7f1b78006eb0).ready entity=mon.2 client_cookie=99a94f590e8de3e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ca81700 1 -- 172.21.15.181:0/1803690715 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78009710 con 0x7f1b880647a0 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ca81700 1 -- 172.21.15.181:0/1803690715 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78009870 con 0x7f1b880647a0 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ca81700 1 -- 172.21.15.181:0/1803690715 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78009bf0 con 0x7f1b880647a0 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 -- 172.21.15.181:0/1803690715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 msgr2=0x7f1b88066530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/1803690715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f1b78006fd0 tx=0x7f1b78006eb0).stop 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 -- 172.21.15.181:0/1803690715 shutdown_connections 2020-11-16T22:29:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/1803690715 >> v2:172.21.15.175:3300/0 conn(0x7f1b881405f0 0x7f1b881409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/1803690715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b880647a0 0x7f1b88066530 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 -- 172.21.15.181:0/1803690715 >> 172.21.15.181:0/1803690715 conn(0x7f1b880d4590 msgr2=0x7f1b880d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:33.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 -- 172.21.15.181:0/1803690715 shutdown_connections 2020-11-16T22:29:33.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.881+0000 7f1b8ea85700 1 -- 172.21.15.181:0/1803690715 wait complete. 2020-11-16T22:29:33.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 Processor -- start 2020-11-16T22:29:33.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- start start 2020-11-16T22:29:33.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b880647a0 0x7f1b880b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 0x7f1b880b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8816d100 con 0x7f1b880647a0 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880b7f00 con 0x7f1b88066a20 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88171370 con 0x7f1b881405f0 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38988/0 (socket says 172.21.15.181:38988) 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8d282700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 0x7f1b880b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 -- 172.21.15.181:0/2725780251 learned_addr learned my addr 172.21.15.181:0/2725780251 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8da83700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b880647a0 0x7f1b880b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 -- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 msgr2=0x7f1b880b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 0x7f1b880b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 -- 172.21.15.181:0/2725780251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b880647a0 msgr2=0x7f1b880b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b880647a0 0x7f1b880b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 -- 172.21.15.181:0/2725780251 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8816d2f0 con 0x7f1b881405f0 2020-11-16T22:29:33.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8d282700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 0x7f1b880b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:33.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8e284700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f1b8000cb50 tx=0x7f1b80007e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000a7f0 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000a970 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8816d590 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000ad30 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.885+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8816df70 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b88171540 con 0x7f1b881405f0 2020-11-16T22:29:33.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1b80010070 con 0x7f1b881405f0 2020-11-16T22:29:33.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b75ffb700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 0x7f1b60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:33.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b8da83700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 0x7f1b60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:33.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b8006dd70 con 0x7f1b881405f0 2020-11-16T22:29:33.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.889+0000 7f1b8da83700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 0x7f1b60057520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1b84006fd0 tx=0x7f1b84006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:33.893+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1b80043ec0 con 0x7f1b881405f0 2020-11-16T22:29:34.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.041+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b88044f60 con 0x7f1b881405f0 2020-11-16T22:29:34.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b75ffb700 1 -- 172.21.15.181:0/2725780251 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f1b800474e0 con 0x7f1b881405f0 2020-11-16T22:29:34.027 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:34.027 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:31.359585+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:34.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 msgr2=0x7f1b60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 0x7f1b60057520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1b84006fd0 tx=0x7f1b84006df0).stop 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 msgr2=0x7f1b8815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f1b8000cb50 tx=0x7f1b80007e80).stop 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 shutdown_connections 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b60055060 0x7f1b60057520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b881405f0 0x7f1b8815ff50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b880647a0 0x7f1b880b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 --2- 172.21.15.181:0/2725780251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b88066a20 0x7f1b880b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 >> 172.21.15.181:0/2725780251 conn(0x7f1b880d4590 msgr2=0x7f1b880d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 shutdown_connections 2020-11-16T22:29:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.045+0000 7f1b8ea85700 1 -- 172.21.15.181:0/2725780251 wait complete. 2020-11-16T22:29:34.042 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:29:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 Processor -- start 2020-11-16T22:29:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 -- start start 2020-11-16T22:29:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2eb0081550 0x7f2eb01612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 0x7f2eb01455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb00d1bc0 con 0x7f2eb0081170 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb00d1e60 con 0x7f2eb0081550 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb64b5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00d2100 con 0x7f2eb01617b0 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2eb0081550 0x7f2eb01612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb5cb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 0x7f2eb01455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2eb0081550 0x7f2eb01612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59386/0 (socket says 172.21.15.181:59386) 2020-11-16T22:29:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb5cb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 0x7f2eb01455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38996/0 (socket says 172.21.15.181:38996) 2020-11-16T22:29:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/4275272381 learned_addr learned my addr 172.21.15.181:0/4275272381 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb54b3700 1 --1- 172.21.15.181:0/4275272381 >> v1:172.21.15.181:6789/0 conn(0x7f2eb0081170 0x7f2eb00d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45456/0 (socket says 172.21.15.181:45456) 2020-11-16T22:29:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 msgr2=0x7f2eb01455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 --2- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 0x7f2eb01455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/4275272381 >> v1:172.21.15.181:6789/0 conn(0x7f2eb0081170 legacy=0x7f2eb00d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.193+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/4275272381 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb00d23a0 con 0x7f2eb0081550 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb4cb2700 1 --2- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb0081550 0x7f2eb01612c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f2ea0006fd0 tx=0x7f2ea0006eb0).ready entity=mon.1 client_cookie=321ec4e43ba68e43 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eaffff700 1 -- 172.21.15.181:0/4275272381 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0009810 con 0x7f2eb0081550 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eaffff700 1 -- 172.21.15.181:0/4275272381 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea0009970 con 0x7f2eb0081550 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eaffff700 1 -- 172.21.15.181:0/4275272381 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0009cf0 con 0x7f2eb0081550 2020-11-16T22:29:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb0081550 msgr2=0x7f2eb01612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb0081550 0x7f2eb01612c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f2ea0006fd0 tx=0x7f2ea0006eb0).stop 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- 172.21.15.181:0/4275272381 shutdown_connections 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb01617b0 0x7f2eb01455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/4275272381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb0081550 0x7f2eb01612c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- 172.21.15.181:0/4275272381 >> 172.21.15.181:0/4275272381 conn(0x7f2eb00d67e0 msgr2=0x7f2eb00d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- 172.21.15.181:0/4275272381 shutdown_connections 2020-11-16T22:29:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- 172.21.15.181:0/4275272381 wait complete. 2020-11-16T22:29:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 Processor -- start 2020-11-16T22:29:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.197+0000 7f2eb64b5700 1 -- start start 2020-11-16T22:29:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 0x7f2eb00b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb01617b0 0x7f2eb00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00b7040 con 0x7f2eb0081170 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0164970 con 0x7f2eb01617b0 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb016f180 con 0x7f2eb0081550 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:38998/0 (socket says 172.21.15.181:38998) 2020-11-16T22:29:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/3543811638 learned_addr learned my addr 172.21.15.181:0/3543811638 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb5cb4700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb01617b0 0x7f2eb00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb54b3700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 0x7f2eb00b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb01617b0 msgr2=0x7f2eb00cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb01617b0 0x7f2eb00cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/3543811638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 msgr2=0x7f2eb00b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 0x7f2eb00b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 -- 172.21.15.181:0/3543811638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb016f3d0 con 0x7f2eb0081550 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb54b3700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 0x7f2eb00b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb4cb2700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f2ea0006ef0 tx=0x7f2ea0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0011d50 con 0x7f2eb0081550 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea001e460 con 0x7f2eb0081550 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb016f670 con 0x7f2eb0081550 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea001e820 con 0x7f2eb0081550 2020-11-16T22:29:34.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb016fbb0 con 0x7f2eb0081550 2020-11-16T22:29:34.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.201+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb0069eb0 con 0x7f2eb0081550 2020-11-16T22:29:34.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.205+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2ea0025030 con 0x7f2eb0081550 2020-11-16T22:29:34.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.205+0000 7f2ead7fa700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 0x7f2e88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.205+0000 7f2eb54b3700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 0x7f2e88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.205+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ea006f520 con 0x7f2eb0081550 2020-11-16T22:29:34.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.205+0000 7f2eb54b3700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 0x7f2e88057520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f2ea4000bc0 tx=0x7f2ea4007100).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.209+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ea0045780 con 0x7f2eb0081550 2020-11-16T22:29:34.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.413+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f2eb005d4e0 con 0x7f2eb0081550 2020-11-16T22:29:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.421+0000 7f2ead7fa700 1 -- 172.21.15.181:0/3543811638 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2ea0023020 con 0x7f2eb0081550 2020-11-16T22:29:34.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 msgr2=0x7f2e88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 0x7f2e88057520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f2ea4000bc0 tx=0x7f2ea4007100).stop 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 msgr2=0x7f2eb0164480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f2ea0006ef0 tx=0x7f2ea0015000).stop 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 shutdown_connections 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e88055060 0x7f2e88057520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb01617b0 0x7f2eb00cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2eb0081170 0x7f2eb00b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 --2- 172.21.15.181:0/3543811638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb0081550 0x7f2eb0164480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 >> 172.21.15.181:0/3543811638 conn(0x7f2eb00d67e0 msgr2=0x7f2eb00d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 shutdown_connections 2020-11-16T22:29:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.425+0000 7f2eb64b5700 1 -- 172.21.15.181:0/3543811638 wait complete. 2020-11-16T22:29:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:29:34.422 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 Processor -- start 2020-11-16T22:29:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 -- start start 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80ec0b3220 0x7f80ec0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec168390 con 0x7f80ec166bb0 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec167770 con 0x7f80ec0b3220 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80f1ea0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0b5f00 con 0x7f80ec167ac0 2020-11-16T22:29:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80ec0b3220 0x7f80ec0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ebfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f80ec166bb0 0x7f80ec167060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45472/0 (socket says 172.21.15.181:45472) 2020-11-16T22:29:34.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ebfff700 1 -- 172.21.15.181:0/387990753 learned_addr learned my addr 172.21.15.181:0/387990753 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:34.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ea7fc700 1 -- 172.21.15.181:0/387990753 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248351299 0 0) 0x7f80ec168390 con 0x7f80ec166bb0 2020-11-16T22:29:34.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 -- 172.21.15.181:0/387990753 >> v2:172.21.15.181:3300/0 conn(0x7f80ec0b3220 msgr2=0x7f80ec0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:34.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 --2- 172.21.15.181:0/387990753 >> v2:172.21.15.181:3300/0 conn(0x7f80ec0b3220 0x7f80ec0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 -- 172.21.15.181:0/387990753 >> v1:172.21.15.181:6789/0 conn(0x7f80ec166bb0 legacy=0x7f80ec167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 -- 172.21.15.181:0/387990753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec0b2060 con 0x7f80ec167ac0 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80eb7fe700 1 --2- 172.21.15.181:0/387990753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec167ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f80dc009c90 tx=0x7f80dc0091d0).ready entity=mon.2 client_cookie=7838d11fd37196d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ea7fc700 1 -- 172.21.15.181:0/387990753 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00ba70 con 0x7f80ec167ac0 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ea7fc700 1 -- 172.21.15.181:0/387990753 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc00bbd0 con 0x7f80ec167ac0 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.561+0000 7f80ea7fc700 1 -- 172.21.15.181:0/387990753 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc011400 con 0x7f80ec167ac0 2020-11-16T22:29:34.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- 172.21.15.181:0/387990753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 msgr2=0x7f80ec167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/387990753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec167ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f80dc009c90 tx=0x7f80dc0091d0).stop 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- 172.21.15.181:0/387990753 shutdown_connections 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/387990753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec167ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/387990753 >> v2:172.21.15.181:3300/0 conn(0x7f80ec0b3220 0x7f80ec0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- 172.21.15.181:0/387990753 >> 172.21.15.181:0/387990753 conn(0x7f80ec060bc0 msgr2=0x7f80ec0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- 172.21.15.181:0/387990753 shutdown_connections 2020-11-16T22:29:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- 172.21.15.181:0/387990753 wait complete. 2020-11-16T22:29:34.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 Processor -- start 2020-11-16T22:29:34.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.565+0000 7f80f1ea0700 1 -- start start 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec166bb0 0x7f80ec162d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec169320 con 0x7f80ec166bb0 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1695c0 con 0x7f80ec0b3220 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec169860 con 0x7f80ec167ac0 2020-11-16T22:29:34.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59404/0 (socket says 172.21.15.181:59404) 2020-11-16T22:29:34.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 -- 172.21.15.181:0/2900786659 learned_addr learned my addr 172.21.15.181:0/2900786659 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:34.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eaffd700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec166bb0 0x7f80ec162d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 -- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 msgr2=0x7f80ec163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 -- 172.21.15.181:0/2900786659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec166bb0 msgr2=0x7f80ec162d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec166bb0 0x7f80ec162d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 -- 172.21.15.181:0/2900786659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec169b00 con 0x7f80ec0b3220 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eb7fe700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f80dc00b770 tx=0x7f80dc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc01c5f0 con 0x7f80ec0b3220 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc01c750 con 0x7f80ec0b3220 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec169da0 con 0x7f80ec0b3220 2020-11-16T22:29:34.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc01cad0 con 0x7f80ec0b3220 2020-11-16T22:29:34.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec168f90 con 0x7f80ec0b3220 2020-11-16T22:29:34.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec069eb0 con 0x7f80ec0b3220 2020-11-16T22:29:34.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f80dc017070 con 0x7f80ec0b3220 2020-11-16T22:29:34.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 0x7f80bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:34.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f80dc0700c0 con 0x7f80ec0b3220 2020-11-16T22:29:34.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.569+0000 7f80eaffd700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 0x7f80bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:34.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.573+0000 7f80eaffd700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 0x7f80bc057520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f80ec0b8540 tx=0x7f80d4006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.581+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f80dc046320 con 0x7f80ec0b3220 2020-11-16T22:29:34.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.777+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80ec061100 con 0x7f80ec0b3220 2020-11-16T22:29:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.777+0000 7f80e37fe700 1 -- 172.21.15.181:0/2900786659 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4629 (secure 0 0 0) 0x7f80dc015020 con 0x7f80ec0b3220 2020-11-16T22:29:34.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:34.762 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":46,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:33.365462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 msgr2=0x7f80bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 0x7f80bc057520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f80ec0b8540 tx=0x7f80d4006df0).stop 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 msgr2=0x7f80ec168aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f80dc00b770 tx=0x7f80dc016000).stop 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 shutdown_connections 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f80bc055060 0x7f80bc057520 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80ec167ac0 0x7f80ec163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80ec0b3220 0x7f80ec168aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 --2- 172.21.15.181:0/2900786659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec166bb0 0x7f80ec162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 >> 172.21.15.181:0/2900786659 conn(0x7f80ec060bc0 msgr2=0x7f80ec164840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 shutdown_connections 2020-11-16T22:29:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:34.781+0000 7f80f1ea0700 1 -- 172.21.15.181:0/2900786659 wait complete. 2020-11-16T22:29:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 46 2020-11-16T22:29:35.681 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:29:30.354786+0000', 'modified': '2020-11-16T22:29:33.365462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5614': {'gid': 5614, 'name': 'i', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2753478724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2753478724}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2753478724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:29:35.681 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:29:35.682 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:29:35.820 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5614,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.4702402909999996,"mdsmap_epoch":46,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":12.798517507} 2020-11-16T22:29:35.821 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5614, "mdsmap_epoch": 46, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 5.470240291, "state": "up:active", "uptime": 12.798517507, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:29:35.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:29:35.822 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:29:35.822 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:29:35.822 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:29:35.822 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:29:35.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:29:35.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:29:35.823 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:29:35.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:35.824 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:29:35.841 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:29:35.841 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:29:35.841 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:29:35.841 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.841 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.843 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:29:35.844 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:29:35.845 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.845 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:29:35.845 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:29:35.845 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:29:35.845 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:29:35.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:35.846 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:29:35.881 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:29:35.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:35.884 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:29:35.943 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:29:35.943 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:29:35.943 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:29:35.944 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:29:35.944 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:29:35.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:35.945 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:29:35.945 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:29:35.946 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:29:35.946 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:29:35.946 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:29:35.946 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:29:36.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:36.046 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:29:36.061 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:29:36.061 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:29:36.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:29:36.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:36.063 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:29:36.092 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:29:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.241+0000 7fe1ee2fa700 1 Processor -- start 2020-11-16T22:29:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.241+0000 7fe1ee2fa700 1 -- start start 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1e8160530 0x7fe1e8061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8063b10 con 0x7fe1e8142660 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e8062700 con 0x7fe1e8160530 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80629a0 con 0x7fe1e8142a40 2020-11-16T22:29:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39020/0 (socket says 172.21.15.181:39020) 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 -- 172.21.15.181:0/4029310128 learned_addr learned my addr 172.21.15.181:0/4029310128 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1edaf9700 1 --2- 172.21.15.181:0/4029310128 >> v2:172.21.15.175:3300/0 conn(0x7fe1e8160530 0x7fe1e8061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 -- 172.21.15.181:0/4029310128 >> v2:172.21.15.175:3300/0 conn(0x7fe1e8160530 msgr2=0x7fe1e8061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 --2- 172.21.15.181:0/4029310128 >> v2:172.21.15.175:3300/0 conn(0x7fe1e8160530 0x7fe1e8061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 -- 172.21.15.181:0/4029310128 >> v1:172.21.15.175:6789/0 conn(0x7fe1e8142660 legacy=0x7fe1e8063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 -- 172.21.15.181:0/4029310128 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8062c40 con 0x7fe1e8142a40 2020-11-16T22:29:36.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ecaf7700 1 --2- 172.21.15.181:0/4029310128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe1d8006fd0 tx=0x7fe1d8006eb0).ready entity=mon.2 client_cookie=31c6a1c708344ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1dffff700 1 -- 172.21.15.181:0/4029310128 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80098d0 con 0x7fe1e8142a40 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1dffff700 1 -- 172.21.15.181:0/4029310128 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d8009a30 con 0x7fe1e8142a40 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1dffff700 1 -- 172.21.15.181:0/4029310128 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8009db0 con 0x7fe1e8142a40 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/4029310128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 msgr2=0x7fe1e8063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/4029310128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe1d8006fd0 tx=0x7fe1d8006eb0).stop 2020-11-16T22:29:36.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/4029310128 shutdown_connections 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/4029310128 >> v2:172.21.15.175:3300/0 conn(0x7fe1e8160530 0x7fe1e8061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/4029310128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8063620 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/4029310128 >> 172.21.15.181:0/4029310128 conn(0x7fe1e80d2dc0 msgr2=0x7fe1e80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/4029310128 shutdown_connections 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/4029310128 wait complete. 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 Processor -- start 2020-11-16T22:29:36.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.245+0000 7fe1ee2fa700 1 -- start start 2020-11-16T22:29:36.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8160530 0x7fe1e8140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815d4e0 con 0x7fe1e8160530 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81456e0 con 0x7fe1e8142660 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8145980 con 0x7fe1e8142a40 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ecaf7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59416/0 (socket says 172.21.15.181:59416) 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 -- 172.21.15.181:0/3009624070 learned_addr learned my addr 172.21.15.181:0/3009624070 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:36.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1edaf9700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8160530 0x7fe1e8140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 -- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 msgr2=0x7fe1e8140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 -- 172.21.15.181:0/3009624070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8160530 msgr2=0x7fe1e8140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8160530 0x7fe1e8140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 -- 172.21.15.181:0/3009624070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8145c20 con 0x7fe1e8142660 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ed2f8700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe1e400c6e0 tx=0x7fe1e4007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4007020 con 0x7fe1e8142660 2020-11-16T22:29:36.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8145ec0 con 0x7fe1e8142660 2020-11-16T22:29:36.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e8145240 con 0x7fe1e8142660 2020-11-16T22:29:36.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4007180 con 0x7fe1e8142660 2020-11-16T22:29:36.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400e560 con 0x7fe1e8142660 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.249+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e815fb10 con 0x7fe1e8142660 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.253+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe1e4010070 con 0x7fe1e8142660 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.253+0000 7fe1dd7fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.253+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe1e406cf90 con 0x7fe1e8142660 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.253+0000 7fe1ecaf7700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.253+0000 7fe1ecaf7700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe1e8163170 tx=0x7fe1d8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.261+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe1e40431f0 con 0x7fe1e8142660 2020-11-16T22:29:36.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.421+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e80b1bb0 con 0x7fe1e8142660 2020-11-16T22:29:36.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.421+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/3009624070 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fe1e4024200 con 0x7fe1e8142660 2020-11-16T22:29:36.406 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:36.406 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:31.359585+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 msgr2=0x7fe1c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe1e8163170 tx=0x7fe1d8015000).stop 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 msgr2=0x7fe1e8144d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe1e400c6e0 tx=0x7fe1e4007c50).stop 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 shutdown_connections 2020-11-16T22:29:36.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8160530 0x7fe1e8140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8142660 0x7fe1e8144d50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 --2- 172.21.15.181:0/3009624070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142a40 0x7fe1e8140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 >> 172.21.15.181:0/3009624070 conn(0x7fe1e80d2dc0 msgr2=0x7fe1e80d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 shutdown_connections 2020-11-16T22:29:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.425+0000 7fe1ee2fa700 1 -- 172.21.15.181:0/3009624070 wait complete. 2020-11-16T22:29:36.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:29:28.703069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:29:29.511094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:29:36.613 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.769+0000 7fb288a17700 1 Processor -- start 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.769+0000 7fb288a17700 1 -- start start 2020-11-16T22:29:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb28415b810 0x7fb28415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840659d0 con 0x7fb28415fbe0 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28415d770 con 0x7fb28415b810 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28415da10 con 0x7fb28415cc00 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39038/0 (socket says 172.21.15.181:39038) 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 -- 172.21.15.181:0/1413821135 learned_addr learned my addr 172.21.15.181:0/1413821135 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb2837fe700 1 --2- 172.21.15.181:0/1413821135 >> v2:172.21.15.175:3300/0 conn(0x7fb28415b810 0x7fb28415f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 -- 172.21.15.181:0/1413821135 >> v2:172.21.15.175:3300/0 conn(0x7fb28415b810 msgr2=0x7fb28415f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 --2- 172.21.15.181:0/1413821135 >> v2:172.21.15.175:3300/0 conn(0x7fb28415b810 0x7fb28415f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 -- 172.21.15.181:0/1413821135 >> v1:172.21.15.181:6789/0 conn(0x7fb28415fbe0 legacy=0x7fb28415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 -- 172.21.15.181:0/1413821135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28415dcb0 con 0x7fb28415cc00 2020-11-16T22:29:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb283fff700 1 --2- 172.21.15.181:0/1413821135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fb26c009d50 tx=0x7fb26c009290).ready entity=mon.2 client_cookie=ef3b1d4fe71a30a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb2827fc700 1 -- 172.21.15.181:0/1413821135 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c00bc80 con 0x7fb28415cc00 2020-11-16T22:29:36.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb2827fc700 1 -- 172.21.15.181:0/1413821135 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c00bde0 con 0x7fb28415cc00 2020-11-16T22:29:36.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb2827fc700 1 -- 172.21.15.181:0/1413821135 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011610 con 0x7fb28415cc00 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- 172.21.15.181:0/1413821135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 msgr2=0x7fb28415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 --2- 172.21.15.181:0/1413821135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fb26c009d50 tx=0x7fb26c009290).stop 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- 172.21.15.181:0/1413821135 shutdown_connections 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 --2- 172.21.15.181:0/1413821135 >> v2:172.21.15.175:3300/0 conn(0x7fb28415b810 0x7fb28415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 --2- 172.21.15.181:0/1413821135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb28415eb00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- 172.21.15.181:0/1413821135 >> 172.21.15.181:0/1413821135 conn(0x7fb2840d2dc0 msgr2=0x7fb2840ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:36.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- 172.21.15.181:0/1413821135 shutdown_connections 2020-11-16T22:29:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.773+0000 7fb288a17700 1 -- 172.21.15.181:0/1413821135 wait complete. 2020-11-16T22:29:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 Processor -- start 2020-11-16T22:29:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- start start 2020-11-16T22:29:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb2840cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840b3230 con 0x7fb28415b810 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28416fc50 con 0x7fb28415fbe0 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840b3460 con 0x7fb28415cc00 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb282ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb2840cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb2837fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59432/0 (socket says 172.21.15.181:59432) 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb2837fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:45200/0 (socket says 172.21.15.181:45200) 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 -- 172.21.15.181:0/2085220814 learned_addr learned my addr 172.21.15.181:0/2085220814 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb282ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb2840cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39042/0 (socket says 172.21.15.181:39042) 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 -- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 msgr2=0x7fb2840cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb2840cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 -- 172.21.15.181:0/2085220814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 msgr2=0x7fb2840cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 -- 172.21.15.181:0/2085220814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840b3700 con 0x7fb28415fbe0 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb2837fe700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb283fff700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fb28415db50 tx=0x7fb26c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c00bc80 con 0x7fb28415fbe0 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c009e90 con 0x7fb28415fbe0 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840b39a0 con 0x7fb28415fbe0 2020-11-16T22:29:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c010cb0 con 0x7fb28415fbe0 2020-11-16T22:29:36.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.777+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb284158d90 con 0x7fb28415fbe0 2020-11-16T22:29:36.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2840b3e40 con 0x7fb28415fbe0 2020-11-16T22:29:36.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb26c017070 con 0x7fb28415fbe0 2020-11-16T22:29:36.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb27b7fe700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 0x7fb254057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:36.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb26c06fec0 con 0x7fb28415fbe0 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb2837fe700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 0x7fb254057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.781+0000 7fb2837fe700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 0x7fb254057520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb27c00da90 tx=0x7fb27c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.789+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb2840b3e40 con 0x7fb28415fbe0 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb2840b3e40 con 0x7fb28415fbe0 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb27b7fe700 1 -- 172.21.15.181:0/2085220814 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v49) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb2840b3e40 con 0x7fb28415fbe0 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 msgr2=0x7fb254057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 0x7fb254057520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb27c00da90 tx=0x7fb27c008040).stop 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 msgr2=0x7fb2841588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.941+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fb28415db50 tx=0x7fb26c016000).stop 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 shutdown_connections 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb254055060 0x7fb254057520 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb28415fbe0 0x7fb2841588a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28415b810 0x7fb2840cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 --2- 172.21.15.181:0/2085220814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb28415cc00 0x7fb2840cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 >> 172.21.15.181:0/2085220814 conn(0x7fb2840d2dc0 msgr2=0x7fb28415bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 shutdown_connections 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:36.945+0000 7fb288a17700 1 -- 172.21.15.181:0/2085220814 wait complete. 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:29:37.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 Processor -- start 2020-11-16T22:29:37.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 -- start start 2020-11-16T22:29:37.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 0x7f7ac8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac8061b50 con 0x7f7ac8145a30 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80618b0 con 0x7f7ac80816e0 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.129+0000 7f7acd50e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80cb930 con 0x7f7ac81492c0 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 0x7f7ac8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 0x7f7ac8145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59440/0 (socket says 172.21.15.181:59440) 2020-11-16T22:29:37.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39052/0 (socket says 172.21.15.181:39052) 2020-11-16T22:29:37.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac77fe700 1 -- 172.21.15.181:0/2358711846 learned_addr learned my addr 172.21.15.181:0/2358711846 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:37.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac7fff700 1 --1- 172.21.15.181:0/2358711846 >> v1:172.21.15.175:6789/0 conn(0x7f7ac8145a30 0x7f7ac8080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:38624/0 (socket says 172.21.15.181:38624) 2020-11-16T22:29:37.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 -- 172.21.15.181:0/2358711846 >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 msgr2=0x7f7ac8145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:37.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 --2- 172.21.15.181:0/2358711846 >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 0x7f7ac8145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 -- 172.21.15.181:0/2358711846 >> v1:172.21.15.175:6789/0 conn(0x7f7ac8145a30 legacy=0x7f7ac8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac6ffd700 1 -- 172.21.15.181:0/2358711846 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382039646 0 0) 0x7f7ac8061b50 con 0x7f7ac8145a30 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 -- 172.21.15.181:0/2358711846 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80cbbd0 con 0x7f7ac81492c0 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7accd0d700 1 --2- 172.21.15.181:0/2358711846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7ac0009c90 tx=0x7f7ac00091d0).ready entity=mon.2 client_cookie=3a547d7e18a4c133 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac6ffd700 1 -- 172.21.15.181:0/2358711846 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000ba70 con 0x7f7ac81492c0 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac6ffd700 1 -- 172.21.15.181:0/2358711846 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000bbd0 con 0x7f7ac81492c0 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7ac6ffd700 1 -- 172.21.15.181:0/2358711846 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0011400 con 0x7f7ac81492c0 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 -- 172.21.15.181:0/2358711846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 msgr2=0x7f7ac80cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 --2- 172.21.15.181:0/2358711846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7ac0009c90 tx=0x7f7ac00091d0).stop 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 -- 172.21.15.181:0/2358711846 shutdown_connections 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 --2- 172.21.15.181:0/2358711846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac80cc5b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 --2- 172.21.15.181:0/2358711846 >> v2:172.21.15.181:3300/0 conn(0x7f7ac80816e0 0x7f7ac8145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 -- 172.21.15.181:0/2358711846 >> 172.21.15.181:0/2358711846 conn(0x7f7ac80d68c0 msgr2=0x7f7ac80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 -- 172.21.15.181:0/2358711846 shutdown_connections 2020-11-16T22:29:37.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.133+0000 7f7acd50e700 1 -- 172.21.15.181:0/2358711846 wait complete. 2020-11-16T22:29:37.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 Processor -- start 2020-11-16T22:29:37.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 -- start start 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ac80816e0 0x7f7ac8170810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac8164880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8171250 con 0x7f7ac80816e0 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81714f0 con 0x7f7ac8145a30 2020-11-16T22:29:37.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7acd50e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8171790 con 0x7f7ac81492c0 2020-11-16T22:29:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59446/0 (socket says 172.21.15.181:59446) 2020-11-16T22:29:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 -- 172.21.15.181:0/1827209989 learned_addr learned my addr 172.21.15.181:0/1827209989 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7accd0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac8164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac7fff700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ac80816e0 0x7f7ac8170810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 -- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 msgr2=0x7f7ac8164880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac8164880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 -- 172.21.15.181:0/1827209989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ac80816e0 msgr2=0x7f7ac8170810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ac80816e0 0x7f7ac8170810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 -- 172.21.15.181:0/1827209989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8171a30 con 0x7f7ac8145a30 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7accd0d700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac8164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7ac77fe700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7ab800f830 tx=0x7f7ab8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8011660 con 0x7f7ac8145a30 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.137+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab80117c0 con 0x7f7ac8145a30 2020-11-16T22:29:37.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac8171cd0 con 0x7f7ac8145a30 2020-11-16T22:29:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8011b40 con 0x7f7ac8145a30 2020-11-16T22:29:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac8172410 con 0x7f7ac8145a30 2020-11-16T22:29:37.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac8044f60 con 0x7f7ac8145a30 2020-11-16T22:29:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7ab8023070 con 0x7f7ac8145a30 2020-11-16T22:29:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7aa7fff700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 0x7f7aa0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7ab806e000 con 0x7f7ac8145a30 2020-11-16T22:29:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7ac7fff700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 0x7f7aa0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.141+0000 7f7ac7fff700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 0x7f7aa0057520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7abc0011a0 tx=0x7f7abc007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.153+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7ab8044260 con 0x7f7ac8145a30 2020-11-16T22:29:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ac8172660 con 0x7f7ac8145a30 2020-11-16T22:29:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7aa7fff700 1 -- 172.21.15.181:0/1827209989 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ab8017070 con 0x7f7ac8145a30 2020-11-16T22:29:37.288 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 msgr2=0x7f7aa0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 0x7f7aa0057520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7abc0011a0 tx=0x7f7abc007a20).stop 2020-11-16T22:29:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 msgr2=0x7f7ac813e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7ab800f830 tx=0x7f7ab8008040).stop 2020-11-16T22:29:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 shutdown_connections 2020-11-16T22:29:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7aa0055060 0x7f7aa0057520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac81492c0 0x7f7ac8164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ac80816e0 0x7f7ac8170810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 --2- 172.21.15.181:0/1827209989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac8145a30 0x7f7ac813e860 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 >> 172.21.15.181:0/1827209989 conn(0x7f7ac80d68c0 msgr2=0x7f7ac80d47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 shutdown_connections 2020-11-16T22:29:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.305+0000 7f7acd50e700 1 -- 172.21.15.181:0/1827209989 wait complete. 2020-11-16T22:29:37.303 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:29:37.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 Processor -- start 2020-11-16T22:29:37.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- start start 2020-11-16T22:29:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae200629b0 0x7fae20080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae20160e40 con 0x7fae2015f5b0 2020-11-16T22:29:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae20160ba0 con 0x7fae200615f0 2020-11-16T22:29:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae201609a0 con 0x7fae200629b0 2020-11-16T22:29:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1f59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae200629b0 0x7fae20080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1e59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae200615f0 0x7fae200622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45524/0 (socket says 172.21.15.181:45524) 2020-11-16T22:29:37.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1e59c700 1 -- 172.21.15.181:0/1359991047 learned_addr learned my addr 172.21.15.181:0/1359991047 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:37.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1f59e700 1 -- 172.21.15.181:0/1359991047 >> v1:172.21.15.181:6789/0 conn(0x7fae200615f0 legacy=0x7fae200622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1f59e700 1 -- 172.21.15.181:0/1359991047 >> v1:172.21.15.175:6789/0 conn(0x7fae2015f5b0 legacy=0x7fae200d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1f59e700 1 -- 172.21.15.181:0/1359991047 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae20060970 con 0x7fae200629b0 2020-11-16T22:29:37.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1dd9b700 1 -- 172.21.15.181:0/1359991047 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270613958 0 0) 0x7fae20160ba0 con 0x7fae200615f0 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1f59e700 1 --2- 172.21.15.181:0/1359991047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae20080fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fae1800b9a0 tx=0x7fae18006df0).ready entity=mon.1 client_cookie=6578831affee6722 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1dd9b700 1 -- 172.21.15.181:0/1359991047 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180097b0 con 0x7fae200629b0 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1dd9b700 1 -- 172.21.15.181:0/1359991047 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18009910 con 0x7fae200629b0 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae1dd9b700 1 -- 172.21.15.181:0/1359991047 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18009c90 con 0x7fae200629b0 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- 172.21.15.181:0/1359991047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 msgr2=0x7fae20080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 --2- 172.21.15.181:0/1359991047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae20080fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fae1800b9a0 tx=0x7fae18006df0).stop 2020-11-16T22:29:37.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- 172.21.15.181:0/1359991047 shutdown_connections 2020-11-16T22:29:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 --2- 172.21.15.181:0/1359991047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae20080fd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- 172.21.15.181:0/1359991047 >> 172.21.15.181:0/1359991047 conn(0x7fae200d69a0 msgr2=0x7fae200d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- 172.21.15.181:0/1359991047 shutdown_connections 2020-11-16T22:29:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.449+0000 7fae25079700 1 -- 172.21.15.181:0/1359991047 wait complete. 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 Processor -- start 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- start start 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae200bb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae2015f5b0 0x7fae200bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200a7b70 con 0x7fae2015f5b0 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20162430 con 0x7fae200629b0 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2016fc50 con 0x7fae200615f0 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae200bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39070/0 (socket says 172.21.15.181:39070) 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae200bb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59460/0 (socket says 172.21.15.181:59460) 2020-11-16T22:29:37.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 -- 172.21.15.181:0/3624525512 learned_addr learned my addr 172.21.15.181:0/3624525512 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1f59e700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae2015f5b0 0x7fae200bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 -- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 msgr2=0x7fae200bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae200bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 -- 172.21.15.181:0/3624525512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae2015f5b0 msgr2=0x7fae200bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae2015f5b0 0x7fae200bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 -- 172.21.15.181:0/3624525512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2016fef0 con 0x7fae200615f0 2020-11-16T22:29:37.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae1ed9d700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fae1000c710 tx=0x7fae10007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10007020 con 0x7fae200615f0 2020-11-16T22:29:37.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae10007180 con 0x7fae200615f0 2020-11-16T22:29:37.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae201612c0 con 0x7fae200615f0 2020-11-16T22:29:37.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10006540 con 0x7fae200615f0 2020-11-16T22:29:37.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.453+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae20161800 con 0x7fae200615f0 2020-11-16T22:29:37.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20069eb0 con 0x7fae200615f0 2020-11-16T22:29:37.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fae10010070 con 0x7fae200615f0 2020-11-16T22:29:37.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae06ffd700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 0x7fadfc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae1e59c700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 0x7fadfc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fae1006d190 con 0x7fae200615f0 2020-11-16T22:29:37.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.457+0000 7fae1e59c700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 0x7fadfc057520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fae20069db0 tx=0x7fae14007d60).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.465+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fae100432e0 con 0x7fae200615f0 2020-11-16T22:29:37.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae200600f0 con 0x7fae200615f0 2020-11-16T22:29:37.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae06ffd700 1 -- 172.21.15.181:0/3624525512 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+10 (secure 0 0 0) 0x7fae10046900 con 0x7fae200615f0 2020-11-16T22:29:37.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 msgr2=0x7fadfc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 0x7fadfc057520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fae20069db0 tx=0x7fae14007d60).stop 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 msgr2=0x7fae20161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fae1000c710 tx=0x7fae10007bb0).stop 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 shutdown_connections 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadfc055060 0x7fadfc057520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae2015f5b0 0x7fae200bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200615f0 0x7fae20161f40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 --2- 172.21.15.181:0/3624525512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae200629b0 0x7fae200bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 >> 172.21.15.181:0/3624525512 conn(0x7fae200d69a0 msgr2=0x7fae200b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:37.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 shutdown_connections 2020-11-16T22:29:37.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.617+0000 7fae25079700 1 -- 172.21.15.181:0/3624525512 wait complete. 2020-11-16T22:29:37.650 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:37.651 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:29:37.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.797+0000 7f348c923700 1 Processor -- start 2020-11-16T22:29:37.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.797+0000 7f348c923700 1 -- start start 2020-11-16T22:29:37.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 0x7f3488061580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348815eb30 con 0x7f348815dfd0 2020-11-16T22:29:37.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348815f0b0 con 0x7f3488061100 2020-11-16T22:29:37.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348815dbe0 con 0x7f3488157a80 2020-11-16T22:29:37.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f3486ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 0x7f3488061580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f3486ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 0x7f3488061580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59468/0 (socket says 172.21.15.181:59468) 2020-11-16T22:29:37.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f3486ffd700 1 -- 172.21.15.181:0/4011943473 learned_addr learned my addr 172.21.15.181:0/4011943473 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:37.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39076/0 (socket says 172.21.15.181:39076) 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 -- 172.21.15.181:0/4011943473 >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 msgr2=0x7f3488061580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 --2- 172.21.15.181:0/4011943473 >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 0x7f3488061580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 -- 172.21.15.181:0/4011943473 >> v1:172.21.15.175:6789/0 conn(0x7f348815dfd0 legacy=0x7f348815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 -- 172.21.15.181:0/4011943473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34881611b0 con 0x7f3488157a80 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34877fe700 1 --2- 172.21.15.181:0/4011943473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f347800b9a0 tx=0x7f3478006df0).ready entity=mon.2 client_cookie=1b9ade203a72cc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:37.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34867fc700 1 -- 172.21.15.181:0/4011943473 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780097b0 con 0x7f3488157a80 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34867fc700 1 -- 172.21.15.181:0/4011943473 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478009910 con 0x7f3488157a80 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f34867fc700 1 -- 172.21.15.181:0/4011943473 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478009c90 con 0x7f3488157a80 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 -- 172.21.15.181:0/4011943473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 msgr2=0x7f34880cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.801+0000 7f348c923700 1 --2- 172.21.15.181:0/4011943473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f347800b9a0 tx=0x7f3478006df0).stop 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 -- 172.21.15.181:0/4011943473 shutdown_connections 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 --2- 172.21.15.181:0/4011943473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3488157a80 0x7f34880cc0c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 --2- 172.21.15.181:0/4011943473 >> v2:172.21.15.181:3300/0 conn(0x7f3488061100 0x7f3488061580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 -- 172.21.15.181:0/4011943473 >> 172.21.15.181:0/4011943473 conn(0x7f34880cf630 msgr2=0x7f34880cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:37.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 -- 172.21.15.181:0/4011943473 shutdown_connections 2020-11-16T22:29:37.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 -- 172.21.15.181:0/4011943473 wait complete. 2020-11-16T22:29:37.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 Processor -- start 2020-11-16T22:29:37.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.805+0000 7f348c923700 1 -- start start 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 0x7f34880cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 0x7f34880cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 0x7f34880b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880c9720 con 0x7f3488157a80 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880b6f00 con 0x7f3488061100 2020-11-16T22:29:37.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880b71a0 con 0x7f348815dfd0 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f34877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 0x7f34880cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 0x7f34880b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f34877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 0x7f34880cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59472/0 (socket says 172.21.15.181:59472) 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f34877fe700 1 -- 172.21.15.181:0/1024697002 learned_addr learned my addr 172.21.15.181:0/1024697002 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3486ffd700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 0x7f34880cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 -- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 msgr2=0x7f34880cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 0x7f34880cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 -- 172.21.15.181:0/1024697002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 msgr2=0x7f34880cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:37.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 0x7f34880cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:37.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 -- 172.21.15.181:0/1024697002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880b7440 con 0x7f348815dfd0 2020-11-16T22:29:37.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3486ffd700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 0x7f34880cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:37.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f3487fff700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 0x7f34880b7e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f348815ecc0 tx=0x7f3480007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000a730 con 0x7f348815dfd0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34880b76e0 con 0x7f348815dfd0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34880b8390 con 0x7f348815dfd0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.809+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348000a890 con 0x7f348815dfd0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000ac50 con 0x7f348815dfd0 2020-11-16T22:29:38.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3488044f60 con 0x7f348815dfd0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3480019030 con 0x7f348815dfd0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34677fe700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 0x7f3460057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f348006d590 con 0x7f348815dfd0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34877fe700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 0x7f3460057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.813+0000 7f34877fe700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 0x7f3460057520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f347800be60 tx=0x7f3478015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.817+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3480043810 con 0x7f348815dfd0 2020-11-16T22:29:38.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.977+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34880b7b80 con 0x7f348815dfd0 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.977+0000 7f34677fe700 1 -- 172.21.15.181:0/1024697002 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3480046e30 con 0x7f348815dfd0 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 msgr2=0x7f3460057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 0x7f3460057520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f347800be60 tx=0x7f3478015000).stop 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 msgr2=0x7f34880b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 0x7f34880b7e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f348815ecc0 tx=0x7f3480007e00).stop 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 shutdown_connections 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3460055060 0x7f3460057520 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f348815dfd0 0x7f34880b7e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3488061100 0x7f34880cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 --2- 172.21.15.181:0/1024697002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3488157a80 0x7f34880cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 >> 172.21.15.181:0/1024697002 conn(0x7f34880cf630 msgr2=0x7f34880cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 shutdown_connections 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:37.981+0000 7f348c923700 1 -- 172.21.15.181:0/1024697002 wait complete. 2020-11-16T22:29:38.004 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:29:38.005 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:29:38.005 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:38.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 Processor -- start 2020-11-16T22:29:38.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 -- start start 2020-11-16T22:29:38.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff8180ce210 0x7ff8180cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff818066480 con 0x7ff8180667b0 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff818066ee0 con 0x7ff8180ce210 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81f4c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818067180 con 0x7ff81815d200 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39092/0 (socket says 172.21.15.181:39092) 2020-11-16T22:29:38.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 -- 172.21.15.181:0/537427366 learned_addr learned my addr 172.21.15.181:0/537427366 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:38.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81da65700 1 --2- 172.21.15.181:0/537427366 >> v2:172.21.15.175:3300/0 conn(0x7ff8180ce210 0x7ff8180cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 -- 172.21.15.181:0/537427366 >> v2:172.21.15.175:3300/0 conn(0x7ff8180ce210 msgr2=0x7ff8180cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 --2- 172.21.15.181:0/537427366 >> v2:172.21.15.175:3300/0 conn(0x7ff8180ce210 0x7ff8180cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 -- 172.21.15.181:0/537427366 >> v1:172.21.15.175:6789/0 conn(0x7ff8180667b0 legacy=0x7ff8180cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 -- 172.21.15.181:0/537427366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818067420 con 0x7ff81815d200 2020-11-16T22:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff81d264700 1 --2- 172.21.15.181:0/537427366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff81000ba20 tx=0x7ff810006eb0).ready entity=mon.2 client_cookie=43441b23cb944e5b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff80ffff700 1 -- 172.21.15.181:0/537427366 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810009890 con 0x7ff81815d200 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff80ffff700 1 -- 172.21.15.181:0/537427366 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8100099f0 con 0x7ff81815d200 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.145+0000 7ff80ffff700 1 -- 172.21.15.181:0/537427366 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810009d70 con 0x7ff81815d200 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- 172.21.15.181:0/537427366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 msgr2=0x7ff81815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/537427366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff81000ba20 tx=0x7ff810006eb0).stop 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- 172.21.15.181:0/537427366 shutdown_connections 2020-11-16T22:29:38.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/537427366 >> v2:172.21.15.175:3300/0 conn(0x7ff8180ce210 0x7ff8180cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/537427366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff81815d5e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- 172.21.15.181:0/537427366 >> 172.21.15.181:0/537427366 conn(0x7ff8180d4790 msgr2=0x7ff8180d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:38.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- 172.21.15.181:0/537427366 shutdown_connections 2020-11-16T22:29:38.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- 172.21.15.181:0/537427366 wait complete. 2020-11-16T22:29:38.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 Processor -- start 2020-11-16T22:29:38.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.149+0000 7ff81f4c8700 1 -- start start 2020-11-16T22:29:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 0x7ff8180cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff818162570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180d02a0 con 0x7ff8180ce210 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818172fb0 con 0x7ff8180667b0 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818173250 con 0x7ff81815d200 2020-11-16T22:29:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59484/0 (socket says 172.21.15.181:59484) 2020-11-16T22:29:38.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 -- 172.21.15.181:0/2619906135 learned_addr learned my addr 172.21.15.181:0/2619906135 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:38.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81da65700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff818162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81ca63700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 0x7ff8180cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 -- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 msgr2=0x7ff818162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff818162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 -- 172.21.15.181:0/2619906135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 msgr2=0x7ff8180cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 0x7ff8180cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 -- 172.21.15.181:0/2619906135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8181734f0 con 0x7ff8180667b0 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81ca63700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 0x7ff8180cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81da65700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff818162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81d264700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7ff810007fc0 tx=0x7ff810015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810011c80 con 0x7ff8180667b0 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810011e00 con 0x7ff8180667b0 2020-11-16T22:29:38.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81001d6a0 con 0x7ff8180667b0 2020-11-16T22:29:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff818173790 con 0x7ff8180667b0 2020-11-16T22:29:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.153+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff818172c70 con 0x7ff8180667b0 2020-11-16T22:29:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff818069eb0 con 0x7ff8180667b0 2020-11-16T22:29:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff81000f070 con 0x7ff8180667b0 2020-11-16T22:29:38.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff80d7fa700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 0x7ff7f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:38.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff81006fbd0 con 0x7ff8180667b0 2020-11-16T22:29:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff81ca63700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 0x7ff7f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.157+0000 7ff81ca63700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 0x7ff7f0057520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ff8180a5820 tx=0x7ff814009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:38.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.161+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff810045e30 con 0x7ff8180667b0 2020-11-16T22:29:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.361+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8180d4340 con 0x7ff8180667b0 2020-11-16T22:29:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.361+0000 7ff80d7fa700 1 -- 172.21.15.181:0/2619906135 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4629 (secure 0 0 0) 0x7ff810049450 con 0x7ff8180667b0 2020-11-16T22:29:38.346 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:38.346 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":46,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:33.365462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 msgr2=0x7ff7f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 0x7ff7f0057520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7ff8180a5820 tx=0x7ff814009040).stop 2020-11-16T22:29:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 msgr2=0x7ff8180b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7ff810007fc0 tx=0x7ff810015000).stop 2020-11-16T22:29:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 shutdown_connections 2020-11-16T22:29:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff7f0055060 0x7ff7f0057520 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff81815d200 0x7ff818162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8180667b0 0x7ff8180b7720 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 --2- 172.21.15.181:0/2619906135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ce210 0x7ff8180cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:38.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 >> 172.21.15.181:0/2619906135 conn(0x7ff8180d4790 msgr2=0x7ff818062900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:38.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 shutdown_connections 2020-11-16T22:29:38.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:38.365+0000 7ff81f4c8700 1 -- 172.21.15.181:0/2619906135 wait complete. 2020-11-16T22:29:38.552 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 46 2020-11-16T22:29:39.457 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:29:30.354786+0000', 'modified': '2020-11-16T22:29:33.365462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5614': {'gid': 5614, 'name': 'i', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2753478724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2753478724}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2753478724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:29:39.457 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:29:39.458 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:29:39.594 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5614,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.2440444530000008,"mdsmap_epoch":46,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":16.572327835999999} 2020-11-16T22:29:39.595 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5614, "mdsmap_epoch": 46, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 9.244044453, "state": "up:active", "uptime": 16.572327836, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:29:39.595 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:29:39.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:39.596 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:29:39.613 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:29:39.614 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:29:39.685 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:29:39.685 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:29:39.707 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:29:39.708 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:29:39.713 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:29:39.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:29:39.714 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:29:39.742 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:29:39.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 Processor -- start 2020-11-16T22:29:39.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 -- start start 2020-11-16T22:29:39.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 0x7f91d80c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d815d9b0 con 0x7f91d80cf6b0 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80c8910 con 0x7f91d815d500 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80b3460 con 0x7f91d80b2900 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39102/0 (socket says 172.21.15.181:39102) 2020-11-16T22:29:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2055361442 learned_addr learned my addr 172.21.15.181:0/2055361442 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d77fe700 1 --2- 172.21.15.181:0/2055361442 >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 0x7f91d80c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2055361442 >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 msgr2=0x7f91d80c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:39.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 --2- 172.21.15.181:0/2055361442 >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 0x7f91d80c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:39.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2055361442 >> v1:172.21.15.175:6789/0 conn(0x7f91d80cf6b0 legacy=0x7f91d815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2055361442 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80b31d0 con 0x7f91d80b2900 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d77fe700 1 --2- 172.21.15.181:0/2055361442 >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 0x7f91d80c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d6ffd700 1 --2- 172.21.15.181:0/2055361442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f91c800ba20 tx=0x7f91c8006eb0).ready entity=mon.2 client_cookie=c452e4aeaafebfcf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d5ffb700 1 -- 172.21.15.181:0/2055361442 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8009890 con 0x7f91d80b2900 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d5ffb700 1 -- 172.21.15.181:0/2055361442 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c80099f0 con 0x7f91d80b2900 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.893+0000 7f91d5ffb700 1 -- 172.21.15.181:0/2055361442 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8009d70 con 0x7f91d80b2900 2020-11-16T22:29:39.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- 172.21.15.181:0/2055361442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 msgr2=0x7f91d80b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2055361442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f91c800ba20 tx=0x7f91c8006eb0).stop 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- 172.21.15.181:0/2055361442 shutdown_connections 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2055361442 >> v2:172.21.15.175:3300/0 conn(0x7f91d815d500 0x7f91d80c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2055361442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b2900 0x7f91d80b2ce0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- 172.21.15.181:0/2055361442 >> 172.21.15.181:0/2055361442 conn(0x7f91d80d2dc0 msgr2=0x7f91d80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- 172.21.15.181:0/2055361442 shutdown_connections 2020-11-16T22:29:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- 172.21.15.181:0/2055361442 wait complete. 2020-11-16T22:29:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 Processor -- start 2020-11-16T22:29:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.897+0000 7f91d7fff700 1 -- start start 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91d80cf6b0 0x7f91d80ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d815d500 0x7f91d8172fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81704a0 con 0x7f91d80cf6b0 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8170a80 con 0x7f91d80b2900 2020-11-16T22:29:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8170d20 con 0x7f91d815d500 2020-11-16T22:29:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d815d500 0x7f91d8172fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59498/0 (socket says 172.21.15.181:59498) 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2310748605 learned_addr learned my addr 172.21.15.181:0/2310748605 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d67fc700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91d80cf6b0 0x7f91d80ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d815d500 msgr2=0x7f91d8172fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d815d500 0x7f91d8172fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2310748605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91d80cf6b0 msgr2=0x7f91d80ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91d80cf6b0 0x7f91d80ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 -- 172.21.15.181:0/2310748605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8170fc0 con 0x7f91d80b2900 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d6ffd700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f91c8006ef0 tx=0x7f91c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8021070 con 0x7f91d80b2900 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8011c80 con 0x7f91d80b2900 2020-11-16T22:29:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8171260 con 0x7f91d80b2900 2020-11-16T22:29:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c801d6a0 con 0x7f91d80b2900 2020-11-16T22:29:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d81734a0 con 0x7f91d80b2900 2020-11-16T22:29:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.901+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8044f60 con 0x7f91d80b2900 2020-11-16T22:29:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.905+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f91c800f070 con 0x7f91d80b2900 2020-11-16T22:29:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.905+0000 7f91beffd700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 0x7f91b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.905+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f91c806f770 con 0x7f91d80b2900 2020-11-16T22:29:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.905+0000 7f91d67fc700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 0x7f91b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.905+0000 7f91d67fc700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 0x7f91b4057520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f91d815c040 tx=0x7f91cc007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:39.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:39.909+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f91c80459d0 con 0x7f91d80b2900 2020-11-16T22:29:40.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.117+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f91d8160db0 con 0x7f91d80b2900 2020-11-16T22:29:40.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.713+0000 7f91beffd700 1 -- 172.21.15.181:0/2310748605 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v47) v1 ==== 105+0+0 (secure 0 0 0) 0x7f91c8006a40 con 0x7f91d80b2900 2020-11-16T22:29:40.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 msgr2=0x7f91b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 0x7f91b4057520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f91d815c040 tx=0x7f91cc007b90).stop 2020-11-16T22:29:40.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 msgr2=0x7f91d80ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f91c8006ef0 tx=0x7f91c8015000).stop 2020-11-16T22:29:40.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 shutdown_connections 2020-11-16T22:29:40.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91b4055060 0x7f91b4057520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d815d500 0x7f91d8172fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d80b2900 0x7f91d80ce010 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 --2- 172.21.15.181:0/2310748605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91d80cf6b0 0x7f91d80ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 >> 172.21.15.181:0/2310748605 conn(0x7f91d80d2dc0 msgr2=0x7f91d815b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:40.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 shutdown_connections 2020-11-16T22:29:40.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.721+0000 7f91d7fff700 1 -- 172.21.15.181:0/2310748605 wait complete. 2020-11-16T22:29:40.725 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.857+0000 7fe24c994700 1 Processor -- start 2020-11-16T22:29:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.857+0000 7fe24c994700 1 -- start start 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe248156a90 0x7fe248061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480cccd0 con 0x7fe2480615f0 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480ccf70 con 0x7fe24815f3c0 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24815e030 con 0x7fe248156a90 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2477fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe248156a90 0x7fe248061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:40.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe247fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe24815f3c0 0x7fe2480cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45592/0 (socket says 172.21.15.181:45592) 2020-11-16T22:29:40.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe247fff700 1 -- 172.21.15.181:0/2794888426 learned_addr learned my addr 172.21.15.181:0/2794888426 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2477fe700 1 -- 172.21.15.181:0/2794888426 >> v1:172.21.15.181:6789/0 conn(0x7fe24815f3c0 legacy=0x7fe2480cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2477fe700 1 -- 172.21.15.181:0/2794888426 >> v1:172.21.15.175:6789/0 conn(0x7fe2480615f0 legacy=0x7fe24815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2467fc700 1 -- 172.21.15.181:0/2794888426 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242992176 0 0) 0x7fe2480ccf70 con 0x7fe24815f3c0 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2477fe700 1 -- 172.21.15.181:0/2794888426 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24815e2d0 con 0x7fe248156a90 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2477fe700 1 --2- 172.21.15.181:0/2794888426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe248061100 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fe23800b9a0 tx=0x7fe238006df0).ready entity=mon.1 client_cookie=d640d9041dcf336a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2467fc700 1 -- 172.21.15.181:0/2794888426 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380097b0 con 0x7fe248156a90 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2467fc700 1 -- 172.21.15.181:0/2794888426 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238009910 con 0x7fe248156a90 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe2467fc700 1 -- 172.21.15.181:0/2794888426 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238009c90 con 0x7fe248156a90 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- 172.21.15.181:0/2794888426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 msgr2=0x7fe248061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 --2- 172.21.15.181:0/2794888426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe248061100 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fe23800b9a0 tx=0x7fe238006df0).stop 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- 172.21.15.181:0/2794888426 shutdown_connections 2020-11-16T22:29:40.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 --2- 172.21.15.181:0/2794888426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe248061100 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- 172.21.15.181:0/2794888426 >> 172.21.15.181:0/2794888426 conn(0x7fe2480cf630 msgr2=0x7fe2480cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:40.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- 172.21.15.181:0/2794888426 shutdown_connections 2020-11-16T22:29:40.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.861+0000 7fe24c994700 1 -- 172.21.15.181:0/2794888426 wait complete. 2020-11-16T22:29:40.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 Processor -- start 2020-11-16T22:29:40.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- start start 2020-11-16T22:29:40.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2480615f0 0x7fe24815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:40.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 0x7fe2480d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24814e530 con 0x7fe2480615f0 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24816fd80 con 0x7fe248156a90 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe248170020 con 0x7fe24815f3c0 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe247fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 0x7fe2480d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59522/0 (socket says 172.21.15.181:59522) 2020-11-16T22:29:40.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 -- 172.21.15.181:0/1201807908 learned_addr learned my addr 172.21.15.181:0/1201807908 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:40.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe2477fe700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2480615f0 0x7fe24815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 -- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 msgr2=0x7fe2480d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 0x7fe2480d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 -- 172.21.15.181:0/1201807908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2480615f0 msgr2=0x7fe24815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2480615f0 0x7fe24815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 -- 172.21.15.181:0/1201807908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2481702c0 con 0x7fe248156a90 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe247fff700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 0x7fe2480d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe246ffd700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fe23000c6e0 tx=0x7fe230007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230007020 con 0x7fe248156a90 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe230007180 con 0x7fe248156a90 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe248170560 con 0x7fe248156a90 2020-11-16T22:29:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300064f0 con 0x7fe248156a90 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.865+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe248170aa0 con 0x7fe248156a90 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248044f60 con 0x7fe248156a90 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe230010070 con 0x7fe248156a90 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe23f7fe700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 0x7fe224057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe2477fe700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 0x7fe224057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:40.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe23006d010 con 0x7fe248156a90 2020-11-16T22:29:40.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.869+0000 7fe2477fe700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 0x7fe224057550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fe238006e30 tx=0x7fe238015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:40.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:40.873+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe230043270 con 0x7fe248156a90 2020-11-16T22:29:41.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.073+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe248151730 con 0x7fe248156a90 2020-11-16T22:29:41.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe23f7fe700 1 -- 172.21.15.181:0/1201807908 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fe230046890 con 0x7fe248156a90 2020-11-16T22:29:41.059 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:41.059 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":48,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:40.717900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5614,"mds_1":5605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:creating","state_seq":1,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:41.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 msgr2=0x7fe224057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:41.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 0x7fe224057550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fe238006e30 tx=0x7fe238015000).stop 2020-11-16T22:29:41.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 msgr2=0x7fe2480cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:41.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fe23000c6e0 tx=0x7fe230007c50).stop 2020-11-16T22:29:41.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 shutdown_connections 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe224055090 0x7fe224057550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe24815f3c0 0x7fe2480d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2480615f0 0x7fe24815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 --2- 172.21.15.181:0/1201807908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe248156a90 0x7fe2480cfbb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 >> 172.21.15.181:0/1201807908 conn(0x7fe2480cf630 msgr2=0x7fe24815ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 shutdown_connections 2020-11-16T22:29:41.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:41.077+0000 7fe24c994700 1 -- 172.21.15.181:0/1201807908 wait complete. 2020-11-16T22:29:41.063 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 48 2020-11-16T22:29:41.980 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:29:30.354786+0000', 'modified': '2020-11-16T22:29:40.717900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5614, 'mds_1': 5605}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5605': {'gid': 5605, 'name': 'b', 'rank': 1, 'incarnation': 48, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.181:6837/2557321074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2557321074}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2557321074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5614': {'gid': 5614, 'name': 'i', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2753478724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2753478724}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2753478724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:29:41.981 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5605:up:creating 2020-11-16T22:29:42.897 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:29:42.916+0000 7f6432437700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:29:42.897 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:29:42.916+0000 7f6432437700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:29:42.982 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:43.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.133+0000 7f4c811ff700 1 Processor -- start 2020-11-16T22:29:43.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.133+0000 7f4c811ff700 1 -- start start 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.133+0000 7f4c811ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c7c15fa10 0x7f4c7c15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c061b70 con 0x7f4c7c0ce750 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c15cda0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c15d040 con 0x7f4c7c0612a0 2020-11-16T22:29:43.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c7c0ce750 0x7f4c7c0cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45618/0 (socket says 172.21.15.181:45618) 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7a59c700 1 -- 172.21.15.181:0/1595819499 learned_addr learned my addr 172.21.15.181:0/1595819499 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 --2- 172.21.15.181:0/1595819499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7b59e700 1 --2- 172.21.15.181:0/1595819499 >> v2:172.21.15.175:3300/0 conn(0x7f4c7c15fa10 0x7f4c7c15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c79d9b700 1 -- 172.21.15.181:0/1595819499 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548481823 0 0) 0x7f4c7c061b70 con 0x7f4c7c0ce750 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c79d9b700 1 -- 172.21.15.181:0/1595819499 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c5c003240 con 0x7f4c7c0ce750 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 -- 172.21.15.181:0/1595819499 >> v2:172.21.15.175:3300/0 conn(0x7f4c7c15fa10 msgr2=0x7f4c7c15fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:43.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 --2- 172.21.15.181:0/1595819499 >> v2:172.21.15.175:3300/0 conn(0x7f4c7c15fa10 0x7f4c7c15fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 -- 172.21.15.181:0/1595819499 >> v1:172.21.15.181:6789/0 conn(0x7f4c7c0ce750 legacy=0x7f4c7c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 -- 172.21.15.181:0/1595819499 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c15d2e0 con 0x7f4c7c0612a0 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c7ad9d700 1 --2- 172.21.15.181:0/1595819499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c061680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f4c6c00ae60 tx=0x7f4c6c00a370).ready entity=mon.2 client_cookie=17d64dfb8dda974d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c79d9b700 1 -- 172.21.15.181:0/1595819499 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00cca0 con 0x7f4c7c0612a0 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c79d9b700 1 -- 172.21.15.181:0/1595819499 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00ce00 con 0x7f4c7c0612a0 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c79d9b700 1 -- 172.21.15.181:0/1595819499 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c010620 con 0x7f4c7c0612a0 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- 172.21.15.181:0/1595819499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 msgr2=0x7f4c7c061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 --2- 172.21.15.181:0/1595819499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c061680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f4c6c00ae60 tx=0x7f4c6c00a370).stop 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- 172.21.15.181:0/1595819499 shutdown_connections 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 --2- 172.21.15.181:0/1595819499 >> v2:172.21.15.175:3300/0 conn(0x7f4c7c15fa10 0x7f4c7c15fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 --2- 172.21.15.181:0/1595819499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c061680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- 172.21.15.181:0/1595819499 >> 172.21.15.181:0/1595819499 conn(0x7f4c7c0cf890 msgr2=0x7f4c7c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- 172.21.15.181:0/1595819499 shutdown_connections 2020-11-16T22:29:43.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.137+0000 7f4c811ff700 1 -- 172.21.15.181:0/1595819499 wait complete. 2020-11-16T22:29:43.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 Processor -- start 2020-11-16T22:29:43.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- start start 2020-11-16T22:29:43.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7c0ce750 0x7f4c7c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b6440 con 0x7f4c7c0ce750 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b66e0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b6980 con 0x7f4c7c0612a0 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59552/0 (socket says 172.21.15.181:59552) 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 -- 172.21.15.181:0/372329809 learned_addr learned my addr 172.21.15.181:0/372329809 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:43.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7a59c700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7c0ce750 0x7f4c7c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 -- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 msgr2=0x7f4c7c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 -- 172.21.15.181:0/372329809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7c0ce750 msgr2=0x7f4c7c0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7c0ce750 0x7f4c7c0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 -- 172.21.15.181:0/372329809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c0b6c20 con 0x7f4c7c15fa10 2020-11-16T22:29:43.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c7b59e700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f4c740067a0 tx=0x7f4c74006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:43.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400b340 con 0x7f4c7c15fa10 2020-11-16T22:29:43.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7400c820 con 0x7f4c7c15fa10 2020-11-16T22:29:43.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c0b7bb0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400c9b0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.141+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c154bd0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0b6270 con 0x7f4c7c15fa10 2020-11-16T22:29:43.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4c74021070 con 0x7f4c7c15fa10 2020-11-16T22:29:43.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c72ffd700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 0x7f4c58057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:43.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c7406f770 con 0x7f4c7c15fa10 2020-11-16T22:29:43.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c7ad9d700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 0x7f4c58057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:43.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.145+0000 7f4c7ad9d700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 0x7f4c58057550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f4c6c00ae30 tx=0x7f4c7c063560).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:43.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.153+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4c740459d0 con 0x7f4c7c15fa10 2020-11-16T22:29:43.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.353+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c7c0b7d40 con 0x7f4c7c15fa10 2020-11-16T22:29:43.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.353+0000 7f4c72ffd700 1 -- 172.21.15.181:0/372329809 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f4c7401f070 con 0x7f4c7c15fa10 2020-11-16T22:29:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:43.338 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":51,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:42.733173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:43.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 msgr2=0x7f4c58057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 0x7f4c58057550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f4c6c00ae30 tx=0x7f4c7c063560).stop 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 msgr2=0x7f4c7c0cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f4c740067a0 tx=0x7f4c74006df0).stop 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 shutdown_connections 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c58055090 0x7f4c58057550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c7c15fa10 0x7f4c7c0cf240 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c7c0612a0 0x7f4c7c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 --2- 172.21.15.181:0/372329809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7c0ce750 0x7f4c7c0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:43.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 >> 172.21.15.181:0/372329809 conn(0x7f4c7c0cf890 msgr2=0x7f4c7c15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:43.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 shutdown_connections 2020-11-16T22:29:43.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:43.357+0000 7f4c811ff700 1 -- 172.21.15.181:0/372329809 wait complete. 2020-11-16T22:29:43.341 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 51 2020-11-16T22:29:44.259 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:29:30.354786+0000', 'modified': '2020-11-16T22:29:42.733173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5614, 'mds_1': 5605, 'mds_2': 5599}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5599': {'gid': 5599, 'name': 'a', 'rank': 2, 'incarnation': 50, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6835/3960101400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3960101400}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3960101400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5605': {'gid': 5605, 'name': 'b', 'rank': 1, 'incarnation': 48, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6837/2557321074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2557321074}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2557321074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5614': {'gid': 5614, 'name': 'i', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2753478724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2753478724}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2753478724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:29:44.260 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:29:44.261 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:29:44.400 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":5599,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.6749807209999998,"mdsmap_epoch":51,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":21.387190142000001} 2020-11-16T22:29:44.401 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5599, "mdsmap_epoch": 51, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 2.674980721, "state": "up:active", "uptime": 21.387190142, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:29:44.401 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:29:44.526 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":5605,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.8147510969999998,"mdsmap_epoch":51,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":21.510587316999999} 2020-11-16T22:29:44.526 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5605, "mdsmap_epoch": 51, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 3.814751097, "state": "up:active", "uptime": 21.510587317, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:29:44.527 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:29:44.665 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":5614,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.315205172000001,"mdsmap_epoch":51,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":21.643488484999999} 2020-11-16T22:29:44.666 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 5614, "mdsmap_epoch": 51, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 14.315205172, "state": "up:active", "uptime": 21.643488485, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:29:44.666 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:29:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 Processor -- start 2020-11-16T22:29:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 -- start start 2020-11-16T22:29:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38cc156a90 0x7f38cc061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 0x7f38cc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cccd0 con 0x7f38cc15f1a0 2020-11-16T22:29:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0ccf70 con 0x7f38cc0615f0 2020-11-16T22:29:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.813+0000 7f38cbfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc15e030 con 0x7f38cc156a90 2020-11-16T22:29:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38cc156a90 0x7f38cc061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38cc156a90 0x7f38cc061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59558/0 (socket says 172.21.15.181:59558) 2020-11-16T22:29:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 -- 172.21.15.181:0/2469847118 learned_addr learned my addr 172.21.15.181:0/2469847118 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38ca7fc700 1 --2- 172.21.15.181:0/2469847118 >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 0x7f38cc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 -- 172.21.15.181:0/2469847118 >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 msgr2=0x7f38cc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 --2- 172.21.15.181:0/2469847118 >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 0x7f38cc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 -- 172.21.15.181:0/2469847118 >> v1:172.21.15.175:6789/0 conn(0x7f38cc15f1a0 legacy=0x7f38cc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 -- 172.21.15.181:0/2469847118 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc15e2d0 con 0x7f38cc156a90 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38ca7fc700 1 --2- 172.21.15.181:0/2469847118 >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 0x7f38cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38caffd700 1 --2- 172.21.15.181:0/2469847118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc156a90 0x7f38cc061100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f38cc0675c0 tx=0x7f38bc006eb0).ready entity=mon.1 client_cookie=4c40faefcb326c8b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38c9ffb700 1 -- 172.21.15.181:0/2469847118 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0098d0 con 0x7f38cc156a90 2020-11-16T22:29:44.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38c9ffb700 1 -- 172.21.15.181:0/2469847118 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc009a30 con 0x7f38cc156a90 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38c9ffb700 1 -- 172.21.15.181:0/2469847118 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc009db0 con 0x7f38cc156a90 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 -- 172.21.15.181:0/2469847118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc156a90 msgr2=0x7f38cc061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2469847118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc156a90 0x7f38cc061100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f38cc0675c0 tx=0x7f38bc006eb0).stop 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 -- 172.21.15.181:0/2469847118 shutdown_connections 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2469847118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc156a90 0x7f38cc061100 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2469847118 >> v2:172.21.15.175:3300/0 conn(0x7f38cc0615f0 0x7f38cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 -- 172.21.15.181:0/2469847118 >> 172.21.15.181:0/2469847118 conn(0x7f38cc0cf630 msgr2=0x7f38cc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 -- 172.21.15.181:0/2469847118 shutdown_connections 2020-11-16T22:29:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.817+0000 7f38cbfff700 1 -- 172.21.15.181:0/2469847118 wait complete. 2020-11-16T22:29:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.821+0000 7f38cbfff700 1 Processor -- start 2020-11-16T22:29:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.821+0000 7f38cbfff700 1 -- start start 2020-11-16T22:29:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38cc156a90 0x7f38cc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 0x7f38cc0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0b7d30 con 0x7f38cc15f1a0 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc16f920 con 0x7f38cc0615f0 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc16fbc0 con 0x7f38cc156a90 2020-11-16T22:29:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59564/0 (socket says 172.21.15.181:59564) 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 -- 172.21.15.181:0/2270331980 learned_addr learned my addr 172.21.15.181:0/2270331980 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38ca7fc700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38cc156a90 0x7f38cc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cb7fe700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 0x7f38cc0d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 -- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38cc156a90 msgr2=0x7f38cc15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38cc156a90 0x7f38cc15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 -- 172.21.15.181:0/2270331980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 msgr2=0x7f38cc0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:44.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 0x7f38cc0d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:44.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 -- 172.21.15.181:0/2270331980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc16fe60 con 0x7f38cc0615f0 2020-11-16T22:29:44.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cb7fe700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 0x7f38cc0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:29:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38caffd700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f38cc15e170 tx=0x7f38bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc022070 con 0x7f38cc0615f0 2020-11-16T22:29:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc01dce0 con 0x7f38cc0615f0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc01d290 con 0x7f38cc0615f0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc170100 con 0x7f38cc0615f0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc0d10e0 con 0x7f38cc0615f0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f38bc00f070 con 0x7f38cc0615f0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 0x7f38a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38ca7fc700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 0x7f38a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:29:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38bc072c90 con 0x7f38cc0615f0 2020-11-16T22:29:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.825+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc044f60 con 0x7f38cc0615f0 2020-11-16T22:29:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.829+0000 7f38ca7fc700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 0x7f38a8057520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f38cc156540 tx=0x7f38b4008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:29:44.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:44.833+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f38bc048e60 con 0x7f38cc0615f0 2020-11-16T22:29:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38cc0b8bf0 con 0x7f38cc0615f0 2020-11-16T22:29:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38c2ffd700 1 -- 172.21.15.181:0/2270331980 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f38bc04c480 con 0x7f38cc0615f0 2020-11-16T22:29:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:29:45.020 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":51,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:29:42.733173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:29:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 msgr2=0x7f38a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 0x7f38a8057520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f38cc156540 tx=0x7f38b4008040).stop 2020-11-16T22:29:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 msgr2=0x7f38cc15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:29:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f38cc15e170 tx=0x7f38bc015000).stop 2020-11-16T22:29:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 shutdown_connections 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38a8055060 0x7f38a8057520 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38cc15f1a0 0x7f38cc0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38cc0615f0 0x7f38cc15ccd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 --2- 172.21.15.181:0/2270331980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38cc156a90 0x7f38cc15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.037+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 >> 172.21.15.181:0/2270331980 conn(0x7f38cc0cf630 msgr2=0x7f38cc065f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.041+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 shutdown_connections 2020-11-16T22:29:45.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:29:45.041+0000 7f38cbfff700 1 -- 172.21.15.181:0/2270331980 wait complete. 2020-11-16T22:29:45.024 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 51 2020-11-16T22:29:45.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:45.035 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:29:45.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:45.103 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:29:48.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:48.032 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:29:48.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:48.072 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:29:48.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:29:48.154 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:29:48.186 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:48.320 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:29:48.321 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:29:48.321 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:50.460 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:29:50.461 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:29:50.461 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:52.463 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:52.599 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:29:52.600 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:29:52.600 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:54.601 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:54.738 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:29:54.739 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:29:54.739 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:56.740 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:56.875 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:29:56.875 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:29:56.876 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:58.877 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:29:59.014 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:29:59.015 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:29:59.016 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:29:59.016 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:29:59.162 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:29:59.162 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:29:59.163 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:29:59.243 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:59.249 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:29:59.252 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:01.164 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:30:01.301 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:30:01.302 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:30:01.302 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:30:03.304 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:30:03.444 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:30:03.445 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:30:03.446 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:30:05.447 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:30:05.583 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:30:05.584 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:30:05.584 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:30:05.585 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:30:05.714 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:30:05.715 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:30:05.715 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:30:05.715 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:05.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 Processor -- start 2020-11-16T22:30:05.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- start start 2020-11-16T22:30:05.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04415c6c0 0x7fb04415fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:05.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04415cb50 con 0x7fb0440612a0 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04415cda0 con 0x7fb044061680 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415d040 con 0x7fb04415c6c0 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04415c6c0 0x7fb04415fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0489b4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb044061680 0x7fb0440cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45648/0 (socket says 172.21.15.181:45648) 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04415c6c0 0x7fb04415fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59580/0 (socket says 172.21.15.181:59580) 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0489b4700 1 -- 172.21.15.181:0/2342559092 learned_addr learned my addr 172.21.15.181:0/2342559092 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:05.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 -- 172.21.15.181:0/2342559092 >> v1:172.21.15.181:6789/0 conn(0x7fb044061680 legacy=0x7fb0440cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 -- 172.21.15.181:0/2342559092 >> v1:172.21.15.175:6789/0 conn(0x7fb0440612a0 legacy=0x7fb0440ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 -- 172.21.15.181:0/2342559092 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04415d2e0 con 0x7fb04415c6c0 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb033fff700 1 -- 172.21.15.181:0/2342559092 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181369855 0 0) 0x7fb04415cda0 con 0x7fb044061680 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb0499b6700 1 --2- 172.21.15.181:0/2342559092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04415c6c0 0x7fb04415fa10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fb03c009c90 tx=0x7fb03c0091d0).ready entity=mon.1 client_cookie=a2a71c7187df0157 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb033fff700 1 -- 172.21.15.181:0/2342559092 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00bbc0 con 0x7fb04415c6c0 2020-11-16T22:30:05.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb033fff700 1 -- 172.21.15.181:0/2342559092 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00bd20 con 0x7fb04415c6c0 2020-11-16T22:30:05.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb033fff700 1 -- 172.21.15.181:0/2342559092 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c011610 con 0x7fb04415c6c0 2020-11-16T22:30:05.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- 172.21.15.181:0/2342559092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04415c6c0 msgr2=0x7fb04415fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 --2- 172.21.15.181:0/2342559092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04415c6c0 0x7fb04415fa10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fb03c009c90 tx=0x7fb03c0091d0).stop 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- 172.21.15.181:0/2342559092 shutdown_connections 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 --2- 172.21.15.181:0/2342559092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04415c6c0 0x7fb04415fa10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- 172.21.15.181:0/2342559092 >> 172.21.15.181:0/2342559092 conn(0x7fb0440cf890 msgr2=0x7fb0440cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- 172.21.15.181:0/2342559092 shutdown_connections 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.853+0000 7fb04b419700 1 -- 172.21.15.181:0/2342559092 wait complete. 2020-11-16T22:30:05.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 Processor -- start 2020-11-16T22:30:05.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 -- start start 2020-11-16T22:30:05.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0440612a0 0x7fb04415e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:05.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:05.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb04415c6c0 0x7fb04414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:05.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04414e790 con 0x7fb0440612a0 2020-11-16T22:30:05.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04415eac0 con 0x7fb044061680 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb04b419700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04415ed60 con 0x7fb04415c6c0 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb0489b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb0499b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb04415c6c0 0x7fb04414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb0489b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59584/0 (socket says 172.21.15.181:59584) 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb0489b4700 1 -- 172.21.15.181:0/343395391 learned_addr learned my addr 172.21.15.181:0/343395391 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.857+0000 7fb0491b5700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0440612a0 0x7fb04415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:05.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 -- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb04415c6c0 msgr2=0x7fb04414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:05.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb04415c6c0 0x7fb04414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:05.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 -- 172.21.15.181:0/343395391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0440612a0 msgr2=0x7fb04415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:05.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0440612a0 0x7fb04415e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:05.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 -- 172.21.15.181:0/343395391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04415f000 con 0x7fb044061680 2020-11-16T22:30:05.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0489b4700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fb03400c730 tx=0x7fb0340079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:05.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340072e0 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034007440 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04415f2a0 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034005820 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04415f7e0 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044069eb0 con 0x7fb044061680 2020-11-16T22:30:05.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb034010070 con 0x7fb044061680 2020-11-16T22:30:05.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 0x7fb01c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:05.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb03406d430 con 0x7fb044061680 2020-11-16T22:30:05.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0491b5700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 0x7fb01c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:05.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.861+0000 7fb0491b5700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 0x7fb01c057520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fb040003300 tx=0x7fb040007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:05.869+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb034043690 con 0x7fb044061680 2020-11-16T22:30:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.073+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb044066280 con 0x7fb044061680 2020-11-16T22:30:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.073+0000 7fb0317fa700 1 -- 172.21.15.181:0/343395391 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fb034046cb0 con 0x7fb044061680 2020-11-16T22:30:06.055 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:06.055 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":53,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:03.819455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.073+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 msgr2=0x7fb01c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.073+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 0x7fb01c057520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fb040003300 tx=0x7fb040007b90).stop 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 msgr2=0x7fb04414e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fb03400c730 tx=0x7fb0340079e0).stop 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 shutdown_connections 2020-11-16T22:30:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb01c055060 0x7fb01c057520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb04415c6c0 0x7fb04414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0440612a0 0x7fb04415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 --2- 172.21.15.181:0/343395391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb044061680 0x7fb04414e2a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 >> 172.21.15.181:0/343395391 conn(0x7fb0440cf890 msgr2=0x7fb04415d980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 shutdown_connections 2020-11-16T22:30:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.077+0000 7fb04b419700 1 -- 172.21.15.181:0/343395391 wait complete. 2020-11-16T22:30:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 53 2020-11-16T22:30:06.076 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i flush journal 2020-11-16T22:30:06.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 Processor -- start 2020-11-16T22:30:06.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 -- start start 2020-11-16T22:30:06.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77980816e0 0x7f77980d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77980d1c70 con 0x7f7798145c60 2020-11-16T22:30:06.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980d1f10 con 0x7f77980816e0 2020-11-16T22:30:06.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779e110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77980d21b0 con 0x7f7798145880 2020-11-16T22:30:06.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39200/0 (socket says 172.21.15.181:39200) 2020-11-16T22:30:06.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 -- 172.21.15.181:0/2830953283 learned_addr learned my addr 172.21.15.181:0/2830953283 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d90f700 1 --2- 172.21.15.181:0/2830953283 >> v2:172.21.15.175:3300/0 conn(0x7f77980816e0 0x7f77980d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 -- 172.21.15.181:0/2830953283 >> v2:172.21.15.175:3300/0 conn(0x7f77980816e0 msgr2=0x7f77980d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 --2- 172.21.15.181:0/2830953283 >> v2:172.21.15.175:3300/0 conn(0x7f77980816e0 0x7f77980d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 -- 172.21.15.181:0/2830953283 >> v1:172.21.15.175:6789/0 conn(0x7f7798145c60 legacy=0x7f7798080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 -- 172.21.15.181:0/2830953283 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980d2450 con 0x7f7798145880 2020-11-16T22:30:06.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f779d10e700 1 --2- 172.21.15.181:0/2830953283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f7788009d50 tx=0x7f7788009290).ready entity=mon.2 client_cookie=629c40747627ec92 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f778ffff700 1 -- 172.21.15.181:0/2830953283 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778800bbc0 con 0x7f7798145880 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f778ffff700 1 -- 172.21.15.181:0/2830953283 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778800bd20 con 0x7f7798145880 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.221+0000 7f778ffff700 1 -- 172.21.15.181:0/2830953283 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788011610 con 0x7f7798145880 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- 172.21.15.181:0/2830953283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 msgr2=0x7f77980d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 --2- 172.21.15.181:0/2830953283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f7788009d50 tx=0x7f7788009290).stop 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- 172.21.15.181:0/2830953283 shutdown_connections 2020-11-16T22:30:06.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 --2- 172.21.15.181:0/2830953283 >> v2:172.21.15.175:3300/0 conn(0x7f77980816e0 0x7f77980d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 --2- 172.21.15.181:0/2830953283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f77980d1110 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- 172.21.15.181:0/2830953283 >> 172.21.15.181:0/2830953283 conn(0x7f77980d6770 msgr2=0x7f77980d6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- 172.21.15.181:0/2830953283 shutdown_connections 2020-11-16T22:30:06.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- 172.21.15.181:0/2830953283 wait complete. 2020-11-16T22:30:06.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 Processor -- start 2020-11-16T22:30:06.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.225+0000 7f779e110700 1 -- start start 2020-11-16T22:30:06.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7798145c60 0x7f7798160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798170940 con 0x7f7798145c60 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779815f850 con 0x7f77980816e0 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7798146140 con 0x7f7798145880 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59596/0 (socket says 172.21.15.181:59596) 2020-11-16T22:30:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39208/0 (socket says 172.21.15.181:39208) 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779d10e700 1 -- 172.21.15.181:0/2022559320 learned_addr learned my addr 172.21.15.181:0/2022559320 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779d90f700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7798145c60 0x7f7798160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 -- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 msgr2=0x7f779816f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 -- 172.21.15.181:0/2022559320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7798145c60 msgr2=0x7f7798160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7798145c60 0x7f7798160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 -- 172.21.15.181:0/2022559320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779816fa80 con 0x7f7798145880 2020-11-16T22:30:06.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779d10e700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779c90d700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f779400c740 tx=0x7f7794007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f778d7fa700 1 -- 172.21.15.181:0/2022559320 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794007020 con 0x7f7798145880 2020-11-16T22:30:06.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f778d7fa700 1 -- 172.21.15.181:0/2022559320 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794007180 con 0x7f7798145880 2020-11-16T22:30:06.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779816ff90 con 0x7f7798145880 2020-11-16T22:30:06.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f778d7fa700 1 -- 172.21.15.181:0/2022559320 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940064f0 con 0x7f7798145880 2020-11-16T22:30:06.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7798148ce0 con 0x7f7798145880 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.229+0000 7f778d7fa700 1 -- 172.21.15.181:0/2022559320 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f779401a060 con 0x7f7798145880 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f778d7fa700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 0x7f7774057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f779d10e700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 0x7f7774057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f778d7fa700 1 -- 172.21.15.181:0/2022559320 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f779406d170 con 0x7f7798145880 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f779d10e700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 0x7f7774057520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f7788009e80 tx=0x7f7788016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f77ade03740 1 Processor -- start 2020-11-16T22:30:06.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f77ade03740 1 -- start start 2020-11-16T22:30:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.233+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1063670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 0x105c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xeee4b0 con 0x1062fd0 2020-11-16T22:30:06.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39214/0 (socket says 172.21.15.181:39214) 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 -- 172.21.15.181:0/1429835892 learned_addr learned my addr 172.21.15.181:0/1429835892 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779c90d700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xfa2930 con 0xdd6bb0 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d90f700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 0x105c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xe626f0 con 0xe447b0 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 -- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 msgr2=0x1063670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1063670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 -- 172.21.15.181:0/1429835892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 msgr2=0x105c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 0x105c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 -- 172.21.15.181:0/1429835892 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe5d5a0 con 0xe447b0 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779c90d700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1063670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d90f700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 0x105c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f779d10e700 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f7788009a50 tx=0x7f778801d000).ready entity=mon.2 client_cookie=4a1827bd4752863b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f777e7fc700 1 -- 172.21.15.181:0/1429835892 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788009a80 con 0xe447b0 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f777e7fc700 1 -- 172.21.15.181:0/1429835892 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778801cab0 con 0xe447b0 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f777e7fc700 1 -- 172.21.15.181:0/1429835892 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778801ce30 con 0xe447b0 2020-11-16T22:30:06.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 msgr2=0x1065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f7788009a50 tx=0x7f778801d000).stop 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 shutdown_connections 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1062fd0 0x105c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1065720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 --2- 172.21.15.181:0/1429835892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1063670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 >> 172.21.15.181:0/1429835892 conn(0x1012e40 msgr2=0x105f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 shutdown_connections 2020-11-16T22:30:06.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- 172.21.15.181:0/1429835892 wait complete. 2020-11-16T22:30:06.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 -- start start 2020-11-16T22:30:06.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.237+0000 7f77ade03740 1 Processor -- start 2020-11-16T22:30:06.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x10af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe447b0 0x10b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf8f6e0 con 0xe447b0 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xf8f820 con 0x1062fd0 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf90a90 con 0xdd6bb0 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 10 client.0 ll_register_callbacks cb 0xf85ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:06.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x10af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59614/0 (socket says 172.21.15.181:59614) 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 -- 172.21.15.181:0/2849819905 learned_addr learned my addr 172.21.15.181:0/2849819905 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779c90d700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe447b0 0x10b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 -- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 msgr2=0x10af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x10af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 -- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe447b0 msgr2=0x10b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe447b0 0x10b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 -- 172.21.15.181:0/2849819905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xf80280 con 0x1062fd0 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d10e700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x10af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d90f700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7790014ce0 tx=0x7f7790006df0).ready entity=mon.1 client_cookie=572e17a4496d6313 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77900129a0 con 0x1062fd0 2020-11-16T22:30:06.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7790012b00 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10436c0 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7790012cc0 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 10 client.5824 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0xdb5fd0 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10436c0 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f7790015420 con 0x1062fd0 2020-11-16T22:30:06.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 10 client.5824 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 10 client.5824 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 20 client.5824 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x105f900 0x10b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 4 client.5824 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xe80c50 con 0x105f900 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.241+0000 7f779d10e700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x105f900 0x10b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77677fe700 10 client.5824 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:06.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f779d10e700 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x105f900 0x10b2f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=976af3abff9dda94 server_cookie=189b0307ef5c5334 in_seq=0 out_seq=0 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77677fe700 1 -- 172.21.15.181:0/2849819905 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xe80c50 con 0x105f900 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77677fe700 10 client.5824 handle_command_reply: tid=0 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 client.5824 shutdown 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 msgr2=0x1085650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7790014ce0 tx=0x7f7790006df0).stop 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 shutdown_connections 2020-11-16T22:30:06.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x105f900 0x10b2f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1062fd0 0x1085650 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x10af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 --2- 172.21.15.181:0/2849819905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe447b0 0x10b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77677fe700 0 client.5824 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:06.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 >> 172.21.15.181:0/2849819905 conn(0x1012e40 msgr2=0x1061da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 shutdown_connections 2020-11-16T22:30:06.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 1 -- 172.21.15.181:0/2849819905 wait complete. 2020-11-16T22:30:06.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.245+0000 7f77ade03740 20 client.5824 trim_cache size 0 max 16384 2020-11-16T22:30:06.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 Processor -- start 2020-11-16T22:30:06.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 -- start start 2020-11-16T22:30:06.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x1057f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1095b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf67e40 con 0x1012e40 2020-11-16T22:30:06.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c1420 con 0xdd6bb0 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x10b4530 con 0xe447b0 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f779c90d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f779d90f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1095b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f779c90d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59618/0 (socket says 172.21.15.181:59618) 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f779c90d700 1 -- 172.21.15.181:0/4117911518 learned_addr learned my addr 172.21.15.181:0/4117911518 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.265+0000 7f779d10e700 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x1057f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 -- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 msgr2=0x1095b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1095b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 -- 172.21.15.181:0/4117911518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 msgr2=0x1057f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x1057f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 -- 172.21.15.181:0/4117911518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0081d0 con 0xdd6bb0 2020-11-16T22:30:06.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d10e700 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x1057f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f7794007960 tx=0x7f779404e040).ready entity=mon.1 client_cookie=21abb95a1b7d47da server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f777effd700 1 -- 172.21.15.181:0/4117911518 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794051040 con 0xdd6bb0 2020-11-16T22:30:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f777effd700 1 -- 172.21.15.181:0/4117911518 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794046520 con 0xdd6bb0 2020-11-16T22:30:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f777effd700 1 -- 172.21.15.181:0/4117911518 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794042e20 con 0xdd6bb0 2020-11-16T22:30:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 msgr2=0x1095610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f7794007960 tx=0x7f779404e040).stop 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- 172.21.15.181:0/4117911518 shutdown_connections 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe447b0 0x1095b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x1057f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- 172.21.15.181:0/4117911518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdd6bb0 0x1095610 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- 172.21.15.181:0/4117911518 >> 172.21.15.181:0/4117911518 conn(0x7f776c002320 msgr2=0x105c700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- 172.21.15.181:0/4117911518 shutdown_connections 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- 172.21.15.181:0/4117911518 wait complete. 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- start start 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 Processor -- start 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 0x110f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x10c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x111d5b0 con 0x1012e40 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x111d850 con 0xe447b0 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x111daf0 con 0xdd6bb0 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f77ade03740 10 client.0 ll_register_callbacks cb 0x1063740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779c90d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 0x110f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39234/0 (socket says 172.21.15.181:39234) 2020-11-16T22:30:06.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d10e700 1 -- 172.21.15.181:0/3759285790 learned_addr learned my addr 172.21.15.181:0/3759285790 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d90f700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x10c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.269+0000 7f779d10e700 1 -- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 msgr2=0x110f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d10e700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 0x110f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d10e700 1 -- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 msgr2=0x10c1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d10e700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x10c1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d10e700 1 -- 172.21.15.181:0/3759285790 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdf4720 con 0xdd6bb0 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779c90d700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 0x110f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d90f700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x10c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779d10e700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f778800b0b0 tx=0x7f778800b040).ready entity=mon.2 client_cookie=6f2d08192e2adc82 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788027c20 con 0xdd6bb0 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7788027d80 con 0xdd6bb0 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f778801cab0 con 0xdd6bb0 2020-11-16T22:30:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77880357d0 con 0xdd6bb0 2020-11-16T22:30:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 10 client.5468 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0xdffe00 con 0xdd6bb0 2020-11-16T22:30:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778801cab0 con 0xdd6bb0 2020-11-16T22:30:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f778803d020 con 0xdd6bb0 2020-11-16T22:30:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 10 client.5468 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 10 client.5468 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 20 client.5468 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1062fd0 0x10c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 4 client.5468 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0xe87f50 con 0x1062fd0 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779c90d700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1062fd0 0x10c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f779c90d700 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1062fd0 0x10c1a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5d81ab3ae21c1691 server_cookie=af1a594e19537b59 in_seq=0 out_seq=0 2020-11-16T22:30:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.273+0000 7f77677fe700 10 client.5468 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:06.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.297+0000 7f77677fe700 1 -- 172.21.15.181:0/3759285790 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0xe87f50 con 0x1062fd0 2020-11-16T22:30:06.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.297+0000 7f77677fe700 10 client.5468 handle_command_reply: tid=0 2020-11-16T22:30:06.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.297+0000 7f77ade03740 1 client.5468 shutdown 2020-11-16T22:30:06.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 msgr2=0x1121240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f778800b0b0 tx=0x7f778800b040).stop 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 shutdown_connections 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1062fd0 0x10c1a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1012e40 0x10c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xdd6bb0 0x1121240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 --2- 172.21.15.181:0/3759285790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe447b0 0x110f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77677fe700 0 client.5468 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 >> 172.21.15.181:0/3759285790 conn(0x7f776c002320 msgr2=0x10ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 shutdown_connections 2020-11-16T22:30:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 1 -- 172.21.15.181:0/3759285790 wait complete. 2020-11-16T22:30:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f77ade03740 20 client.5468 trim_cache size 0 max 16384 2020-11-16T22:30:06.285 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:06.285 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:30:06.285 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:30:06.285 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 msgr2=0x7f7774057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 0x7f7774057520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f7788009e80 tx=0x7f7788016000).stop 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 msgr2=0x7f779815fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f779400c740 tx=0x7f7794007b90).stop 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 shutdown_connections 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7774055060 0x7f7774057520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7798145c60 0x7f7798160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980816e0 0x7f779816f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 --2- 172.21.15.181:0/2022559320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7798145880 0x7f779815fb90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 >> 172.21.15.181:0/2022559320 conn(0x7f77980d6770 msgr2=0x7f7798163b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 shutdown_connections 2020-11-16T22:30:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.301+0000 7f779e110700 1 -- 172.21.15.181:0/2022559320 wait complete. 2020-11-16T22:30:06.297 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:06.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 Processor -- start 2020-11-16T22:30:06.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 -- start start 2020-11-16T22:30:06.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595013e870 0x7f595013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 0x7f5950160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595015d180 con 0x7f5950061100 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5950160730 con 0x7f595013e870 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f5954c47700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595013f140 con 0x7f595015fdc0 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595013e870 0x7f595013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 0x7f5950160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595013e870 0x7f595013ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59636/0 (socket says 172.21.15.181:59636) 2020-11-16T22:30:06.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 -- 172.21.15.181:0/674815597 learned_addr learned my addr 172.21.15.181:0/674815597 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 -- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 msgr2=0x7f5950160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 0x7f5950160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 -- 172.21.15.181:0/674815597 >> v1:172.21.15.175:6789/0 conn(0x7f5950061100 legacy=0x7f5950061590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594e7fc700 1 -- 172.21.15.181:0/674815597 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749633437 0 0) 0x7f595015d180 con 0x7f5950061100 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594f7fe700 1 -- 172.21.15.181:0/674815597 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595015d810 con 0x7f595013e870 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.441+0000 7f594effd700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 0x7f5950160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f594f7fe700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595013e870 0x7f595013ec50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5940009d50 tx=0x7f5940009290).ready entity=mon.1 client_cookie=b09a1730e66b8981 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f594e7fc700 1 -- 172.21.15.181:0/674815597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000bc50 con 0x7f595013e870 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f594e7fc700 1 -- 172.21.15.181:0/674815597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000bdb0 con 0x7f595013e870 2020-11-16T22:30:06.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f594e7fc700 1 -- 172.21.15.181:0/674815597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011610 con 0x7f595013e870 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 -- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595013e870 msgr2=0x7f595013ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595013e870 0x7f595013ec50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5940009d50 tx=0x7f5940009290).stop 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 -- 172.21.15.181:0/674815597 shutdown_connections 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595013e870 0x7f595013ec50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 --2- 172.21.15.181:0/674815597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595015fdc0 0x7f5950160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 -- 172.21.15.181:0/674815597 >> 172.21.15.181:0/674815597 conn(0x7f59500cf630 msgr2=0x7f59500cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 -- 172.21.15.181:0/674815597 shutdown_connections 2020-11-16T22:30:06.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.445+0000 7f5954c47700 1 -- 172.21.15.181:0/674815597 wait complete. 2020-11-16T22:30:06.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 Processor -- start 2020-11-16T22:30:06.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- start start 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5950061100 0x7f59500c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 0x7f5950157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595016cef0 con 0x7f5950061100 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950170270 con 0x7f595015fdc0 2020-11-16T22:30:06.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595016c270 con 0x7f595013e870 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 0x7f5950157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39250/0 (socket says 172.21.15.181:39250) 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 0x7f5950157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59642/0 (socket says 172.21.15.181:59642) 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 -- 172.21.15.181:0/2275874756 learned_addr learned my addr 172.21.15.181:0/2275874756 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594f7fe700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5950061100 0x7f59500c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 -- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 msgr2=0x7f5950157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 0x7f5950157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 -- 172.21.15.181:0/2275874756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5950061100 msgr2=0x7f59500c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5950061100 0x7f59500c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 -- 172.21.15.181:0/2275874756 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595016c510 con 0x7f595013e870 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f594effd700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f593800ef50 tx=0x7f5938006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800ca80 con 0x7f595013e870 2020-11-16T22:30:06.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800cbe0 con 0x7f595013e870 2020-11-16T22:30:06.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595016c7b0 con 0x7f595013e870 2020-11-16T22:30:06.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380123f0 con 0x7f595013e870 2020-11-16T22:30:06.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595016bfa0 con 0x7f595013e870 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950044f60 con 0x7f595013e870 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5938012760 con 0x7f595013e870 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.449+0000 7f59477fe700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 0x7f592c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.453+0000 7f594f7fe700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 0x7f592c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.453+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f593806e340 con 0x7f595013e870 2020-11-16T22:30:06.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.453+0000 7f594f7fe700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 0x7f592c057550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f59400092d0 tx=0x7f5940016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.457+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f59380445a0 con 0x7f595013e870 2020-11-16T22:30:06.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.661+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f595013ef20 con 0x7f595013e870 2020-11-16T22:30:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f59477fe700 1 -- 172.21.15.181:0/2275874756 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f5938047bc0 con 0x7f595013e870 2020-11-16T22:30:06.647 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:06.647 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":53,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:03.819455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 msgr2=0x7f592c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 0x7f592c057550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f59400092d0 tx=0x7f5940016000).stop 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 msgr2=0x7f59500c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f593800ef50 tx=0x7f5938006df0).stop 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 shutdown_connections 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f592c055090 0x7f592c057550 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015fdc0 0x7f5950157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5950061100 0x7f59500c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 --2- 172.21.15.181:0/2275874756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f595013e870 0x7f59500c9780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 >> 172.21.15.181:0/2275874756 conn(0x7f59500cf630 msgr2=0x7f5950061530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 shutdown_connections 2020-11-16T22:30:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.665+0000 7f5954c47700 1 -- 172.21.15.181:0/2275874756 wait complete. 2020-11-16T22:30:06.651 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 53 2020-11-16T22:30:06.661 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:30:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.805+0000 7f8350912700 1 Processor -- start 2020-11-16T22:30:06.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.805+0000 7f8350912700 1 -- start start 2020-11-16T22:30:06.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f834c15d520 0x7f834c064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f834c0654c0 0x7f834c160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c0c85e0 con 0x7f834c15d140 2020-11-16T22:30:06.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834c15ff70 con 0x7f834c0654c0 2020-11-16T22:30:06.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834c0c8840 con 0x7f834c15d520 2020-11-16T22:30:06.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f834c15d520 0x7f834c064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f834c15d520 0x7f834c064fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59648/0 (socket says 172.21.15.181:59648) 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 -- 172.21.15.181:0/2643080758 learned_addr learned my addr 172.21.15.181:0/2643080758 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834bfff700 1 --2- 172.21.15.181:0/2643080758 >> v2:172.21.15.175:3300/0 conn(0x7f834c0654c0 0x7f834c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 -- 172.21.15.181:0/2643080758 >> v2:172.21.15.175:3300/0 conn(0x7f834c0654c0 msgr2=0x7f834c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 --2- 172.21.15.181:0/2643080758 >> v2:172.21.15.175:3300/0 conn(0x7f834c0654c0 0x7f834c160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 -- 172.21.15.181:0/2643080758 >> v1:172.21.15.175:6789/0 conn(0x7f834c15d140 legacy=0x7f834c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 -- 172.21.15.181:0/2643080758 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0c8ae0 con 0x7f834c15d520 2020-11-16T22:30:06.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834affd700 1 --2- 172.21.15.181:0/2643080758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c064fd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f834c044850 tx=0x7f833c006eb0).ready entity=mon.1 client_cookie=632f25534bbbf0f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834a7fc700 1 -- 172.21.15.181:0/2643080758 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c009810 con 0x7f834c15d520 2020-11-16T22:30:06.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834a7fc700 1 -- 172.21.15.181:0/2643080758 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c009970 con 0x7f834c15d520 2020-11-16T22:30:06.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f834a7fc700 1 -- 172.21.15.181:0/2643080758 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c009cf0 con 0x7f834c15d520 2020-11-16T22:30:06.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- 172.21.15.181:0/2643080758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 msgr2=0x7f834c064fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 --2- 172.21.15.181:0/2643080758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c064fd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f834c044850 tx=0x7f833c006eb0).stop 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- 172.21.15.181:0/2643080758 shutdown_connections 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 --2- 172.21.15.181:0/2643080758 >> v2:172.21.15.175:3300/0 conn(0x7f834c0654c0 0x7f834c160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 --2- 172.21.15.181:0/2643080758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c064fd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- 172.21.15.181:0/2643080758 >> 172.21.15.181:0/2643080758 conn(0x7f834c0d2dc0 msgr2=0x7f834c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- 172.21.15.181:0/2643080758 shutdown_connections 2020-11-16T22:30:06.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.809+0000 7f8350912700 1 -- 172.21.15.181:0/2643080758 wait complete. 2020-11-16T22:30:06.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 Processor -- start 2020-11-16T22:30:06.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- start start 2020-11-16T22:30:06.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834c0654c0 0x7f834c16b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 0x7f834c1441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c145380 con 0x7f834c0654c0 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c16ad40 con 0x7f834c15d520 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c16f060 con 0x7f834c15d140 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 0x7f834c1441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59654/0 (socket says 172.21.15.181:59654) 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 -- 172.21.15.181:0/1215160565 learned_addr learned my addr 172.21.15.181:0/1215160565 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834b7fe700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834c0654c0 0x7f834c16b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 -- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 msgr2=0x7f834c1441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 0x7f834c1441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 -- 172.21.15.181:0/1215160565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834c0654c0 msgr2=0x7f834c16b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834c0654c0 0x7f834c16b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 -- 172.21.15.181:0/1215160565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c16eb20 con 0x7f834c15d520 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834affd700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 0x7f834c1441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f834bfff700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f834400ef50 tx=0x7f83440091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f832b7fe700 1 -- 172.21.15.181:0/1215160565 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344009690 con 0x7f834c15d520 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f832b7fe700 1 -- 172.21.15.181:0/1215160565 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834400cb40 con 0x7f834c15d520 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f832b7fe700 1 -- 172.21.15.181:0/1215160565 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400cee0 con 0x7f834c15d520 2020-11-16T22:30:06.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c16edc0 con 0x7f834c15d520 2020-11-16T22:30:06.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.813+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c1446c0 con 0x7f834c15d520 2020-11-16T22:30:06.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.817+0000 7f832b7fe700 1 -- 172.21.15.181:0/1215160565 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8344022070 con 0x7f834c15d520 2020-11-16T22:30:06.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.817+0000 7f832b7fe700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 0x7f8324057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.817+0000 7f832b7fe700 1 -- 172.21.15.181:0/1215160565 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f834406e4a0 con 0x7f834c15d520 2020-11-16T22:30:06.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.817+0000 7f834b7fe700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 0x7f8324057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.817+0000 7f834b7fe700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 0x7f8324057520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8340003300 tx=0x7f8340007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.821+0000 7f8360605740 1 Processor -- start 2020-11-16T22:30:06.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.821+0000 7f8360605740 1 -- start start 2020-11-16T22:30:06.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 0x12ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12ab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11364b0 con 0x108c7b0 2020-11-16T22:30:06.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ea930 con 0x101ebb0 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x10aa6f0 con 0x12aafd0 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39272/0 (socket says 172.21.15.181:39272) 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 -- 172.21.15.181:0/538214553 learned_addr learned my addr 172.21.15.181:0/538214553 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834b7fe700 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 0x12ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 -- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 msgr2=0x12ab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12ab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 -- 172.21.15.181:0/538214553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 msgr2=0x12ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 0x12ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 -- 172.21.15.181:0/538214553 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10a55a0 con 0x12aafd0 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834b7fe700 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 0x12ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f834bfff700 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8344048060 tx=0x7f8344015000).ready entity=mon.2 client_cookie=29eda0e938086ca8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8328ff9700 1 -- 172.21.15.181:0/538214553 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834406e0a0 con 0x12aafd0 2020-11-16T22:30:06.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8328ff9700 1 -- 172.21.15.181:0/538214553 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834406e200 con 0x12aafd0 2020-11-16T22:30:06.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8328ff9700 1 -- 172.21.15.181:0/538214553 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344047ac0 con 0x12aafd0 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 msgr2=0x12a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8344048060 tx=0x7f8344015000).stop 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- 172.21.15.181:0/538214553 shutdown_connections 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12aafd0 0x12a4700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x108c7b0 0x12ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 --2- 172.21.15.181:0/538214553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- 172.21.15.181:0/538214553 >> 172.21.15.181:0/538214553 conn(0x125ae40 msgr2=0x12a73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- 172.21.15.181:0/538214553 shutdown_connections 2020-11-16T22:30:06.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.825+0000 7f8360605740 1 -- 172.21.15.181:0/538214553 wait complete. 2020-11-16T22:30:06.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 -- start start 2020-11-16T22:30:06.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 Processor -- start 2020-11-16T22:30:06.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12aafd0 0x12cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11d76e0 con 0x12aafd0 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11d7820 con 0x101ebb0 2020-11-16T22:30:06.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59664/0 (socket says 172.21.15.181:59664) 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x11d8a90 con 0x108c7b0 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 -- 172.21.15.181:0/2616515871 learned_addr learned my addr 172.21.15.181:0/2616515871 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834affd700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8360605740 10 client.0 ll_register_callbacks cb 0x11cdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834bfff700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12aafd0 0x12cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 -- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 msgr2=0x12a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 -- 172.21.15.181:0/2616515871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12aafd0 msgr2=0x12cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12aafd0 0x12cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 -- 172.21.15.181:0/2616515871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11c8280 con 0x101ebb0 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f834b7fe700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f8340015750 tx=0x7f834000a000).ready entity=mon.1 client_cookie=413cbfa171a0110d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.829+0000 7f8321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:06.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834001c070 con 0x101ebb0 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x128b6c0 con 0x101ebb0 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8340012630 con 0x101ebb0 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8340012a00 con 0x101ebb0 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 10 client.5848 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0xffdfd0 con 0x101ebb0 2020-11-16T22:30:06.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x128b6c0 con 0x101ebb0 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f8340012b60 con 0x101ebb0 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 10 client.5848 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 10 client.5848 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 20 client.5848 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12a9920 0x130d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 4 client.5848 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:06.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x10c8c50 con 0x12a9920 2020-11-16T22:30:06.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f834affd700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12a9920 0x130d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 10 client.5848 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:06.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f834affd700 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12a9920 0x130d6e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=8f56030b726e2265 server_cookie=1f102049bb2f94e8 in_seq=0 out_seq=0 2020-11-16T22:30:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 1 -- 172.21.15.181:0/2616515871 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x10c8c50 con 0x12a9920 2020-11-16T22:30:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 10 client.5848 handle_command_reply: tid=0 2020-11-16T22:30:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 client.5848 shutdown 2020-11-16T22:30:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 msgr2=0x12fb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f8340015750 tx=0x7f834000a000).stop 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 shutdown_connections 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12a9920 0x130d6e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12aafd0 0x12cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x101ebb0 0x12fb6d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 --2- 172.21.15.181:0/2616515871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8321ffb700 0 client.5848 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:06.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 >> 172.21.15.181:0/2616515871 conn(0x125ae40 msgr2=0x12de270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 shutdown_connections 2020-11-16T22:30:06.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 1 -- 172.21.15.181:0/2616515871 wait complete. 2020-11-16T22:30:06.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.833+0000 7f8360605740 20 client.5848 trim_cache size 0 max 16384 2020-11-16T22:30:06.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 Processor -- start 2020-11-16T22:30:06.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 -- start start 2020-11-16T22:30:06.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 0x12ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x101ebb0 0x12f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11afe40 con 0x101ebb0 2020-11-16T22:30:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x130e670 con 0x125ae40 2020-11-16T22:30:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12cb340 con 0x108c7b0 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 0x12ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 0x12ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59672/0 (socket says 172.21.15.181:59672) 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834b7fe700 1 -- 172.21.15.181:0/336936946 learned_addr learned my addr 172.21.15.181:0/336936946 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834affd700 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x101ebb0 0x12f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 -- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 msgr2=0x12ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 0x12ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 -- 172.21.15.181:0/336936946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x101ebb0 msgr2=0x12f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x101ebb0 0x12f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 -- 172.21.15.181:0/336936946 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180081d0 con 0x108c7b0 2020-11-16T22:30:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f834bfff700 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12aa1b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x12ab480 tx=0x7f834401c000).ready entity=mon.2 client_cookie=5e3690ae27d0852e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f83297fa700 1 -- 172.21.15.181:0/336936946 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344035070 con 0x108c7b0 2020-11-16T22:30:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f83297fa700 1 -- 172.21.15.181:0/336936946 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8344046840 con 0x108c7b0 2020-11-16T22:30:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.853+0000 7f83297fa700 1 -- 172.21.15.181:0/336936946 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834406e0a0 con 0x108c7b0 2020-11-16T22:30:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 msgr2=0x12aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12aa1b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x12ab480 tx=0x7f834401c000).stop 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- 172.21.15.181:0/336936946 shutdown_connections 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x108c7b0 0x12aa1b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125ae40 0x12ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- 172.21.15.181:0/336936946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x101ebb0 0x12f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- 172.21.15.181:0/336936946 >> 172.21.15.181:0/336936946 conn(0x7f8318002320 msgr2=0x12a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- 172.21.15.181:0/336936946 shutdown_connections 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- 172.21.15.181:0/336936946 wait complete. 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- start start 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 Processor -- start 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x108c7b0 0x12acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 0x1359e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x12a0120 con 0x125ae40 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x135a380 con 0x108c7b0 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x135c910 con 0x101ebb0 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x108c7b0 0x12acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39288/0 (socket says 172.21.15.181:39288) 2020-11-16T22:30:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 -- 172.21.15.181:0/135862704 learned_addr learned my addr 172.21.15.181:0/135862704 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8360605740 10 client.0 ll_register_callbacks cb 0x12abc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834bfff700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 0x1359e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 -- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x108c7b0 msgr2=0x12acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x108c7b0 0x12acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 -- 172.21.15.181:0/135862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 msgr2=0x1359e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 0x1359e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 -- 172.21.15.181:0/135862704 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x103c720 con 0x101ebb0 2020-11-16T22:30:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834bfff700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 0x1359e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f834b7fe700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f8340017f90 tx=0x7f8340021000).ready entity=mon.2 client_cookie=51e156587a5399ab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.857+0000 7f8321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340017320 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8340017480 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f834406e200 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8340023a20 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 10 client.5489 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x1047e00 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834406e200 con 0x101ebb0 2020-11-16T22:30:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f8340034030 con 0x101ebb0 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 10 client.5489 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 10 client.5489 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 20 client.5489 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12aafd0 0x12f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 4 client.5489 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x10cff50 con 0x12aafd0 2020-11-16T22:30:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f834affd700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12aafd0 0x12f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f8321ffb700 10 client.5489 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.861+0000 7f834affd700 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12aafd0 0x12f88e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=c624f3b6af6831fd server_cookie=4717985c9fcaafc2 in_seq=0 out_seq=0 2020-11-16T22:30:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8321ffb700 1 -- 172.21.15.181:0/135862704 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x10cff50 con 0x12aafd0 2020-11-16T22:30:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8321ffb700 10 client.5489 handle_command_reply: tid=0 2020-11-16T22:30:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 client.5489 shutdown 2020-11-16T22:30:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 msgr2=0x12a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f8340017f90 tx=0x7f8340021000).stop 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 shutdown_connections 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x12aafd0 0x12f88e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125ae40 0x1359e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x101ebb0 0x12a9f80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 --2- 172.21.15.181:0/135862704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x108c7b0 0x12acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8321ffb700 0 client.5489 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 >> 172.21.15.181:0/135862704 conn(0x7f8318002320 msgr2=0x12a52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 shutdown_connections 2020-11-16T22:30:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 1 -- 172.21.15.181:0/135862704 wait complete. 2020-11-16T22:30:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8360605740 20 client.5489 trim_cache size 0 max 16384 2020-11-16T22:30:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 msgr2=0x7f8324057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 0x7f8324057520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8340003300 tx=0x7f8340007b00).stop 2020-11-16T22:30:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 msgr2=0x7f834c16e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f834400ef50 tx=0x7f83440091d0).stop 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 shutdown_connections 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8324055060 0x7f8324057520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f834c15d520 0x7f834c16e150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834c0654c0 0x7f834c16b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 --2- 172.21.15.181:0/1215160565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f834c15d140 0x7f834c1441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 >> 172.21.15.181:0/1215160565 conn(0x7f834c0d2dc0 msgr2=0x7f834c161720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 shutdown_connections 2020-11-16T22:30:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:06.881+0000 7f8350912700 1 -- 172.21.15.181:0/1215160565 wait complete. 2020-11-16T22:30:06.867 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:06.867 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:30:06.867 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:30:06.867 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:06.881 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:07.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 Processor -- start 2020-11-16T22:30:07.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 -- start start 2020-11-16T22:30:07.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64ec161cc0 0x7f64ec0637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec063ca0 con 0x7f64ec0633d0 2020-11-16T22:30:07.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec064970 con 0x7f64ec061810 2020-11-16T22:30:07.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f3097700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec065930 con 0x7f64ec161cc0 2020-11-16T22:30:07.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64ebfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f64ec061810 0x7f64ec062330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45758/0 (socket says 172.21.15.181:45758) 2020-11-16T22:30:07.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64ebfff700 1 -- 172.21.15.181:0/4165417629 learned_addr learned my addr 172.21.15.181:0/4165417629 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:07.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.021+0000 7f64f1634700 1 --2- 172.21.15.181:0/4165417629 >> v2:172.21.15.181:3300/0 conn(0x7f64ec161cc0 0x7f64ec0637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652751591 0 0) 0x7f64ec064970 con 0x7f64ec061810 2020-11-16T22:30:07.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d8003240 con 0x7f64ec061810 2020-11-16T22:30:07.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376428540 0 0) 0x7f64ec063ca0 con 0x7f64ec0633d0 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f1634700 1 -- 172.21.15.181:0/4165417629 >> v1:172.21.15.181:6789/0 conn(0x7f64ec061810 legacy=0x7f64ec062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f1634700 1 -- 172.21.15.181:0/4165417629 >> v1:172.21.15.175:6789/0 conn(0x7f64ec0633d0 legacy=0x7f64ec061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f1634700 1 -- 172.21.15.181:0/4165417629 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec064c10 con 0x7f64ec161cc0 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f1634700 1 --2- 172.21.15.181:0/4165417629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec0637b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f64e4009c90 tx=0x7f64e40091d0).ready entity=mon.1 client_cookie=e2ae0ad115dbce11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400ba70 con 0x7f64ec161cc0 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400bbd0 con 0x7f64ec161cc0 2020-11-16T22:30:07.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64eb7fe700 1 -- 172.21.15.181:0/4165417629 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011400 con 0x7f64ec161cc0 2020-11-16T22:30:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 -- 172.21.15.181:0/4165417629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 msgr2=0x7f64ec0637b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 --2- 172.21.15.181:0/4165417629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec0637b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f64e4009c90 tx=0x7f64e40091d0).stop 2020-11-16T22:30:07.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 -- 172.21.15.181:0/4165417629 shutdown_connections 2020-11-16T22:30:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 --2- 172.21.15.181:0/4165417629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec0637b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 -- 172.21.15.181:0/4165417629 >> 172.21.15.181:0/4165417629 conn(0x7f64ec0cf890 msgr2=0x7f64ec0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 -- 172.21.15.181:0/4165417629 shutdown_connections 2020-11-16T22:30:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.025+0000 7f64f3097700 1 -- 172.21.15.181:0/4165417629 wait complete. 2020-11-16T22:30:07.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 Processor -- start 2020-11-16T22:30:07.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- start start 2020-11-16T22:30:07.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ec0633d0 0x7f64ec1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec16cef0 con 0x7f64ec0633d0 2020-11-16T22:30:07.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec16d190 con 0x7f64ec161cc0 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec16d430 con 0x7f64ec061810 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f1634700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ec0633d0 0x7f64ec1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39304/0 (socket says 172.21.15.181:39304) 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f1634700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec1579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59696/0 (socket says 172.21.15.181:59696) 2020-11-16T22:30:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 -- 172.21.15.181:0/3323727508 learned_addr learned my addr 172.21.15.181:0/3323727508 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 -- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 msgr2=0x7f64ec1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 -- 172.21.15.181:0/3323727508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ec0633d0 msgr2=0x7f64ec1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ec0633d0 0x7f64ec1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 -- 172.21.15.181:0/3323727508 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec16d6d0 con 0x7f64ec061810 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f0e33700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f64dc007fd0 tx=0x7f64dc007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00e490 con 0x7f64ec061810 2020-11-16T22:30:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc00e5f0 con 0x7f64ec061810 2020-11-16T22:30:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec16d970 con 0x7f64ec061810 2020-11-16T22:30:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00e9b0 con 0x7f64ec061810 2020-11-16T22:30:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.029+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec1534f0 con 0x7f64ec061810 2020-11-16T22:30:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec069eb0 con 0x7f64ec061810 2020-11-16T22:30:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f64dc019030 con 0x7f64ec061810 2020-11-16T22:30:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64e8ff9700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 0x7f64c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f64dc06d280 con 0x7f64ec061810 2020-11-16T22:30:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64ebfff700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 0x7f64c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.033+0000 7f64ebfff700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 0x7f64c4057520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f64ec0613e0 tx=0x7f64e0007980).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.037+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f64dc0434e0 con 0x7f64ec061810 2020-11-16T22:30:07.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ec05d4e0 con 0x7f64ec061810 2020-11-16T22:30:07.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64e8ff9700 1 -- 172.21.15.181:0/3323727508 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f64dc046b00 con 0x7f64ec061810 2020-11-16T22:30:07.219 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:07.219 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":53,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:03.819455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:07.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 msgr2=0x7f64c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 0x7f64c4057520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f64ec0613e0 tx=0x7f64e0007980).stop 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 msgr2=0x7f64ec153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f64dc007fd0 tx=0x7f64dc007ac0).stop 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 shutdown_connections 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f64c4055060 0x7f64c4057520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ec161cc0 0x7f64ec1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64ec061810 0x7f64ec153000 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 --2- 172.21.15.181:0/3323727508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ec0633d0 0x7f64ec1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.237+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 >> 172.21.15.181:0/3323727508 conn(0x7f64ec0cf890 msgr2=0x7f64ec0d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.241+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 shutdown_connections 2020-11-16T22:30:07.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.241+0000 7f64f3097700 1 -- 172.21.15.181:0/3323727508 wait complete. 2020-11-16T22:30:07.224 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 53 2020-11-16T22:30:07.240 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:30:07.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.381+0000 7fe00759e700 1 Processor -- start 2020-11-16T22:30:07.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.381+0000 7fe00759e700 1 -- start start 2020-11-16T22:30:07.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 0x7fe008160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080c85e0 con 0x7fe00815d140 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00815ff70 con 0x7fe0080654c0 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080c8840 con 0x7fe00815d520 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39310/0 (socket says 172.21.15.181:39310) 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 -- 172.21.15.181:0/1730062285 learned_addr learned my addr 172.21.15.181:0/1730062285 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe006d9d700 1 --2- 172.21.15.181:0/1730062285 >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 0x7fe008160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 -- 172.21.15.181:0/1730062285 >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 msgr2=0x7fe008160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 --2- 172.21.15.181:0/1730062285 >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 0x7fe008160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 -- 172.21.15.181:0/1730062285 >> v1:172.21.15.175:6789/0 conn(0x7fe00815d140 legacy=0x7fe0080cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 -- 172.21.15.181:0/1730062285 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080c8ae0 con 0x7fe00815d520 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe006d9d700 1 --2- 172.21.15.181:0/1730062285 >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 0x7fe008160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe005d9b700 1 --2- 172.21.15.181:0/1730062285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fdffc006fd0 tx=0x7fdffc006eb0).ready entity=mon.2 client_cookie=d8d9e7f7b0b4649d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00559a700 1 -- 172.21.15.181:0/1730062285 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc009890 con 0x7fe00815d520 2020-11-16T22:30:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00559a700 1 -- 172.21.15.181:0/1730062285 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc0099f0 con 0x7fe00815d520 2020-11-16T22:30:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00559a700 1 -- 172.21.15.181:0/1730062285 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc009d70 con 0x7fe00815d520 2020-11-16T22:30:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- 172.21.15.181:0/1730062285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 msgr2=0x7fe008064fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 --2- 172.21.15.181:0/1730062285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fdffc006fd0 tx=0x7fdffc006eb0).stop 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- 172.21.15.181:0/1730062285 shutdown_connections 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 --2- 172.21.15.181:0/1730062285 >> v2:172.21.15.175:3300/0 conn(0x7fe0080654c0 0x7fe008160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 --2- 172.21.15.181:0/1730062285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00815d520 0x7fe008064fd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- 172.21.15.181:0/1730062285 >> 172.21.15.181:0/1730062285 conn(0x7fe0080d2dc0 msgr2=0x7fe0080ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- 172.21.15.181:0/1730062285 shutdown_connections 2020-11-16T22:30:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- 172.21.15.181:0/1730062285 wait complete. 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 Processor -- start 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.385+0000 7fe00759e700 1 -- start start 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00815d140 0x7fe0080cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00812ea70 0x7fe0080cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080d0e70 con 0x7fe00812ea70 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080d1110 con 0x7fe00815d140 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080d13b0 con 0x7fe0080654c0 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39316/0 (socket says 172.21.15.181:39316) 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 -- 172.21.15.181:0/2116695492 learned_addr learned my addr 172.21.15.181:0/2116695492 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00815d140 0x7fe0080cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe006d9d700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00812ea70 0x7fe0080cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 -- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00815d140 msgr2=0x7fe0080cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00815d140 0x7fe0080cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 -- 172.21.15.181:0/2116695492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00812ea70 msgr2=0x7fe0080cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00812ea70 0x7fe0080cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 -- 172.21.15.181:0/2116695492 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080d1650 con 0x7fe0080654c0 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00659c700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fdff800c760 tx=0x7fdff8007ba0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fdff67fc700 1 -- 172.21.15.181:0/2116695492 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800e560 con 0x7fe0080654c0 2020-11-16T22:30:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fdff67fc700 1 -- 172.21.15.181:0/2116695492 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff800e6e0 con 0x7fe0080654c0 2020-11-16T22:30:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0080d18f0 con 0x7fe0080654c0 2020-11-16T22:30:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fdff67fc700 1 -- 172.21.15.181:0/2116695492 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800eaa0 con 0x7fe0080654c0 2020-11-16T22:30:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00815b8b0 con 0x7fe0080654c0 2020-11-16T22:30:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.389+0000 7fdff67fc700 1 -- 172.21.15.181:0/2116695492 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdff8019030 con 0x7fe0080654c0 2020-11-16T22:30:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.393+0000 7fdff67fc700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 0x7fdfdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.393+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 0x7fdfdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.393+0000 7fdff67fc700 1 -- 172.21.15.181:0/2116695492 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdff806d3e0 con 0x7fe0080654c0 2020-11-16T22:30:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.393+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 0x7fdfdc057520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fe008044850 tx=0x7fdffc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 Processor -- start 2020-11-16T22:30:07.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 -- start start 2020-11-16T22:30:07.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1db8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c434b0 con 0x1b2bbb0 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf7930 con 0x1b997b0 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1bb76f0 con 0x1db7fd0 2020-11-16T22:30:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe00659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe00659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59714/0 (socket says 172.21.15.181:59714) 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe00659c700 1 -- 172.21.15.181:0/3803998527 learned_addr learned my addr 172.21.15.181:0/3803998527 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39328/0 (socket says 172.21.15.181:39328) 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe005d9b700 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1db8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 -- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 msgr2=0x1dba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 -- 172.21.15.181:0/3803998527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 msgr2=0x1db8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1db8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 -- 172.21.15.181:0/3803998527 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1bb25a0 con 0x1db7fd0 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fe006d9d700 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fe00000abf0 tx=0x7fe000008040).ready entity=mon.2 client_cookie=4f39d49d7e8565e4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fdfe37fe700 1 -- 172.21.15.181:0/3803998527 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000017070 con 0x1db7fd0 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fdfe37fe700 1 -- 172.21.15.181:0/3803998527 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00000c030 con 0x1db7fd0 2020-11-16T22:30:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.397+0000 7fdfe37fe700 1 -- 172.21.15.181:0/3803998527 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000fc40 con 0x1db7fd0 2020-11-16T22:30:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 msgr2=0x1db1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fe00000abf0 tx=0x7fe000008040).stop 2020-11-16T22:30:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- 172.21.15.181:0/3803998527 shutdown_connections 2020-11-16T22:30:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1db1700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/3803998527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1db8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- 172.21.15.181:0/3803998527 >> 172.21.15.181:0/3803998527 conn(0x1d67e40 msgr2=0x1db43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- 172.21.15.181:0/3803998527 shutdown_connections 2020-11-16T22:30:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- 172.21.15.181:0/3803998527 wait complete. 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- start start 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 Processor -- start 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1e1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce46e0 con 0x1b2bbb0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce4820 con 0x1b997b0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ce5a90 con 0x1db7fd0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe01b9b7740 10 client.0 ll_register_callbacks cb 0x1cdaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59722/0 (socket says 172.21.15.181:59722) 2020-11-16T22:30:07.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39334/0 (socket says 172.21.15.181:39334) 2020-11-16T22:30:07.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe005d9b700 1 -- 172.21.15.181:0/1290305930 learned_addr learned my addr 172.21.15.181:0/1290305930 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.401+0000 7fe00659c700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1e1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 -- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 msgr2=0x1e05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 -- 172.21.15.181:0/1290305930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 msgr2=0x1e1a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1e1a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 -- 172.21.15.181:0/1290305930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1cd5280 con 0x1b997b0 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe006d9d700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe005d9b700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fdffc010000 tx=0x7fdffc01e000).ready entity=mon.1 client_cookie=c533f73e188d8cb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00b600 con 0x1b997b0 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdffc01f8d0 con 0x1b997b0 2020-11-16T22:30:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d986c0 con 0x1b997b0 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdffc00b760 con 0x1b997b0 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 10 client.5872 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x1b0afd0 con 0x1b997b0 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d986c0 con 0x1b997b0 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fdffc01fca0 con 0x1b997b0 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 10 client.5872 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 10 client.5872 resolve_mds: resolved ID 'a' to GID 5599 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 20 client.5872 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1e05b30 0x1dba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 4 client.5872 mds_command: new command op to 5599 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 --> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1bd5c50 con 0x1e05b30 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe00659c700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1e05b30 0x1dba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 10 client.5872 ms_handle_connect on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe00659c700 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1e05b30 0x1dba2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=9caf4e227faaa9bc server_cookie=277e812d26ac09d1 in_seq=0 out_seq=0 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/1290305930 <== mds.2 v2:172.21.15.181:6834/3960101400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1bd5c50 con 0x1e05b30 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 10 client.5872 handle_command_reply: tid=0 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 client.5872 shutdown 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 msgr2=0x1db4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fdffc010000 tx=0x7fdffc01e000).stop 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 shutdown_connections 2020-11-16T22:30:07.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1e05b30 0x1dba2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1db7fd0 0x1e05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fdfe0ff9700 0 client.5872 ms_handle_reset on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b2bbb0 0x1e1a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/1290305930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1db4530 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 >> 172.21.15.181:0/1290305930 conn(0x1d67e40 msgr2=0x1db9340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 shutdown_connections 2020-11-16T22:30:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 1 -- 172.21.15.181:0/1290305930 wait complete. 2020-11-16T22:30:07.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.405+0000 7fe01b9b7740 20 client.5872 trim_cache size 0 max 16384 2020-11-16T22:30:07.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.421+0000 7fe01b9b7740 1 Processor -- start 2020-11-16T22:30:07.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.421+0000 7fe01b9b7740 1 -- start start 2020-11-16T22:30:07.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dacd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cbce40 con 0x1d67e40 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ded130 con 0x1b997b0 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1deb9d0 con 0x1b2bbb0 2020-11-16T22:30:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dacd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39338/0 (socket says 172.21.15.181:39338) 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 -- 172.21.15.181:0/2757150992 learned_addr learned my addr 172.21.15.181:0/2757150992 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe006d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dacd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59732/0 (socket says 172.21.15.181:59732) 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe00659c700 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 -- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 msgr2=0x1dacd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dacd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 -- 172.21.15.181:0/2757150992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 msgr2=0x1db4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 -- 172.21.15.181:0/2757150992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0008160 con 0x1b2bbb0 2020-11-16T22:30:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fdffc00b150 tx=0x7fdffc029000).ready entity=mon.2 client_cookie=5be827ec5d8af4bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fdfe3fff700 1 -- 172.21.15.181:0/2757150992 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc03e070 con 0x1b2bbb0 2020-11-16T22:30:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fdfe3fff700 1 -- 172.21.15.181:0/2757150992 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc01fca0 con 0x1b2bbb0 2020-11-16T22:30:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fdfe3fff700 1 -- 172.21.15.181:0/2757150992 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc01fe10 con 0x1b2bbb0 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 msgr2=0x1dac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fdffc00b150 tx=0x7fdffc029000).stop 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2757150992 shutdown_connections 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b997b0 0x1dacd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2757150992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b2bbb0 0x1dac870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2757150992 >> 172.21.15.181:0/2757150992 conn(0x7fdfd0002320 msgr2=0x1e0a260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2757150992 shutdown_connections 2020-11-16T22:30:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2757150992 wait complete. 2020-11-16T22:30:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 -- start start 2020-11-16T22:30:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.425+0000 7fe01b9b7740 1 Processor -- start 2020-11-16T22:30:07.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 0x1dea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 0x1db2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1db2bf0 con 0x1d67e40 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e69ac0 con 0x1b2bbb0 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e69d60 con 0x1b997b0 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 10 client.0 ll_register_callbacks cb 0x1db8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 0x1db2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 0x1dea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe005d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 0x1db2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39346/0 (socket says 172.21.15.181:39346) 2020-11-16T22:30:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe005d9b700 1 -- 172.21.15.181:0/2875173386 learned_addr learned my addr 172.21.15.181:0/2875173386 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe006d9d700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 -- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 msgr2=0x1db2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 0x1db2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 -- 172.21.15.181:0/2875173386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 msgr2=0x1db6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 -- 172.21.15.181:0/2875173386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b49720 con 0x1b2bbb0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe00659c700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 0x1dea6a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fdff8046fa0 tx=0x7fdff801e040).ready entity=mon.1 client_cookie=d7d7bb5fc941b7a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff80469a0 con 0x1b2bbb0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdff806baf0 con 0x1b2bbb0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe00000c030 con 0x1b2bbb0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdff8043090 con 0x1b2bbb0 2020-11-16T22:30:07.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 10 client.5878 fetch_fsmap learned FSMap version 53 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x1b54e00 con 0x1b2bbb0 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00000c030 con 0x1b2bbb0 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fdff80431f0 con 0x1b2bbb0 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 10 client.5878 fetch_fsmap finished waiting for FSMap version 53 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 10 client.5878 resolve_mds: resolved ID 'a' to GID 5599 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 20 client.5878 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:07.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1db7fd0 0x1dacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:07.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 4 client.5878 mds_command: new command op to 5599 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:30:07.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 --> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1bdcf50 con 0x1db7fd0 2020-11-16T22:30:07.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1db7fd0 0x1dacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:07.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fdfe0ff9700 10 client.5878 ms_handle_connect on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:07.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.429+0000 7fe005d9b700 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1db7fd0 0x1dacc30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=cda556600d017b2e server_cookie=da2627db95704834 in_seq=0 out_seq=0 2020-11-16T22:30:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fdfe0ff9700 1 -- 172.21.15.181:0/2875173386 <== mds.2 v2:172.21.15.181:6834/3960101400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1bdcf50 con 0x1db7fd0 2020-11-16T22:30:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fdfe0ff9700 10 client.5878 handle_command_reply: tid=0 2020-11-16T22:30:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 client.5878 shutdown 2020-11-16T22:30:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 msgr2=0x1dea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 0x1dea6a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fdff8046fa0 tx=0x7fdff801e040).stop 2020-11-16T22:30:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 shutdown_connections 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0x1db7fd0 0x1dacc30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fdfe0ff9700 0 client.5878 ms_handle_reset on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d67e40 0x1db6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b2bbb0 0x1dea6a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 --2- 172.21.15.181:0/2875173386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b997b0 0x1db2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 >> 172.21.15.181:0/2875173386 conn(0x7fdfd0002320 msgr2=0x1e1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 shutdown_connections 2020-11-16T22:30:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 1 -- 172.21.15.181:0/2875173386 wait complete. 2020-11-16T22:30:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.445+0000 7fe01b9b7740 20 client.5878 trim_cache size 0 max 16384 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 msgr2=0x7fdfdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 0x7fdfdc057520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fe008044850 tx=0x7fdffc015000).stop 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 msgr2=0x7fe00812e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fdff800c760 tx=0x7fdff8007ba0).stop 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 shutdown_connections 2020-11-16T22:30:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdfdc055060 0x7fdfdc057520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe00812ea70 0x7fe0080cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080654c0 0x7fe00812e580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 --2- 172.21.15.181:0/2116695492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00815d140 0x7fe0080cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 >> 172.21.15.181:0/2116695492 conn(0x7fe0080d2dc0 msgr2=0x7fe008139720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 shutdown_connections 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:07.449+0000 7fe00759e700 1 -- 172.21.15.181:0/2116695492 wait complete. 2020-11-16T22:30:07.433 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:07.434 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:30:07.434 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:30:07.434 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:07.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:30:07.446 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:30:07.502 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/ 2020-11-16T22:30:07.502 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4 2020-11-16T22:30:07.502 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5 2020-11-16T22:30:07.503 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:30:07.503 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:30:07.503 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:30:07.507 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:07.507 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:07.508 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:07.508 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:07.508 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:30:07.509 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:07.622 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2020-11-16T22:30:08.115 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:08.116 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:08.116 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:08.117 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:08.117 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:30:08.117 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:08.233 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:08.532 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627781 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:08.874 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627782 2020-11-16T22:30:09.062 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:30:09.063 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:30:09.063 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:30:09.064 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:30:09.064 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:30:09.064 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:30:09.174 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627783 2020-11-16T22:30:09.363 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:30:09.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 Processor -- start 2020-11-16T22:30:09.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 -- start start 2020-11-16T22:30:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fec0 0x55f5a3d22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3dae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3c751f0 con 0x55f5a3dac4a0 2020-11-16T22:30:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3c43be0 con 0x55f5a3d1fec0 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3d23130 con 0x55f5a3d1fa60 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a7a7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fec0 0x55f5a3d22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39360/0 (socket says 172.21.15.181:39360) 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 -- 172.21.15.181:0/1977440200 learned_addr learned my addr 172.21.15.181:0/1977440200 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:09.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a8a7e700 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3dae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 -- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fec0 msgr2=0x55f5a3d22c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fec0 0x55f5a3d22c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 -- 172.21.15.181:0/1977440200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 msgr2=0x55f5a3dae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3dae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a827d700 1 -- 172.21.15.181:0/1977440200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a3c6d8a0 con 0x55f5a3d1fa60 2020-11-16T22:30:09.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.417+0000 7fb8a8a7e700 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3dae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x55f5a3cb3c30 tx=0x7fb890015000).ready entity=mon.2 client_cookie=37081b9d8e7a4fc6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8a6a7a700 1 -- 172.21.15.181:0/1977440200 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890008e80 con 0x55f5a3d1fa60 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8a6a7a700 1 -- 172.21.15.181:0/1977440200 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000a690 con 0x55f5a3d1fa60 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8a6a7a700 1 -- 172.21.15.181:0/1977440200 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890012810 con 0x55f5a3d1fa60 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 msgr2=0x55f5a3daeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x55f5a3cb3c30 tx=0x7fb890015000).stop 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1977440200 shutdown_connections 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3dae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fa60 0x55f5a3daeea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1977440200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fec0 0x55f5a3d22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1977440200 >> 172.21.15.181:0/1977440200 conn(0x55f5a3d203e0 msgr2=0x55f5a3d02980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:09.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1977440200 shutdown_connections 2020-11-16T22:30:09.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1977440200 wait complete. 2020-11-16T22:30:09.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.421+0000 7fb8b6c77c40 1 Processor -- start 2020-11-16T22:30:09.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- start start 2020-11-16T22:30:09.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fa60 0x55f5a3dab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 0x55f5a3da68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3da6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3d191c0 con 0x55f5a3dac4a0 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3da73e0 con 0x55f5a3d1fa60 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5a3da7680 con 0x55f5a3d1fec0 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 0x55f5a3da68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 0x55f5a3da68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39364/0 (socket says 172.21.15.181:39364) 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 -- 172.21.15.181:0/1903438724 learned_addr learned my addr 172.21.15.181:0/1903438724 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fa60 0x55f5a3dab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a8a7e700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3da6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 -- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fa60 msgr2=0x55f5a3dab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f5a3d1fa60 0x55f5a3dab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 -- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 msgr2=0x55f5a3da6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5a3dac4a0 0x55f5a3da6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 -- 172.21.15.181:0/1903438724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a3c69280 con 0x55f5a3d1fec0 2020-11-16T22:30:09.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a7a7c700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 0x55f5a3da68c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb898000bc0 tx=0x7fb8980016c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:09.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a5277700 1 -- 172.21.15.181:0/1903438724 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898001b60 con 0x55f5a3d1fec0 2020-11-16T22:30:09.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a5277700 1 -- 172.21.15.181:0/1903438724 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89800f6e0 con 0x55f5a3d1fec0 2020-11-16T22:30:09.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5a3c47620 con 0x55f5a3d1fec0 2020-11-16T22:30:09.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8a5277700 1 -- 172.21.15.181:0/1903438724 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800fa80 con 0x55f5a3d1fec0 2020-11-16T22:30:09.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.425+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5a3dbd710 con 0x55f5a3d1fec0 2020-11-16T22:30:09.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a5277700 1 -- 172.21.15.181:0/1903438724 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb89801f070 con 0x55f5a3d1fec0 2020-11-16T22:30:09.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a5277700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb888053ce0 0x7fb8880561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb888053ce0 0x7fb8880561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a5277700 1 -- 172.21.15.181:0/1903438724 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb89806d520 con 0x55f5a3d1fec0 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x55f5a3dbde50 0x55f5a3da5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a827d700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb888053ce0 0x7fb8880561a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb890021040 tx=0x7fb89001c000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- osd_op(unknown.0.0:1 7.2 7:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x55f5a3d1f600 con 0x55f5a3dbde50 2020-11-16T22:30:09.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a8a7e700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x55f5a3dbde50 0x55f5a3da5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.429+0000 7fb8a8a7e700 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x55f5a3dbde50 0x55f5a3da5390 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8a8a7e700 1 -- 172.21.15.181:0/1903438724 <== osd.4 v2:172.21.15.175:6800/15329 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v49'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb8a0002700 con 0x55f5a3dbde50 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x55f5a3dbde50 msgr2=0x55f5a3da5390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x55f5a3dbde50 0x55f5a3da5390 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb888053ce0 msgr2=0x7fb8880561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb888053ce0 0x7fb8880561a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb890021040 tx=0x7fb89001c000).stop 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 msgr2=0x55f5a3da68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 --2- 172.21.15.181:0/1903438724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f5a3d1fec0 0x55f5a3da68c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb898000bc0 tx=0x7fb8980016c0).stop 2020-11-16T22:30:09.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8a8a7e700 1 -- 172.21.15.181:0/1903438724 reap_dead start 2020-11-16T22:30:09.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 shutdown_connections 2020-11-16T22:30:09.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 >> 172.21.15.181:0/1903438724 conn(0x55f5a3d203e0 msgr2=0x55f5a3d02980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:09.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 shutdown_connections 2020-11-16T22:30:09.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.433+0000 7fb8b6c77c40 1 -- 172.21.15.181:0/1903438724 wait complete. 2020-11-16T22:30:09.821 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:30:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.873+0000 7fbbfd4b5c40 1 Processor -- start 2020-11-16T22:30:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.873+0000 7fbbfd4b5c40 1 -- start start 2020-11-16T22:30:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 0x56362d6ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 0x56362d6b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362d6241f0 con 0x56362d6afcd0 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362d5f2be0 con 0x56362d6c5690 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56362d740240 con 0x56362d6b00b0 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39382/0 (socket says 172.21.15.181:39382) 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 -- 172.21.15.181:0/95045897 learned_addr learned my addr 172.21.15.181:0/95045897 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbeeabb700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 0x56362d6ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbef2bc700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 0x56362d6b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 -- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 msgr2=0x56362d6b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 0x56362d6b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 -- 172.21.15.181:0/95045897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 msgr2=0x56362d6ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 0x56362d6ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 -- 172.21.15.181:0/95045897 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56362d61c8a0 con 0x56362d6b00b0 2020-11-16T22:30:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbef2bc700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 0x56362d6b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:30:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbeeabb700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 0x56362d6ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fbbd800b690 tx=0x7fbbd8001680).ready entity=mon.2 client_cookie=77c210d16cb5d425 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:10.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbed2b8700 1 -- 172.21.15.181:0/95045897 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800d5a0 con 0x56362d6b00b0 2020-11-16T22:30:10.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbed2b8700 1 -- 172.21.15.181:0/95045897 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800d700 con 0x56362d6b00b0 2020-11-16T22:30:10.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbed2b8700 1 -- 172.21.15.181:0/95045897 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800daa0 con 0x56362d6b00b0 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 msgr2=0x56362d6cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fbbd800b690 tx=0x7fbbd8001680).stop 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/95045897 shutdown_connections 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6c5690 0x56362d6b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6afcd0 0x56362d6ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/95045897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6cea50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/95045897 >> 172.21.15.181:0/95045897 conn(0x56362d6cd1d0 msgr2=0x56362d75a010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:10.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/95045897 shutdown_connections 2020-11-16T22:30:10.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.877+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/95045897 wait complete. 2020-11-16T22:30:10.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 Processor -- start 2020-11-16T22:30:10.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 -- start start 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 0x56362d758fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6c5690 0x56362d6b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362d75e490 con 0x56362d6c5690 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362d7594b0 con 0x56362d6afcd0 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbfd4b5c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56362d76d8a0 con 0x56362d6b00b0 2020-11-16T22:30:10.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 0x56362d758fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 0x56362d758fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59776/0 (socket says 172.21.15.181:59776) 2020-11-16T22:30:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 -- 172.21.15.181:0/102062240 learned_addr learned my addr 172.21.15.181:0/102062240 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbef2bc700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6c5690 0x56362d6b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 -- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 msgr2=0x56362d6b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56362d6b00b0 0x56362d6b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 -- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6c5690 msgr2=0x56362d6b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56362d6c5690 0x56362d6b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.881+0000 7fbbeeabb700 1 -- 172.21.15.181:0/102062240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56362d618280 con 0x56362d6afcd0 2020-11-16T22:30:10.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbeeabb700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 0x56362d758fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbbe0010000 tx=0x7fbbe0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 -- 172.21.15.181:0/102062240 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe001e400 con 0x56362d6afcd0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 -- 172.21.15.181:0/102062240 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe001e560 con 0x56362d6afcd0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56362d5f6620 con 0x56362d6afcd0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 -- 172.21.15.181:0/102062240 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe001e8e0 con 0x56362d6afcd0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56362d75e990 con 0x56362d6afcd0 2020-11-16T22:30:10.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 -- 172.21.15.181:0/102062240 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbbe001ec90 con 0x56362d6afcd0 2020-11-16T22:30:10.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbbd0053ce0 0x7fbbd00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbe77fe700 1 -- 172.21.15.181:0/102062240 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbbe006f270 con 0x56362d6afcd0 2020-11-16T22:30:10.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbbd0053ce0 0x7fbbd00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x56362d73ad50 0x56362d73b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:10.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- osd_op(unknown.0.0:1 7.2 7:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x56362d73b640 con 0x56362d73ad50 2020-11-16T22:30:10.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbee2ba700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbbd0053ce0 0x7fbbd00561a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x56362d7602a0 tx=0x7fbbd8001680).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:10.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbef2bc700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x56362d73ad50 0x56362d73b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.885+0000 7fbbef2bc700 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x56362d73ad50 0x56362d73b150 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.889+0000 7fbbef2bc700 1 -- 172.21.15.181:0/102062240 <== osd.4 v2:172.21.15.175:6800/15329 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v49'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbbe800a770 con 0x56362d73ad50 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.889+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x56362d73ad50 msgr2=0x56362d73b150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.889+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x56362d73ad50 0x56362d73b150 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbbd0053ce0 msgr2=0x7fbbd00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbbd0053ce0 0x7fbbd00561a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x56362d7602a0 tx=0x7fbbd8001680).stop 2020-11-16T22:30:10.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 msgr2=0x56362d758fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:10.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 --2- 172.21.15.181:0/102062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56362d6afcd0 0x56362d758fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbbe0010000 tx=0x7fbbe0015000).stop 2020-11-16T22:30:11.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbef2bc700 1 -- 172.21.15.181:0/102062240 reap_dead start 2020-11-16T22:30:11.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 shutdown_connections 2020-11-16T22:30:11.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 >> 172.21.15.181:0/102062240 conn(0x56362d6cd1d0 msgr2=0x56362d6cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 shutdown_connections 2020-11-16T22:30:11.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:09.893+0000 7fbbfd4b5c40 1 -- 172.21.15.181:0/102062240 wait complete. 2020-11-16T22:30:11.087 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:30:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.137+0000 7f1aa01d7c40 1 Processor -- start 2020-11-16T22:30:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.137+0000 7f1aa01d7c40 1 -- start start 2020-11-16T22:30:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 0x55b119c61df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 0x55b119bde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119b341f0 con 0x55b119bc2760 2020-11-16T22:30:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119b02be0 con 0x55b119c63690 2020-11-16T22:30:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b119c48270 con 0x55b119c6d450 2020-11-16T22:30:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a91fde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a90fdc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 0x55b119bde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a91fde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39406/0 (socket says 172.21.15.181:39406) 2020-11-16T22:30:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a91fde700 1 -- 172.21.15.181:0/526471749 learned_addr learned my addr 172.21.15.181:0/526471749 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:11.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 0x55b119c61df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 -- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 msgr2=0x55b119bdad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 -- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 msgr2=0x55b119bde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 0x55b119bde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 -- 172.21.15.181:0/526471749 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b119b2c8a0 con 0x55b119bc2760 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a91fde700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a90fdc700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 0x55b119bde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a917dd700 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 0x55b119c61df0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f1a8400b5e0 tx=0x7f1a84001630).ready entity=mon.0 client_cookie=e414001ea9db6067 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a8b7fe700 1 -- 172.21.15.181:0/526471749 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400bdc0 con 0x55b119bc2760 2020-11-16T22:30:11.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a8b7fe700 1 -- 172.21.15.181:0/526471749 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8400e430 con 0x55b119bc2760 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1a8b7fe700 1 -- 172.21.15.181:0/526471749 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400e7b0 con 0x55b119bc2760 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/526471749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 msgr2=0x55b119c61df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 0x55b119c61df0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f1a8400b5e0 tx=0x7f1a84001630).stop 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/526471749 shutdown_connections 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bdad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119bc2760 0x55b119c61df0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/526471749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119c63690 0x55b119bde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/526471749 >> 172.21.15.181:0/526471749 conn(0x55b119bdd1d0 msgr2=0x55b119bdc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/526471749 shutdown_connections 2020-11-16T22:30:11.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/526471749 wait complete. 2020-11-16T22:30:11.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 Processor -- start 2020-11-16T22:30:11.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.141+0000 7f1aa01d7c40 1 -- start start 2020-11-16T22:30:11.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 0x55b119b85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119c63690 0x55b119c46750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bd74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119c7b270 con 0x55b119c63690 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119c7a250 con 0x55b119bc2760 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b119c7a4f0 con 0x55b119c6d450 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 0x55b119b85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a91fde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bd74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 0x55b119b85eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59798/0 (socket says 172.21.15.181:59798) 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 -- 172.21.15.181:0/39765006 learned_addr learned my addr 172.21.15.181:0/39765006 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a90fdc700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119c63690 0x55b119c46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 -- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 msgr2=0x55b119bd74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b119c6d450 0x55b119bd74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 -- 172.21.15.181:0/39765006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119c63690 msgr2=0x55b119c46750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b119c63690 0x55b119c46750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 -- 172.21.15.181:0/39765006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b119b28280 con 0x55b119bc2760 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a917dd700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 0x55b119b85eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f1a84000bc0 tx=0x55b119c65200).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a89ffb700 1 -- 172.21.15.181:0/39765006 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8401b650 con 0x55b119bc2760 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a89ffb700 1 -- 172.21.15.181:0/39765006 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8401b7b0 con 0x55b119bc2760 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b119b06620 con 0x55b119bc2760 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1a89ffb700 1 -- 172.21.15.181:0/39765006 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8401bb30 con 0x55b119bc2760 2020-11-16T22:30:11.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.145+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b119c7aa30 con 0x55b119bc2760 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a89ffb700 1 -- 172.21.15.181:0/39765006 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1a8402b070 con 0x55b119bc2760 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a89ffb700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a74053ce0 0x7f1a740561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a90fdc700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a74053ce0 0x7f1a740561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a89ffb700 1 -- 172.21.15.181:0/39765006 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a8406f3a0 con 0x55b119bc2760 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55b119c79d90 0x55b119c66b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a90fdc700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a74053ce0 0x7f1a740561a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1a7c00a940 tx=0x7f1a7c013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 7.5 7:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x55b119b863a0 con 0x55b119c79d90 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a91fde700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55b119c79d90 0x55b119c66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.149+0000 7f1a91fde700 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55b119c79d90 0x55b119c66b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1a91fde700 1 -- 172.21.15.181:0/39765006 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v49'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1a8c002b30 con 0x55b119c79d90 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55b119c79d90 msgr2=0x55b119c66b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55b119c79d90 0x55b119c66b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a74053ce0 msgr2=0x7f1a740561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a74053ce0 0x7f1a740561a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1a7c00a940 tx=0x7f1a7c013000).stop 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 msgr2=0x55b119b85eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 --2- 172.21.15.181:0/39765006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b119bc2760 0x55b119b85eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f1a84000bc0 tx=0x55b119c65200).stop 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1a91fde700 1 -- 172.21.15.181:0/39765006 reap_dead start 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 shutdown_connections 2020-11-16T22:30:11.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 >> 172.21.15.181:0/39765006 conn(0x55b119bdd1d0 msgr2=0x55b119c47460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 shutdown_connections 2020-11-16T22:30:11.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.153+0000 7f1aa01d7c40 1 -- 172.21.15.181:0/39765006 wait complete. 2020-11-16T22:30:11.339 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:30:11.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.421+0000 7f54819dbc40 1 Processor -- start 2020-11-16T22:30:11.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.421+0000 7f54819dbc40 1 -- start start 2020-11-16T22:30:11.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208b6e0 0x558292016630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208bac0 0x558292088160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558291f70190 con 0x55829208bac0 2020-11-16T22:30:11.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558291f711f0 con 0x55829208b6e0 2020-11-16T22:30:11.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54819dbc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558291f3fbe0 con 0x558292016c30 2020-11-16T22:30:11.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54727e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208b6e0 0x558292016630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39422/0 (socket says 172.21.15.181:39422) 2020-11-16T22:30:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 -- 172.21.15.181:0/3435775314 learned_addr learned my addr 172.21.15.181:0/3435775314 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54737e2700 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208bac0 0x558292088160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 -- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208b6e0 msgr2=0x558292016630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208b6e0 0x558292016630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 -- 172.21.15.181:0/3435775314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208bac0 msgr2=0x558292088160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208bac0 0x558292088160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 -- 172.21.15.181:0/3435775314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558291f65280 con 0x558292016c30 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f5472fe1700 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f546800b5b0 tx=0x7f5468002870).ready entity=mon.2 client_cookie=f4dac24a38ac1655 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54717de700 1 -- 172.21.15.181:0/3435775314 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546800d490 con 0x558292016c30 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54717de700 1 -- 172.21.15.181:0/3435775314 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546800d5f0 con 0x558292016c30 2020-11-16T22:30:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.425+0000 7f54717de700 1 -- 172.21.15.181:0/3435775314 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546800d970 con 0x558292016c30 2020-11-16T22:30:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 msgr2=0x5582920a8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f546800b5b0 tx=0x7f5468002870).stop 2020-11-16T22:30:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- 172.21.15.181:0/3435775314 shutdown_connections 2020-11-16T22:30:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208bac0 0x558292088160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920a8860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3435775314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208b6e0 0x558292016630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- 172.21.15.181:0/3435775314 >> 172.21.15.181:0/3435775314 conn(0x5582920190d0 msgr2=0x558291ffcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- 172.21.15.181:0/3435775314 shutdown_connections 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- 172.21.15.181:0/3435775314 wait complete. 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 Processor -- start 2020-11-16T22:30:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- start start 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208b6e0 0x5582920875b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 0x558292087aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829208d960 con 0x55829208b6e0 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829208dc00 con 0x55829208bac0 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54819dbc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829208dea0 con 0x558292016c30 2020-11-16T22:30:11.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 0x558292087aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f5472fe1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 0x558292087aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59822/0 (socket says 172.21.15.181:59822) 2020-11-16T22:30:11.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 -- 172.21.15.181:0/3941650845 learned_addr learned my addr 172.21.15.181:0/3941650845 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:11.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54727e0700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208b6e0 0x5582920875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 -- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 msgr2=0x5582920b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558292016c30 0x5582920b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 -- 172.21.15.181:0/3941650845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208b6e0 msgr2=0x5582920875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.429+0000 7f54737e2700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829208b6e0 0x5582920875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54737e2700 1 -- 172.21.15.181:0/3941650845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558291f43620 con 0x55829208bac0 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54737e2700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 0x558292087aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f546c0014b0 tx=0x7f546c0014e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 -- 172.21.15.181:0/3941650845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c010580 con 0x55829208bac0 2020-11-16T22:30:11.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 -- 172.21.15.181:0/3941650845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0106e0 con 0x55829208bac0 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 -- 172.21.15.181:0/3941650845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c010a80 con 0x55829208bac0 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55829208e140 con 0x55829208bac0 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55829208d620 con 0x55829208bac0 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 -- 172.21.15.181:0/3941650845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f546c019030 con 0x55829208bac0 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5458053ce0 0x7f54580561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f5472fe1700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5458053ce0 0x7f54580561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.433+0000 7f54637fe700 1 -- 172.21.15.181:0/3941650845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f546c06c470 con 0x55829208bac0 2020-11-16T22:30:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x558292088980 0x5582920a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f5472fe1700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5458053ce0 0x7f54580561a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f5468000bc0 tx=0x7f5468002870).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- osd_op(unknown.0.0:1 7.1 7:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x558291ffc450 con 0x558292088980 2020-11-16T22:30:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f54727e0700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x558292088980 0x5582920a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f54727e0700 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x558292088980 0x5582920a7360 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.437+0000 7f54727e0700 1 -- 172.21.15.181:0/3941650845 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v49'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5464000b80 con 0x558292088980 2020-11-16T22:30:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x558292088980 msgr2=0x5582920a7360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x558292088980 0x5582920a7360 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5458053ce0 msgr2=0x7f54580561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5458053ce0 0x7f54580561a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f5468000bc0 tx=0x7f5468002870).stop 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 msgr2=0x558292087aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 --2- 172.21.15.181:0/3941650845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55829208bac0 0x558292087aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f546c0014b0 tx=0x7f546c0014e0).stop 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54737e2700 1 -- 172.21.15.181:0/3941650845 reap_dead start 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 shutdown_connections 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 >> 172.21.15.181:0/3941650845 conn(0x5582920190d0 msgr2=0x5582920aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 shutdown_connections 2020-11-16T22:30:11.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.441+0000 7f54819dbc40 1 -- 172.21.15.181:0/3941650845 wait complete. 2020-11-16T22:30:11.425 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:30:11.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 Processor -- start 2020-11-16T22:30:11.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 -- start start 2020-11-16T22:30:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c5d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04c1e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d04bca190 con 0x556d04ce8b70 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d04bcb1f0 con 0x556d04d06d90 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d04b99be0 con 0x556d04ce8f50 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efbf4d62700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efbf5d64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04c1e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efbf4d62700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39446/0 (socket says 172.21.15.181:39446) 2020-11-16T22:30:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efbf4d62700 1 -- 172.21.15.181:0/1477497262 learned_addr learned my addr 172.21.15.181:0/1477497262 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.513+0000 7efbf5563700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c5d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 -- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 msgr2=0x556d04c709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 -- 172.21.15.181:0/1477497262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 msgr2=0x556d04c5d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c5d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 -- 172.21.15.181:0/1477497262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d04bbf280 con 0x556d04d06d90 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5563700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c5d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf4d62700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04c1e960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7efbf0012000 tx=0x7efbf0015000).ready entity=mon.1 client_cookie=6253b763430c58b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbef7fe700 1 -- 172.21.15.181:0/1477497262 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00123f0 con 0x556d04d06d90 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbef7fe700 1 -- 172.21.15.181:0/1477497262 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf000ae90 con 0x556d04d06d90 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbef7fe700 1 -- 172.21.15.181:0/1477497262 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00129f0 con 0x556d04d06d90 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 msgr2=0x556d04c1e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04c1e960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7efbf0012000 tx=0x7efbf0015000).stop 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- 172.21.15.181:0/1477497262 shutdown_connections 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04c1e960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c5d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/1477497262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- 172.21.15.181:0/1477497262 >> 172.21.15.181:0/1477497262 conn(0x556d04c76400 msgr2=0x556d04c6e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- 172.21.15.181:0/1477497262 shutdown_connections 2020-11-16T22:30:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- 172.21.15.181:0/1477497262 wait complete. 2020-11-16T22:30:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 Processor -- start 2020-11-16T22:30:15.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- start start 2020-11-16T22:30:15.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04ce7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04ce19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d04ce7770 con 0x556d04ce8b70 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d04d12e10 con 0x556d04d06d90 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efc03f5dc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d04c790e0 con 0x556d04ce8f50 2020-11-16T22:30:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04ce19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf4d62700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04ce7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04ce19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59842/0 (socket says 172.21.15.181:59842) 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5d64700 1 -- 172.21.15.181:0/2924249088 learned_addr learned my addr 172.21.15.181:0/2924249088 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf4d62700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04ce7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39454/0 (socket says 172.21.15.181:39454) 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.517+0000 7efbf5563700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:15.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 -- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 msgr2=0x556d04ce19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d04d06d90 0x556d04ce19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 -- 172.21.15.181:0/2924249088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 msgr2=0x556d04c732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d04ce8b70 0x556d04c732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 -- 172.21.15.181:0/2924249088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d04b9d620 con 0x556d04ce8f50 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbf4d62700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04ce7280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x556d04bab340 tx=0x7efbe80018b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbedffb700 1 -- 172.21.15.181:0/2924249088 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8010710 con 0x556d04ce8f50 2020-11-16T22:30:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbedffb700 1 -- 172.21.15.181:0/2924249088 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe8010870 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d04d13070 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbedffb700 1 -- 172.21.15.181:0/2924249088 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8010bf0 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d04d135b0 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.521+0000 7efbedffb700 1 -- 172.21.15.181:0/2924249088 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efbe8011410 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbedffb700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd8053d10 0x7efbd80561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbedffb700 1 -- 172.21.15.181:0/2924249088 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efbe806cce0 con 0x556d04ce8f50 2020-11-16T22:30:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbf5563700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd8053d10 0x7efbd80561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x556d04d13a50 0x556d04d04420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:15.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 7.5 7:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x556d04ce7fa0 con 0x556d04d13a50 2020-11-16T22:30:15.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbf5563700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd8053d10 0x7efbd80561d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7efbe0006c90 tx=0x7efbe0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:15.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbf5d64700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x556d04d13a50 0x556d04d04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:15.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.525+0000 7efbf5d64700 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x556d04d13a50 0x556d04d04420 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efbf5d64700 1 -- 172.21.15.181:0/2924249088 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v49'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efbf0025040 con 0x556d04d13a50 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x556d04d13a50 msgr2=0x556d04d04420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x556d04d13a50 0x556d04d04420 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd8053d10 msgr2=0x7efbd80561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd8053d10 0x7efbd80561d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7efbe0006c90 tx=0x7efbe0008040).stop 2020-11-16T22:30:15.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 msgr2=0x556d04ce7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:15.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 --2- 172.21.15.181:0/2924249088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556d04ce8f50 0x556d04ce7280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x556d04bab340 tx=0x7efbe80018b0).stop 2020-11-16T22:30:15.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efbf5d64700 1 -- 172.21.15.181:0/2924249088 reap_dead start 2020-11-16T22:30:15.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 shutdown_connections 2020-11-16T22:30:16.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 >> 172.21.15.181:0/2924249088 conn(0x556d04c76400 msgr2=0x556d04c1e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:16.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 shutdown_connections 2020-11-16T22:30:16.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:11.529+0000 7efc03f5dc40 1 -- 172.21.15.181:0/2924249088 wait complete. 2020-11-16T22:30:16.043 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:30:16.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 Processor -- start 2020-11-16T22:30:16.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 -- start start 2020-11-16T22:30:16.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x56114362b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611435f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x5611435f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114359d190 con 0x5611436d5880 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114359e1f0 con 0x561143642da0 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56114356cbe0 con 0x5611436d54a0 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5ea8ccd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x56114362b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5ea84cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611435f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5ea8ccd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x56114362b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59856/0 (socket says 172.21.15.181:59856) 2020-11-16T22:30:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.093+0000 7f5ea8ccd700 1 -- 172.21.15.181:0/842010355 learned_addr learned my addr 172.21.15.181:0/842010355 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea94ce700 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x5611435f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 -- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 msgr2=0x56114362b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x56114362b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 -- 172.21.15.181:0/842010355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 msgr2=0x5611435f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x5611435f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 -- 172.21.15.181:0/842010355 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561143592280 con 0x5611436d54a0 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611435f78e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5e9800a970 tx=0x7f5e98013000).ready entity=mon.2 client_cookie=619dbef970544b59 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea74ca700 1 -- 172.21.15.181:0/842010355 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98008bd0 con 0x5611436d54a0 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea74ca700 1 -- 172.21.15.181:0/842010355 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98008d30 con 0x5611436d54a0 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5ea74ca700 1 -- 172.21.15.181:0/842010355 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98008ea0 con 0x5611436d54a0 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 msgr2=0x5611435f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611435f78e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5e9800a970 tx=0x7f5e98013000).stop 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/842010355 shutdown_connections 2020-11-16T22:30:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x5611435f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x56114362b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/842010355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611435f78e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/842010355 >> 172.21.15.181:0/842010355 conn(0x5611436493e0 msgr2=0x5611436425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:16.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/842010355 shutdown_connections 2020-11-16T22:30:16.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.097+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/842010355 wait complete. 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 Processor -- start 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 -- start start 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x5611436d3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611436d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x56114362fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611436e6e10 con 0x5611436d5880 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611436e5f60 con 0x561143642da0 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5eb76c7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56114362fa20 con 0x5611436d54a0 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x5611436d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea84cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611436d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x5611436d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59862/0 (socket says 172.21.15.181:59862) 2020-11-16T22:30:16.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea84cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611436d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39474/0 (socket says 172.21.15.181:39474) 2020-11-16T22:30:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 -- 172.21.15.181:0/102429336 learned_addr learned my addr 172.21.15.181:0/102429336 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea94ce700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x56114362fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 -- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 msgr2=0x5611436d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611436d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 -- 172.21.15.181:0/102429336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 msgr2=0x56114362fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611436d5880 0x56114362fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 -- 172.21.15.181:0/102429336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561143570620 con 0x561143642da0 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5611436d54a0 0x5611436d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.101+0000 7f5ea8ccd700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x5611436d3690 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f5e9000c5a0 tx=0x7f5e90002830).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 -- 172.21.15.181:0/102429336 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90002c20 con 0x561143642da0 2020-11-16T22:30:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 -- 172.21.15.181:0/102429336 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90002d80 con 0x561143642da0 2020-11-16T22:30:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5611436484c0 con 0x561143642da0 2020-11-16T22:30:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 -- 172.21.15.181:0/102429336 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90003260 con 0x561143642da0 2020-11-16T22:30:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611436e6430 con 0x561143642da0 2020-11-16T22:30:16.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 -- 172.21.15.181:0/102429336 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5e900035d0 con 0x561143642da0 2020-11-16T22:30:16.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e88053ce0 0x7f5e880561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e88053ce0 0x7f5e880561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.105+0000 7f5ea5cc7700 1 -- 172.21.15.181:0/102429336 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e9000b070 con 0x561143642da0 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.109+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x56114362ed50 0x5611436ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.109+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 7.5 7:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e49) v8 -- 0x56114364acb0 con 0x56114362ed50 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.109+0000 7f5ea84cc700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e88053ce0 0x7f5e880561a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x5611436479c0 tx=0x7f5e9801c000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.109+0000 7f5ea94ce700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x56114362ed50 0x5611436ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.109+0000 7f5ea94ce700 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x56114362ed50 0x5611436ccfb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:16.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5ea94ce700 1 -- 172.21.15.181:0/102429336 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v49'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5ea0006210 con 0x56114362ed50 2020-11-16T22:30:16.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x56114362ed50 msgr2=0x5611436ccfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:16.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x56114362ed50 0x5611436ccfb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:16.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e88053ce0 msgr2=0x7f5e880561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e88053ce0 0x7f5e880561a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x5611436479c0 tx=0x7f5e9801c000).stop 2020-11-16T22:30:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 msgr2=0x5611436d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 --2- 172.21.15.181:0/102429336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561143642da0 0x5611436d3690 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f5e9000c5a0 tx=0x7f5e90002830).stop 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5ea94ce700 1 -- 172.21.15.181:0/102429336 reap_dead start 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 shutdown_connections 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 >> 172.21.15.181:0/102429336 conn(0x5611436493e0 msgr2=0x5611436425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 shutdown_connections 2020-11-16T22:30:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:16.113+0000 7f5eb76c7c40 1 -- 172.21.15.181:0/102429336 wait complete. 2020-11-16T22:30:17.104 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:17.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 Processor -- start 2020-11-16T22:30:17.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 -- start start 2020-11-16T22:30:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aec156a90 0x7f3aec061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3aec15f3c0 0x7f3aec0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec0cccd0 con 0x7f3aec0615f0 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0ccf70 con 0x7f3aec15f3c0 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3af0faa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec15e030 con 0x7f3aec156a90 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aec156a90 0x7f3aec061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aec156a90 0x7f3aec061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59872/0 (socket says 172.21.15.181:59872) 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeaffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3aec0615f0 0x7f3aec15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45946/0 (socket says 172.21.15.181:45946) 2020-11-16T22:30:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/1566971799 learned_addr learned my addr 172.21.15.181:0/1566971799 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aebfff700 1 --2- 172.21.15.181:0/1566971799 >> v2:172.21.15.175:3300/0 conn(0x7f3aec15f3c0 0x7f3aec0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/1566971799 >> v2:172.21.15.175:3300/0 conn(0x7f3aec15f3c0 msgr2=0x7f3aec0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 --2- 172.21.15.181:0/1566971799 >> v2:172.21.15.175:3300/0 conn(0x7f3aec15f3c0 0x7f3aec0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/1566971799 >> v1:172.21.15.181:6789/0 conn(0x7f3aec0615f0 legacy=0x7f3aec15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/1566971799 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec15e2d0 con 0x7f3aec156a90 2020-11-16T22:30:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aeb7fe700 1 --2- 172.21.15.181:0/1566971799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec156a90 0x7f3aec061100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f3aec0675c0 tx=0x7f3adc006eb0).ready entity=mon.1 client_cookie=12e317422a8bd41a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aea7fc700 1 -- 172.21.15.181:0/1566971799 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0098d0 con 0x7f3aec156a90 2020-11-16T22:30:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.257+0000 7f3aea7fc700 1 -- 172.21.15.181:0/1566971799 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc009a30 con 0x7f3aec156a90 2020-11-16T22:30:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3aea7fc700 1 -- 172.21.15.181:0/1566971799 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc009db0 con 0x7f3aec156a90 2020-11-16T22:30:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- 172.21.15.181:0/1566971799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec156a90 msgr2=0x7f3aec061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 --2- 172.21.15.181:0/1566971799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec156a90 0x7f3aec061100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f3aec0675c0 tx=0x7f3adc006eb0).stop 2020-11-16T22:30:17.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- 172.21.15.181:0/1566971799 shutdown_connections 2020-11-16T22:30:17.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 --2- 172.21.15.181:0/1566971799 >> v2:172.21.15.175:3300/0 conn(0x7f3aec15f3c0 0x7f3aec0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 --2- 172.21.15.181:0/1566971799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec156a90 0x7f3aec061100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- 172.21.15.181:0/1566971799 >> 172.21.15.181:0/1566971799 conn(0x7f3aec0cf630 msgr2=0x7f3aec0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:17.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- 172.21.15.181:0/1566971799 shutdown_connections 2020-11-16T22:30:17.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- 172.21.15.181:0/1566971799 wait complete. 2020-11-16T22:30:17.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 Processor -- start 2020-11-16T22:30:17.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.261+0000 7f3af0faa700 1 -- start start 2020-11-16T22:30:17.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 0x7f3aec0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 0x7f3aec0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0cd7f0 con 0x7f3aec156a90 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec1713d0 con 0x7f3aec15f3c0 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0c8a40 con 0x7f3aec0615f0 2020-11-16T22:30:17.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39488/0 (socket says 172.21.15.181:39488) 2020-11-16T22:30:17.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 0x7f3aec0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/3693341190 learned_addr learned my addr 172.21.15.181:0/3693341190 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeaffd700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 0x7f3aec0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 msgr2=0x7f3aec0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 0x7f3aec0cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/3693341190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 msgr2=0x7f3aec0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 0x7f3aec0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 -- 172.21.15.181:0/3693341190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0cda20 con 0x7f3aec0615f0 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeaffd700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 0x7f3aec0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aebfff700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 0x7f3aec0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3aeb7fe700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f3adc006fd0 tx=0x7f3adc017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc010dc0 con 0x7f3aec0615f0 2020-11-16T22:30:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec0cdcc0 con 0x7f3aec0615f0 2020-11-16T22:30:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc018400 con 0x7f3aec0615f0 2020-11-16T22:30:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc018780 con 0x7f3aec0615f0 2020-11-16T22:30:17.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec0ceb90 con 0x7f3aec0615f0 2020-11-16T22:30:17.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.265+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec066570 con 0x7f3aec0615f0 2020-11-16T22:30:17.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.269+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3adc018af0 con 0x7f3aec0615f0 2020-11-16T22:30:17.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.269+0000 7f3acb7fe700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 0x7f3ac4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:17.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.269+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3adc06f9b0 con 0x7f3aec0615f0 2020-11-16T22:30:17.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.269+0000 7f3aeaffd700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 0x7f3ac4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:17.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.269+0000 7f3aeaffd700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 0x7f3ac4057520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f3aec0d1780 tx=0x7f3ae0007120).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:17.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.277+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3aec066570 con 0x7f3aec0615f0 2020-11-16T22:30:17.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.481+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aec066570 con 0x7f3aec0615f0 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3acb7fe700 1 -- 172.21.15.181:0/3693341190 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4663 (secure 0 0 0) 0x7f3aec066570 con 0x7f3aec0615f0 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 msgr2=0x7f3ac4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 0x7f3ac4057520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f3aec0d1780 tx=0x7f3ae0007120).stop 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 msgr2=0x7f3aec170ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f3adc006fd0 tx=0x7f3adc017000).stop 2020-11-16T22:30:17.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 shutdown_connections 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3ac4055060 0x7f3ac4057520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec15f3c0 0x7f3aec0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0615f0 0x7f3aec170ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 --2- 172.21.15.181:0/3693341190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aec156a90 0x7f3aec0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 >> 172.21.15.181:0/3693341190 conn(0x7f3aec0cf630 msgr2=0x7f3aec065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 shutdown_connections 2020-11-16T22:30:17.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:17.485+0000 7f3af0faa700 1 -- 172.21.15.181:0/3693341190 wait complete. 2020-11-16T22:30:17.863 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 54 2020-11-16T22:30:17.863 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:17.863 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:08.850935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:18.066 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub start /d1/d2/d3 recursive force 2020-11-16T22:30:18.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.221+0000 7f1dfe477700 1 Processor -- start 2020-11-16T22:30:18.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- start start 2020-11-16T22:30:18.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1df8145a60 0x7f1df8145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1df8061a30 0x7f1df80814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80cb950 con 0x7f1df8061a30 2020-11-16T22:30:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80cbbf0 con 0x7f1df8145a60 2020-11-16T22:30:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80cbe90 con 0x7f1df80615b0 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1df8145a60 0x7f1df8145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39494/0 (socket says 172.21.15.181:39494) 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 -- 172.21.15.181:0/2791837677 learned_addr learned my addr 172.21.15.181:0/2791837677 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfca14700 1 --2- 172.21.15.181:0/2791837677 >> v2:172.21.15.175:3300/0 conn(0x7f1df8061a30 0x7f1df80814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 -- 172.21.15.181:0/2791837677 >> v2:172.21.15.181:3300/0 conn(0x7f1df8145a60 msgr2=0x7f1df8145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2791837677 >> v2:172.21.15.181:3300/0 conn(0x7f1df8145a60 0x7f1df8145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 -- 172.21.15.181:0/2791837677 >> v2:172.21.15.175:3300/0 conn(0x7f1df8061a30 msgr2=0x7f1df80814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2791837677 >> v2:172.21.15.175:3300/0 conn(0x7f1df8061a30 0x7f1df80814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 -- 172.21.15.181:0/2791837677 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80cc130 con 0x7f1df80615b0 2020-11-16T22:30:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2791837677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f1de4006fd0 tx=0x7f1de4006eb0).ready entity=mon.2 client_cookie=752d400e06dd6e7c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:18.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df6ffd700 1 -- 172.21.15.181:0/2791837677 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40096e0 con 0x7f1df80615b0 2020-11-16T22:30:18.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df6ffd700 1 -- 172.21.15.181:0/2791837677 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de4009840 con 0x7f1df80615b0 2020-11-16T22:30:18.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1df6ffd700 1 -- 172.21.15.181:0/2791837677 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4009bc0 con 0x7f1df80615b0 2020-11-16T22:30:18.625 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:30:18.280+0000 7f642ec30700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1) n(v4 rc2020-11-16T22:29:48.195730+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@5},l=5782 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5569a01c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:29:48.195730+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:30:18.625 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:30:18.280+0000 7f6434c3c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1) n(v3 rc2020-11-16T22:29:48.195730+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5782=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5569a01c6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:29:48.195730+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:30:18.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- 172.21.15.181:0/2791837677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 msgr2=0x7f1df8080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2791837677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f1de4006fd0 tx=0x7f1de4006eb0).stop 2020-11-16T22:30:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- 172.21.15.181:0/2791837677 shutdown_connections 2020-11-16T22:30:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2791837677 >> v2:172.21.15.175:3300/0 conn(0x7f1df8061a30 0x7f1df80814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2791837677 >> v2:172.21.15.181:3300/0 conn(0x7f1df8145a60 0x7f1df8145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2791837677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df8080fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- 172.21.15.181:0/2791837677 >> 172.21.15.181:0/2791837677 conn(0x7f1df80d68e0 msgr2=0x7f1df80d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- 172.21.15.181:0/2791837677 shutdown_connections 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.225+0000 7f1dfe477700 1 -- 172.21.15.181:0/2791837677 wait complete. 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 Processor -- start 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 -- start start 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1df8061a30 0x7f1df80d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 0x7f1df80d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df81707c0 con 0x7f1df8145a60 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80d2130 con 0x7f1df8061a30 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1dfe477700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80d1e90 con 0x7f1df80615b0 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39500/0 (socket says 172.21.15.181:39500) 2020-11-16T22:30:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1df7fff700 1 -- 172.21.15.181:0/2483738465 learned_addr learned my addr 172.21.15.181:0/2483738465 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.229+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1df8061a30 0x7f1df80d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1dfca14700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 0x7f1df80d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 -- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1df8061a30 msgr2=0x7f1df80d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1df8061a30 0x7f1df80d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 -- 172.21.15.181:0/2483738465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 msgr2=0x7f1df80d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 0x7f1df80d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 -- 172.21.15.181:0/2483738465 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8170a20 con 0x7f1df80615b0 2020-11-16T22:30:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1dfca14700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 0x7f1df80d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f1de800eba0 tx=0x7f1de8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1deffff700 1 -- 172.21.15.181:0/2483738465 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800c9b0 con 0x7f1df80615b0 2020-11-16T22:30:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1deffff700 1 -- 172.21.15.181:0/2483738465 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800cb10 con 0x7f1df80615b0 2020-11-16T22:30:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1deffff700 1 -- 172.21.15.181:0/2483738465 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800ce90 con 0x7f1df80615b0 2020-11-16T22:30:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df8170cc0 con 0x7f1df80615b0 2020-11-16T22:30:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80d16a0 con 0x7f1df80615b0 2020-11-16T22:30:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.233+0000 7f1deffff700 1 -- 172.21.15.181:0/2483738465 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1de8012600 con 0x7f1df80615b0 2020-11-16T22:30:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1deffff700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 0x7f1dcc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 0x7f1dcc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1deffff700 1 -- 172.21.15.181:0/2483738465 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1de806e260 con 0x7f1df80615b0 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 0x7f1dcc057520 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1de400b1b0 tx=0x7f1de4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 Processor -- start 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 -- start start 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1c665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c29520 0x1c83d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aba560 con 0x1c29520 2020-11-16T22:30:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b6e9e0 con 0x19a2c60 2020-11-16T22:30:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a2e7a0 con 0x1a10860 2020-11-16T22:30:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39508/0 (socket says 172.21.15.181:39508) 2020-11-16T22:30:19.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df7fff700 1 -- 172.21.15.181:0/956013507 learned_addr learned my addr 172.21.15.181:0/956013507 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:19.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1df77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1c665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.237+0000 7f1dfca14700 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c29520 0x1c83d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 -- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 msgr2=0x1c665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1c665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 -- 172.21.15.181:0/956013507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c29520 msgr2=0x1c83d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c29520 0x1c83d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 -- 172.21.15.181:0/956013507 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a29650 con 0x1a10860 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1df7fff700 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f1de8047c80 tx=0x7f1de8014000).ready entity=mon.2 client_cookie=39738a3d3b242c82 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:19.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1ded7fa700 1 -- 172.21.15.181:0/956013507 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8019ec0 con 0x1a10860 2020-11-16T22:30:19.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1ded7fa700 1 -- 172.21.15.181:0/956013507 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de806c3c0 con 0x1a10860 2020-11-16T22:30:19.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1ded7fa700 1 -- 172.21.15.181:0/956013507 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de806c530 con 0x1a10860 2020-11-16T22:30:19.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 msgr2=0x1c27d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:19.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f1de8047c80 tx=0x7f1de8014000).stop 2020-11-16T22:30:19.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- 172.21.15.181:0/956013507 shutdown_connections 2020-11-16T22:30:19.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c29520 0x1c83d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c27d80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/956013507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1c665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- 172.21.15.181:0/956013507 >> 172.21.15.181:0/956013507 conn(0x1bdeef0 msgr2=0x1c83630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- 172.21.15.181:0/956013507 shutdown_connections 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- 172.21.15.181:0/956013507 wait complete. 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 -- start start 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.241+0000 7f1e0cf08740 1 Processor -- start 2020-11-16T22:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1c2ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 0x1c2f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b5b790 con 0x1a10860 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b5b8d0 con 0x1c29520 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b5cb40 con 0x19a2c60 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 10 client.0 ll_register_callbacks cb 0x1b51b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dfca14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 0x1c2f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39514/0 (socket says 172.21.15.181:39514) 2020-11-16T22:30:19.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dfca14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 0x1c2f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59906/0 (socket says 172.21.15.181:59906) 2020-11-16T22:30:19.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 -- 172.21.15.181:0/1482410974 learned_addr learned my addr 172.21.15.181:0/1482410974 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:19.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df77fe700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1c2ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 -- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 msgr2=0x1c2f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 0x1c2f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 -- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 msgr2=0x1c2ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1c2ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 -- 172.21.15.181:0/1482410974 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b4c330 con 0x19a2c60 2020-11-16T22:30:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df77fe700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1c2ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:19.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df7fff700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f1de80444b0 tx=0x7f1de801e070).ready entity=mon.2 client_cookie=e60893a7e20d2302 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:19.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8019e40 con 0x19a2c60 2020-11-16T22:30:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1de8046f70 con 0x19a2c60 2020-11-16T22:30:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c23ec0 con 0x19a2c60 2020-11-16T22:30:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de806c970 con 0x19a2c60 2020-11-16T22:30:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 10 client.5576 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x1982080 con 0x19a2c60 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c23ec0 con 0x19a2c60 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1de806cad0 con 0x19a2c60 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 10 client.5576 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 10 client.5576 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 20 client.5576 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c7f5a0 0x1c2b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 4 client.5576 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1a4cd00 con 0x1c7f5a0 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df77fe700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c7f5a0 0x1c2b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1dca7fc700 10 client.5576 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.245+0000 7f1df77fe700 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c7f5a0 0x1c2b940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=47c2d7c7f2bf0805 server_cookie=ff45d5988680e136 in_seq=0 out_seq=0 2020-11-16T22:30:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1dca7fc700 1 -- 172.21.15.181:0/1482410974 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1a4cd00 con 0x1c7f5a0 2020-11-16T22:30:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1dca7fc700 10 client.5576 handle_command_reply: tid=0 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 client.5576 shutdown 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 msgr2=0x1c287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f1de80444b0 tx=0x7f1de801e070).stop 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 shutdown_connections 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c7f5a0 0x1c2b940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c29520 0x1c2f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19a2c60 0x1c287a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1dca7fc700 0 client.5576 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/1482410974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1c2ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 >> 172.21.15.181:0/1482410974 conn(0x1bdeef0 msgr2=0x1c666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 shutdown_connections 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 1 -- 172.21.15.181:0/1482410974 wait complete. 2020-11-16T22:30:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.249+0000 7f1e0cf08740 20 client.5576 trim_cache size 0 max 16384 2020-11-16T22:30:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 Processor -- start 2020-11-16T22:30:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 -- start start 2020-11-16T22:30:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 0x1c26df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b33ef0 con 0x19a2c60 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c8f410 con 0x1bdeef0 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c9e6f0 con 0x1a10860 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1dfca14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59912/0 (socket says 172.21.15.181:59912) 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 -- 172.21.15.181:0/2316425768 learned_addr learned my addr 172.21.15.181:0/2316425768 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 0x1c26df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 -- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 msgr2=0x1c2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c2ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 -- 172.21.15.181:0/2316425768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 msgr2=0x1c26df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 0x1c26df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 -- 172.21.15.181:0/2316425768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40081d0 con 0x1bdeef0 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df77fe700 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 0x1c26df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1df7fff700 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1de8006f70 tx=0x7f1de8014040).ready entity=mon.1 client_cookie=98e0e30645305bbd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1dedffb700 1 -- 172.21.15.181:0/2316425768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de806e0d0 con 0x1bdeef0 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1dedffb700 1 -- 172.21.15.181:0/2316425768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8046f70 con 0x1bdeef0 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.269+0000 7f1dedffb700 1 -- 172.21.15.181:0/2316425768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de806cb80 con 0x1bdeef0 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 msgr2=0x1c2ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1de8006f70 tx=0x7f1de8014040).stop 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/2316425768 shutdown_connections 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a10860 0x1c2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdeef0 0x1c2ccb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/2316425768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19a2c60 0x1c26df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/2316425768 >> 172.21.15.181:0/2316425768 conn(0x7f1dc4002320 msgr2=0x1c287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/2316425768 shutdown_connections 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/2316425768 wait complete. 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- start start 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 Processor -- start 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1cc9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdeef0 0x1c30a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c27ab0 con 0x1a10860 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce46a0 con 0x19a2c60 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ce4940 con 0x1bdeef0 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 10 client.0 ll_register_callbacks cb 0x1c68820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59918/0 (socket says 172.21.15.181:59918) 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 -- 172.21.15.181:0/3181024664 learned_addr learned my addr 172.21.15.181:0/3181024664 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1dfca14700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdeef0 0x1c30a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df77fe700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1cc9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 -- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdeef0 msgr2=0x1c30a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdeef0 0x1c30a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 -- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 msgr2=0x1cc9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1cc9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 -- 172.21.15.181:0/3181024664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19c07d0 con 0x19a2c60 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df77fe700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1cc9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1df7fff700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1de8043040 tx=0x7f1de8027000).ready entity=mon.1 client_cookie=7cbd454f692f7fad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1dca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de806e0d0 con 0x19a2c60 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1de806c3c0 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1de8012aa0 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de8047d80 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 10 client.5968 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.273+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x19cbeb0 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de806c3c0 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1de8046670 con 0x19a2c60 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 10 client.5968 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 10 client.5968 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 20 client.5968 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c29520 0x1c2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 4 client.5968 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:30:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x1a54000 con 0x1c29520 2020-11-16T22:30:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1df77fe700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c29520 0x1c2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1df77fe700 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c29520 0x1c2a4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6151859558ed8823 server_cookie=5281882ca2217d97 in_seq=0 out_seq=0 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1dca7fc700 10 client.5968 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1dca7fc700 1 -- 172.21.15.181:0/3181024664 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1a54000 con 0x1c29520 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1dca7fc700 10 client.5968 handle_command_reply: tid=0 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 1 client.5968 shutdown 2020-11-16T22:30:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 msgr2=0x1ce3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1de8043040 tx=0x7f1de8027000).stop 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 shutdown_connections 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x1c29520 0x1c2a4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdeef0 0x1c30a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19a2c60 0x1ce3e90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 --2- 172.21.15.181:0/3181024664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a10860 0x1cc9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 >> 172.21.15.181:0/3181024664 conn(0x7f1dc4002320 msgr2=0x1c28720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dca7fc700 0 client.5968 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 shutdown_connections 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 1 -- 172.21.15.181:0/3181024664 wait complete. 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1e0cf08740 20 client.5968 trim_cache size 0 max 16384 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 msgr2=0x7f1dcc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 0x7f1dcc057520 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1de400b1b0 tx=0x7f1de4015000).stop 2020-11-16T22:30:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 msgr2=0x7f1df80d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f1de800eba0 tx=0x7f1de8006df0).stop 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 shutdown_connections 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1dcc055060 0x7f1dcc057520 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1df8145a60 0x7f1df80d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1df80615b0 0x7f1df80d37e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 --2- 172.21.15.181:0/2483738465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1df8061a30 0x7f1df80d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 >> 172.21.15.181:0/2483738465 conn(0x7f1df80d68e0 msgr2=0x7f1df8161750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 shutdown_connections 2020-11-16T22:30:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f1dfe477700 1 -- 172.21.15.181:0/2483738465 wait complete. 2020-11-16T22:30:20.479 INFO:tasks.ceph.mds.b.smithi181.stderr:2020-11-16T22:30:18.277+0000 7f1794df4700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1) n(v2 rc2020-11-16T22:29:48.195730+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@4},l=5782 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563a50ced180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:29:48.195730+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:30:20.479 INFO:tasks.ceph.mds.b.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f179ae00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1) n(v1 rc2020-11-16T22:29:48.195730+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5782=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563a50ced700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:29:48.195730+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:30:20.479 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f07ae857700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1) n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@4},l=5782 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560bd09f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:30:20.480 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:30:18.281+0000 7f07ae857700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/AsLsXsFsx@2},l=5782 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560bd0ac6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:30:20.480 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:20.480 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2020-11-16T22:30:20.480 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "e4baabdf-e471-437f-a951-916e78873b52", 2020-11-16T22:30:20.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2020-11-16T22:30:20.480 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:20.481 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:20.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.629+0000 7f04dffff700 1 Processor -- start 2020-11-16T22:30:20.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.629+0000 7f04dffff700 1 -- start start 2020-11-16T22:30:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e0140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04e00cef10 0x7f04e0063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00cf3a0 con 0x7f04e0140550 2020-11-16T22:30:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e015ebb0 con 0x7f04e00cef10 2020-11-16T22:30:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00cf5d0 con 0x7f04e00d0a00 2020-11-16T22:30:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e0140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04deffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f04e0140550 0x7f04e00d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45998/0 (socket says 172.21.15.181:45998) 2020-11-16T22:30:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04deffd700 1 -- 172.21.15.181:0/958221938 learned_addr learned my addr 172.21.15.181:0/958221938 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04df7fe700 1 --2- 172.21.15.181:0/958221938 >> v2:172.21.15.175:3300/0 conn(0x7f04e00cef10 0x7f04e0063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04ddffb700 1 -- 172.21.15.181:0/958221938 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97370874 0 0) 0x7f04e00cf3a0 con 0x7f04e0140550 2020-11-16T22:30:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 -- 172.21.15.181:0/958221938 >> v2:172.21.15.175:3300/0 conn(0x7f04e00cef10 msgr2=0x7f04e0063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 --2- 172.21.15.181:0/958221938 >> v2:172.21.15.175:3300/0 conn(0x7f04e00cef10 0x7f04e0063ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 -- 172.21.15.181:0/958221938 >> v1:172.21.15.181:6789/0 conn(0x7f04e0140550 legacy=0x7f04e00d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 -- 172.21.15.181:0/958221938 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00cf870 con 0x7f04e00d0a00 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04de7fc700 1 --2- 172.21.15.181:0/958221938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e0140930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f04c8009c90 tx=0x7f04c80091d0).ready entity=mon.2 client_cookie=932711e10773b086 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04ddffb700 1 -- 172.21.15.181:0/958221938 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800ba70 con 0x7f04e00d0a00 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04ddffb700 1 -- 172.21.15.181:0/958221938 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c800bbd0 con 0x7f04e00d0a00 2020-11-16T22:30:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04ddffb700 1 -- 172.21.15.181:0/958221938 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8011400 con 0x7f04e00d0a00 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- 172.21.15.181:0/958221938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 msgr2=0x7f04e0140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 --2- 172.21.15.181:0/958221938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e0140930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f04c8009c90 tx=0x7f04c80091d0).stop 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- 172.21.15.181:0/958221938 shutdown_connections 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 --2- 172.21.15.181:0/958221938 >> v2:172.21.15.175:3300/0 conn(0x7f04e00cef10 0x7f04e0063ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 --2- 172.21.15.181:0/958221938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e0140930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- 172.21.15.181:0/958221938 >> 172.21.15.181:0/958221938 conn(0x7f04e00d4570 msgr2=0x7f04e00d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- 172.21.15.181:0/958221938 shutdown_connections 2020-11-16T22:30:20.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.633+0000 7f04dffff700 1 -- 172.21.15.181:0/958221938 wait complete. 2020-11-16T22:30:20.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 Processor -- start 2020-11-16T22:30:20.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- start start 2020-11-16T22:30:20.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e00ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e0140550 0x7f04e00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d35b0 con 0x7f04e0140550 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d3850 con 0x7f04e00cef10 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d3af0 con 0x7f04e00d0a00 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04de7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e00ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59932/0 (socket says 172.21.15.181:59932) 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 -- 172.21.15.181:0/3523560962 learned_addr learned my addr 172.21.15.181:0/3523560962 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04df7fe700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e0140550 0x7f04e00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 -- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 msgr2=0x7f04e00ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e00ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 -- 172.21.15.181:0/3523560962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e0140550 msgr2=0x7f04e00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e0140550 0x7f04e00cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 -- 172.21.15.181:0/3523560962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0162400 con 0x7f04e00cef10 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04deffd700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f04d000c710 tx=0x7f04d0007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d00072e0 con 0x7f04e00cef10 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d0007440 con 0x7f04e00cef10 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e01626a0 con 0x7f04e00cef10 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0005800 con 0x7f04e00cef10 2020-11-16T22:30:20.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.637+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e00d3140 con 0x7f04e00cef10 2020-11-16T22:30:20.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e00d33f0 con 0x7f04e00cef10 2020-11-16T22:30:20.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f04d0010070 con 0x7f04e00cef10 2020-11-16T22:30:20.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04d6ffd700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 0x7f04b0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f04d006d460 con 0x7f04e00cef10 2020-11-16T22:30:20.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04de7fc700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 0x7f04b0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.641+0000 7f04de7fc700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 0x7f04b0057550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f04c800b770 tx=0x7f04c8016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.645+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f04d00436c0 con 0x7f04e00cef10 2020-11-16T22:30:20.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.849+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04e0044f60 con 0x7f04e00cef10 2020-11-16T22:30:20.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.849+0000 7f04d6ffd700 1 -- 172.21.15.181:0/3523560962 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4663 (secure 0 0 0) 0x7f04d0007b60 con 0x7f04e00cef10 2020-11-16T22:30:20.832 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:20.832 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:08.850935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:20.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 msgr2=0x7f04b0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 0x7f04b0057550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f04c800b770 tx=0x7f04c8016000).stop 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 msgr2=0x7f04e00d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f04d000c710 tx=0x7f04d0007a20).stop 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 shutdown_connections 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f04b0055090 0x7f04b0057550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04e0140550 0x7f04e00cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e00cef10 0x7f04e00d2c50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 --2- 172.21.15.181:0/3523560962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e00d0a00 0x7f04e00ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 >> 172.21.15.181:0/3523560962 conn(0x7f04e00d4570 msgr2=0x7f04e0066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 shutdown_connections 2020-11-16T22:30:20.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.853+0000 7f04dffff700 1 -- 172.21.15.181:0/3523560962 wait complete. 2020-11-16T22:30:20.836 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 54 2020-11-16T22:30:20.850 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub abort 2020-11-16T22:30:20.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 Processor -- start 2020-11-16T22:30:20.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 -- start start 2020-11-16T22:30:20.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161480 0x7f5bb8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb8164e10 0x7f5bb8062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 0x7f5bb8062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80c8db0 con 0x7f5bb8161860 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80c8330 con 0x7f5bb8164e10 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbe0b1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80c85d0 con 0x7f5bb8161480 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb8164e10 0x7f5bb8062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bb8164e10 0x7f5bb8062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59940/0 (socket says 172.21.15.181:59940) 2020-11-16T22:30:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/1932174807 learned_addr learned my addr 172.21.15.181:0/1932174807 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161480 0x7f5bb8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1932174807 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 0x7f5bb8062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161480 msgr2=0x7f5bb8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161480 0x7f5bb8164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/1932174807 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 msgr2=0x7f5bb8062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/1932174807 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 0x7f5bb8062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/1932174807 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80c8870 con 0x7f5bb8164e10 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:20.997+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1932174807 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 0x7f5bb8062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8164e10 0x7f5bb8062250 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f5bb8045680 tx=0x7f5bac009290).ready entity=mon.1 client_cookie=dbfbaeade126fa6f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bb7fff700 1 -- 172.21.15.181:0/1932174807 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00bbc0 con 0x7f5bb8164e10 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bb7fff700 1 -- 172.21.15.181:0/1932174807 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac00bd20 con 0x7f5bb8164e10 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bb7fff700 1 -- 172.21.15.181:0/1932174807 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011610 con 0x7f5bb8164e10 2020-11-16T22:30:20.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8164e10 msgr2=0x7f5bb8062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8164e10 0x7f5bb8062250 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f5bb8045680 tx=0x7f5bac009290).stop 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1932174807 shutdown_connections 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1932174807 >> v2:172.21.15.175:3300/0 conn(0x7f5bb8161860 0x7f5bb8062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161480 0x7f5bb8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1932174807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8164e10 0x7f5bb8062250 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1932174807 >> 172.21.15.181:0/1932174807 conn(0x7f5bb80d2dc0 msgr2=0x7f5bb80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1932174807 shutdown_connections 2020-11-16T22:30:20.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1932174807 wait complete. 2020-11-16T22:30:20.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 Processor -- start 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.001+0000 7f5bbe0b1700 1 -- start start 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8161480 0x7f5bb80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 0x7f5bb8174330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8178360 con 0x7f5bb8161480 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8174b10 con 0x7f5bb8173ed0 2020-11-16T22:30:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8174db0 con 0x7f5bb8161860 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39558/0 (socket says 172.21.15.181:39558) 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/1707483304 learned_addr learned my addr 172.21.15.181:0/1707483304 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 0x7f5bb8174330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8161480 0x7f5bb80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 msgr2=0x7f5bb80ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/1707483304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8161480 msgr2=0x7f5bb80cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8161480 0x7f5bb80cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/1707483304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8146a30 con 0x7f5bb8173ed0 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 0x7f5bb8174330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f5bb8161c90 tx=0x7f5bb0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bb57fa700 1 -- 172.21.15.181:0/1707483304 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000f5e0 con 0x7f5bb8173ed0 2020-11-16T22:30:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bb57fa700 1 -- 172.21.15.181:0/1707483304 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb000f740 con 0x7f5bb8173ed0 2020-11-16T22:30:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb8146cd0 con 0x7f5bb8173ed0 2020-11-16T22:30:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bb57fa700 1 -- 172.21.15.181:0/1707483304 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000fae0 con 0x7f5bb8173ed0 2020-11-16T22:30:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.005+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb8174820 con 0x7f5bb8173ed0 2020-11-16T22:30:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bb57fa700 1 -- 172.21.15.181:0/1707483304 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5bb001a030 con 0x7f5bb8173ed0 2020-11-16T22:30:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bb57fa700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bb57fa700 1 -- 172.21.15.181:0/1707483304 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bb006d660 con 0x7f5bb8173ed0 2020-11-16T22:30:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f5ba8006fd0 tx=0x7f5ba8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bcdda4740 1 Processor -- start 2020-11-16T22:30:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.009+0000 7f5bcdda4740 1 -- start start 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x2443710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2440fc0 0x243a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cc4a0 con 0x22227a0 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2380920 con 0x21b4ba0 2020-11-16T22:30:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22406e0 con 0x2440fc0 2020-11-16T22:30:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59954/0 (socket says 172.21.15.181:59954) 2020-11-16T22:30:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/2943694046 learned_addr learned my addr 172.21.15.181:0/2943694046 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2440fc0 0x243a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x2443710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:20.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2440fc0 msgr2=0x243a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2440fc0 0x243a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/2943694046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 msgr2=0x2443710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x2443710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/2943694046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x223b590 con 0x21b4ba0 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5bb80c8470 tx=0x7f5bac016000).ready entity=mon.1 client_cookie=27e930aa5d4fbb60 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5b967fc700 1 -- 172.21.15.181:0/2943694046 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac006ca0 con 0x21b4ba0 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5b967fc700 1 -- 172.21.15.181:0/2943694046 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac021070 con 0x21b4ba0 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5b967fc700 1 -- 172.21.15.181:0/2943694046 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011400 con 0x21b4ba0 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 msgr2=0x2441660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5bb80c8470 tx=0x7f5bac016000).stop 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- 172.21.15.181:0/2943694046 shutdown_connections 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2440fc0 0x243a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x2443710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/2943694046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x2441660 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- 172.21.15.181:0/2943694046 >> 172.21.15.181:0/2943694046 conn(0x23f0e30 msgr2=0x243d390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- 172.21.15.181:0/2943694046 shutdown_connections 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- 172.21.15.181:0/2943694046 wait complete. 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 -- start start 2020-11-16T22:30:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.013+0000 7f5bcdda4740 1 Processor -- start 2020-11-16T22:30:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b4ba0 0x243d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x248fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x236d6d0 con 0x21b4ba0 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x236d810 con 0x2440fc0 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x236ea80 con 0x22227a0 2020-11-16T22:30:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 10 client.0 ll_register_callbacks cb 0x2363ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x248fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59964/0 (socket says 172.21.15.181:59964) 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/3827231855 learned_addr learned my addr 172.21.15.181:0/3827231855 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b4ba0 0x243d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 msgr2=0x248fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x248fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b4ba0 msgr2=0x243d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b4ba0 0x243d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 -- 172.21.15.181:0/3827231855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x235e270 con 0x2440fc0 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x248fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f5bb0046ee0 tx=0x7f5bb0016000).ready entity=mon.1 client_cookie=e1bf02177f4c5afd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0043700 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bb006d0a0 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24216b0 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb006d470 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 10 client.5986 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x2193fc0 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24216b0 con 0x2440fc0 2020-11-16T22:30:21.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f5bb0006040 con 0x2440fc0 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 10 client.5986 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 10 client.5986 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 20 client.5986 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x248e1e0 0x2463840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 4 client.5986 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x225ec40 con 0x248e1e0 2020-11-16T22:30:21.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.017+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x248e1e0 0x2463840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5b877fe700 10 client.5986 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x248e1e0 0x2463840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=36804f4c69abccd1 server_cookie=7ee0b824ab31f416 in_seq=0 out_seq=0 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5b877fe700 1 -- 172.21.15.181:0/3827231855 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x225ec40 con 0x248e1e0 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5b877fe700 10 client.5986 handle_command_reply: tid=0 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 client.5986 shutdown 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 msgr2=0x2490010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f5bb0046ee0 tx=0x7f5bb0016000).stop 2020-11-16T22:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 shutdown_connections 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x248e1e0 0x2463840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2440fc0 0x2490010 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5b877fe700 0 client.5986 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21b4ba0 0x243d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3827231855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x248fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 >> 172.21.15.181:0/3827231855 conn(0x23f0e30 msgr2=0x24433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 shutdown_connections 2020-11-16T22:30:21.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3827231855 wait complete. 2020-11-16T22:30:21.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.021+0000 7f5bcdda4740 20 client.5986 trim_cache size 0 max 16384 2020-11-16T22:30:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 Processor -- start 2020-11-16T22:30:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- start start 2020-11-16T22:30:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x248f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x248fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2345e30 con 0x22227a0 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24954c0 con 0x21b4ba0 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2440090 con 0x23f0e30 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39580/0 (socket says 172.21.15.181:39580) 2020-11-16T22:30:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x248f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 -- 172.21.15.181:0/3389351665 learned_addr learned my addr 172.21.15.181:0/3389351665 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x248fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 -- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 msgr2=0x248f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x248f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 -- 172.21.15.181:0/3389351665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 msgr2=0x248fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x248fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 -- 172.21.15.181:0/3389351665 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0081d0 con 0x23f0e30 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x248f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f5ba8014e40 tx=0x7f5ba8016040).ready entity=mon.2 client_cookie=3985f00eaa8fcfa4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5b96ffd700 1 -- 172.21.15.181:0/3389351665 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8025850 con 0x23f0e30 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5b96ffd700 1 -- 172.21.15.181:0/3389351665 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba80259b0 con 0x23f0e30 2020-11-16T22:30:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5b96ffd700 1 -- 172.21.15.181:0/3389351665 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8025d30 con 0x23f0e30 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 msgr2=0x248b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f5ba8014e40 tx=0x7f5ba8016040).stop 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3389351665 shutdown_connections 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22227a0 0x248fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f0e30 0x248b510 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3389351665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x248f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3389351665 >> 172.21.15.181:0/3389351665 conn(0x7f5b8c002320 msgr2=0x24a0870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3389351665 shutdown_connections 2020-11-16T22:30:21.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3389351665 wait complete. 2020-11-16T22:30:21.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 -- start start 2020-11-16T22:30:21.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.041+0000 7f5bcdda4740 1 Processor -- start 2020-11-16T22:30:21.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f0e30 0x24e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2494c70 con 0x23f0e30 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24eb880 con 0x21b4ba0 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24ebb20 con 0x22227a0 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 10 client.0 ll_register_callbacks cb 0x2441a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59974/0 (socket says 172.21.15.181:59974) 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 -- 172.21.15.181:0/3383059992 learned_addr learned my addr 172.21.15.181:0/3383059992 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39582/0 (socket says 172.21.15.181:39582) 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd8b0700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f0e30 0x24e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 msgr2=0x24a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f0e30 msgr2=0x24e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f0e30 0x24e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 -- 172.21.15.181:0/3383059992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21d2710 con 0x22227a0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbc8ae700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f5bac022000 tx=0x7f5bac01f070).ready entity=mon.2 client_cookie=a564369116e6afc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac02d030 con 0x22227a0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bac00b790 con 0x22227a0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bac021070 con 0x22227a0 2020-11-16T22:30:21.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bac00be90 con 0x22227a0 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 10 client.5606 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x21dddf0 con 0x22227a0 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac021070 con 0x22227a0 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f5bac027870 con 0x22227a0 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 10 client.5606 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 10 client.5606 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:21.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 20 client.5606 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2440fc0 0x2471200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 4 client.5606 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,a,b,o,r,t,",}] 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " s c r u b a b o r t " }) v1 -- 0x2265f40 con 0x2440fc0 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2440fc0 0x2471200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5b877fe700 10 client.5606 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.045+0000 7f5bbd0af700 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2440fc0 0x2471200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cdb1ab41600168ea server_cookie=5c51be3383f6baae in_seq=0 out_seq=0 2020-11-16T22:30:21.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5b877fe700 1 -- 172.21.15.181:0/3383059992 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x2265f40 con 0x2440fc0 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5b877fe700 10 client.5606 handle_command_reply: tid=0 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 client.5606 shutdown 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 msgr2=0x243a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f5bac022000 tx=0x7f5bac01f070).stop 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 shutdown_connections 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2440fc0 0x2471200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f0e30 0x24e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5b877fe700 0 client.5606 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21b4ba0 0x24a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 --2- 172.21.15.181:0/3383059992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22227a0 0x243a6f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 >> 172.21.15.181:0/3383059992 conn(0x7f5b8c002320 msgr2=0x243d230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 shutdown_connections 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 1 -- 172.21.15.181:0/3383059992 wait complete. 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bcdda4740 20 client.5606 trim_cache size 0 max 16384 2020-11-16T22:30:21.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 msgr2=0x7f5b90057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f5ba8006fd0 tx=0x7f5ba8006df0).stop 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 msgr2=0x7f5bb8174330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 0x7f5bb8174330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f5bb8161c90 tx=0x7f5bb0008040).stop 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 shutdown_connections 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5b90055060 0x7f5b90057520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb8173ed0 0x7f5bb8174330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bb8161480 0x7f5bb80cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 --2- 172.21.15.181:0/1707483304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8161860 0x7f5bb80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 >> 172.21.15.181:0/1707483304 conn(0x7f5bb80d2dc0 msgr2=0x7f5bb8062ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 shutdown_connections 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.049+0000 7f5bbe0b1700 1 -- 172.21.15.181:0/1707483304 wait complete. 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:21.040 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:30:21.041 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:21.045 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:21.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 Processor -- start 2020-11-16T22:30:21.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 -- start start 2020-11-16T22:30:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1974161480 0x7f1974164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1974164e10 0x7f1974062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19740c8db0 con 0x7f1974161860 2020-11-16T22:30:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19740c8330 con 0x7f1974164e10 2020-11-16T22:30:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1979224700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19740c85d0 con 0x7f1974161480 2020-11-16T22:30:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1973fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1974161480 0x7f1974164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1973fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1974161480 0x7f1974164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59982/0 (socket says 172.21.15.181:59982) 2020-11-16T22:30:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.193+0000 7f1973fff700 1 -- 172.21.15.181:0/3401048184 learned_addr learned my addr 172.21.15.181:0/3401048184 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f19737fe700 1 --2- 172.21.15.181:0/3401048184 >> v2:172.21.15.175:3300/0 conn(0x7f1974164e10 0x7f1974062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1973fff700 1 -- 172.21.15.181:0/3401048184 >> v2:172.21.15.175:3300/0 conn(0x7f1974164e10 msgr2=0x7f1974062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1973fff700 1 --2- 172.21.15.181:0/3401048184 >> v2:172.21.15.175:3300/0 conn(0x7f1974164e10 0x7f1974062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1973fff700 1 -- 172.21.15.181:0/3401048184 >> v1:172.21.15.175:6789/0 conn(0x7f1974161860 legacy=0x7f1974062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1973fff700 1 -- 172.21.15.181:0/3401048184 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740c8870 con 0x7f1974161480 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1973fff700 1 --2- 172.21.15.181:0/3401048184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1974161480 0x7f1974164920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f19740c8260 tx=0x7f196c006eb0).ready entity=mon.1 client_cookie=b70fc93ffe8eb577 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1972ffd700 1 -- 172.21.15.181:0/3401048184 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c009810 con 0x7f1974161480 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1972ffd700 1 -- 172.21.15.181:0/3401048184 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c009970 con 0x7f1974161480 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1972ffd700 1 -- 172.21.15.181:0/3401048184 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c009cf0 con 0x7f1974161480 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 -- 172.21.15.181:0/3401048184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1974161480 msgr2=0x7f1974164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 --2- 172.21.15.181:0/3401048184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1974161480 0x7f1974164920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f19740c8260 tx=0x7f196c006eb0).stop 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 -- 172.21.15.181:0/3401048184 shutdown_connections 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 --2- 172.21.15.181:0/3401048184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1974161480 0x7f1974164920 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 --2- 172.21.15.181:0/3401048184 >> v2:172.21.15.175:3300/0 conn(0x7f1974164e10 0x7f1974062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 -- 172.21.15.181:0/3401048184 >> 172.21.15.181:0/3401048184 conn(0x7f19740d2dc0 msgr2=0x7f19740ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 -- 172.21.15.181:0/3401048184 shutdown_connections 2020-11-16T22:30:21.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 -- 172.21.15.181:0/3401048184 wait complete. 2020-11-16T22:30:21.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.197+0000 7f1979224700 1 Processor -- start 2020-11-16T22:30:21.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- start start 2020-11-16T22:30:21.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1974161480 0x7f197415e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1974161860 0x7f19740d1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740ca0d0 con 0x7f1974161860 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974177150 con 0x7f19740d1960 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19741773f0 con 0x7f1974161480 2020-11-16T22:30:21.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59986/0 (socket says 172.21.15.181:59986) 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 -- 172.21.15.181:0/71333155 learned_addr learned my addr 172.21.15.181:0/71333155 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1973fff700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1974161480 0x7f197415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f19737fe700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1974161860 0x7f19740d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 -- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1974161480 msgr2=0x7f197415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1974161480 0x7f197415e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 -- 172.21.15.181:0/71333155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1974161860 msgr2=0x7f19740d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1974161860 0x7f19740d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 -- 172.21.15.181:0/71333155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974177690 con 0x7f19740d1960 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1978a23700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f196400ca90 tx=0x7f1964007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400a730 con 0x7f19740d1960 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1974177930 con 0x7f19740d1960 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196400a890 con 0x7f19740d1960 2020-11-16T22:30:21.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19740d1dc0 con 0x7f19740d1960 2020-11-16T22:30:21.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.201+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400ac10 con 0x7f19740d1960 2020-11-16T22:30:21.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974044f60 con 0x7f19740d1960 2020-11-16T22:30:21.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1964010070 con 0x7f19740d1960 2020-11-16T22:30:21.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f196bfff700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 0x7f1944057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f1973fff700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 0x7f1944057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f196406da50 con 0x7f19740d1960 2020-11-16T22:30:21.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.205+0000 7f1973fff700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 0x7f1944057550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f19740c8260 tx=0x7f196c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.213+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1964043ba0 con 0x7f19740d1960 2020-11-16T22:30:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19740cf740 con 0x7f19740d1960 2020-11-16T22:30:21.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f196bfff700 1 -- 172.21.15.181:0/71333155 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4663 (secure 0 0 0) 0x7f19640471c0 con 0x7f19740d1960 2020-11-16T22:30:21.394 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:21.395 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:08.850935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 msgr2=0x7f1944057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 0x7f1944057550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f19740c8260 tx=0x7f196c015000).stop 2020-11-16T22:30:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 msgr2=0x7f1974174ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f196400ca90 tx=0x7f1964007e80).stop 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 shutdown_connections 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1944055090 0x7f1944057550 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19740d1960 0x7f1974174ed0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1974161480 0x7f197415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 --2- 172.21.15.181:0/71333155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1974161860 0x7f19740d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 >> 172.21.15.181:0/71333155 conn(0x7f19740d2dc0 msgr2=0x7f19741630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 shutdown_connections 2020-11-16T22:30:21.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.413+0000 7f1979224700 1 -- 172.21.15.181:0/71333155 wait complete. 2020-11-16T22:30:21.398 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 54 2020-11-16T22:30:21.409 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:30:21.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 Processor -- start 2020-11-16T22:30:21.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 -- start start 2020-11-16T22:30:21.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2dc40cf390 0x7f2dc40cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc4065190 con 0x7f2dc4163970 2020-11-16T22:30:21.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc4065430 con 0x7f2dc4060800 2020-11-16T22:30:21.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.557+0000 7f2dcabb7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc40cfc60 con 0x7f2dc40cf390 2020-11-16T22:30:21.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dca3b6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2dc4060800 0x7f2dc40660b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46070/0 (socket says 172.21.15.181:46070) 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3905172179 learned_addr learned my addr 172.21.15.181:0/3905172179 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3905172179 >> v2:172.21.15.175:3300/0 conn(0x7f2dc40cf390 0x7f2dc40cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462812187 0 0) 0x7f2dc4065430 con 0x7f2dc4060800 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dac003240 con 0x7f2dc4060800 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816769251 0 0) 0x7f2dc4065190 con 0x7f2dc4163970 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc4065430 con 0x7f2dc4163970 2020-11-16T22:30:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2411959245 0 0) 0x7f2dac003240 con 0x7f2dc4060800 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 >> v2:172.21.15.175:3300/0 conn(0x7f2dc40cf390 msgr2=0x7f2dc40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 --2- 172.21.15.181:0/3905172179 >> v2:172.21.15.175:3300/0 conn(0x7f2dc40cf390 0x7f2dc40cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 >> v1:172.21.15.175:6789/0 conn(0x7f2dc4163970 legacy=0x7f2dc40600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40656d0 con 0x7f2dc4060800 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f2dbc001860 con 0x7f2dc4060800 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 >> v1:172.21.15.181:6789/0 conn(0x7f2dc4060800 legacy=0x7f2dc40660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac004920 0x7f2dac004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 0x7f2dac003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dac003df0 0x7f2dac00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40cfc60 con 0x7f2dac003df0 2020-11-16T22:30:21.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac004300 con 0x7f2dac003480 2020-11-16T22:30:21.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac00bc50 con 0x7f2dac004920 2020-11-16T22:30:21.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 0x7f2dac003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac004920 msgr2=0x7f2dac004d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac004920 0x7f2dac004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dac003df0 0x7f2dac00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3905172179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dac003df0 msgr2=0x7f2dac00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dac003df0 0x7f2dac00b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3905172179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40656d0 con 0x7f2dac003480 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 0x7f2dac003900 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f2db40018d0 tx=0x7f2db4007b00).ready entity=mon.1 client_cookie=44cdc4a92f1ea106 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3905172179 reap_dead start 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4007020 con 0x7f2dac003480 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4007180 con 0x7f2dac003480 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.561+0000 7f2dc8bb3700 1 -- 172.21.15.181:0/3905172179 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400c8f0 con 0x7f2dac003480 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 msgr2=0x7f2dac003900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 0x7f2dac003900 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f2db40018d0 tx=0x7f2db4007b00).stop 2020-11-16T22:30:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- 172.21.15.181:0/3905172179 shutdown_connections 2020-11-16T22:30:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/3905172179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac003480 0x7f2dac003900 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- 172.21.15.181:0/3905172179 >> 172.21.15.181:0/3905172179 conn(0x7f2dc40cfe90 msgr2=0x7f2dc40d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- 172.21.15.181:0/3905172179 shutdown_connections 2020-11-16T22:30:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- 172.21.15.181:0/3905172179 wait complete. 2020-11-16T22:30:21.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 Processor -- start 2020-11-16T22:30:21.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.565+0000 7f2dcabb7700 1 -- start start 2020-11-16T22:30:21.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 0x7f2dc415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415cf40 0x7f2dc4171450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4063f90 con 0x7f2dc415cf40 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4170530 con 0x7f2dc41410a0 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41707d0 con 0x7f2dac003480 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39616/0 (socket says 172.21.15.181:39616) 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 0x7f2dc415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc9bb5700 1 -- 172.21.15.181:0/443015446 learned_addr learned my addr 172.21.15.181:0/443015446 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415cf40 0x7f2dc4171450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 -- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 msgr2=0x7f2dc4140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 -- 172.21.15.181:0/443015446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415cf40 msgr2=0x7f2dc4171450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415cf40 0x7f2dc4171450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 -- 172.21.15.181:0/443015446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4170a70 con 0x7f2dc41410a0 2020-11-16T22:30:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 0x7f2dc415ca50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f2db40018d0 tx=0x7f2db4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2db1ffb700 1 -- 172.21.15.181:0/443015446 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4012b70 con 0x7f2dc41410a0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2db1ffb700 1 -- 172.21.15.181:0/443015446 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4012cd0 con 0x7f2dc41410a0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4170d10 con 0x7f2dc41410a0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2db1ffb700 1 -- 172.21.15.181:0/443015446 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db401d650 con 0x7f2dc41410a0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.569+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc415d3c0 con 0x7f2dc41410a0 2020-11-16T22:30:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.573+0000 7f2db1ffb700 1 -- 172.21.15.181:0/443015446 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2db4010070 con 0x7f2dc41410a0 2020-11-16T22:30:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.573+0000 7f2db1ffb700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 0x7f2d9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.573+0000 7f2db1ffb700 1 -- 172.21.15.181:0/443015446 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2db406ece0 con 0x7f2dc41410a0 2020-11-16T22:30:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.573+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 0x7f2d9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.573+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 0x7f2d9c057520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2dc0006d00 tx=0x7f2dc0006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 Processor -- start 2020-11-16T22:30:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- start start 2020-11-16T22:30:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x26c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c3fd0 0x26bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x254f4b0 con 0x26c3fd0 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2603930 con 0x2437bb0 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24c36f0 con 0x24a57b0 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60016/0 (socket says 172.21.15.181:60016) 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 -- 172.21.15.181:0/775461647 learned_addr learned my addr 172.21.15.181:0/775461647 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x26c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c3fd0 0x26bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 -- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 msgr2=0x26c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x26c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 -- 172.21.15.181:0/775461647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c3fd0 msgr2=0x26bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c3fd0 0x26bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 -- 172.21.15.181:0/775461647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24be5a0 con 0x2437bb0 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x26c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f2db40487f0 tx=0x7f2db4012000).ready entity=mon.1 client_cookie=8307383111d9a31c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2da2ffd700 1 -- 172.21.15.181:0/775461647 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40453c0 con 0x2437bb0 2020-11-16T22:30:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2da2ffd700 1 -- 172.21.15.181:0/775461647 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4045520 con 0x2437bb0 2020-11-16T22:30:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2da2ffd700 1 -- 172.21.15.181:0/775461647 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40483b0 con 0x2437bb0 2020-11-16T22:30:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 msgr2=0x26c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f2db40487f0 tx=0x7f2db4012000).stop 2020-11-16T22:30:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- 172.21.15.181:0/775461647 shutdown_connections 2020-11-16T22:30:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c3fd0 0x26bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x26c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/775461647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2437bb0 0x26c4670 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.577+0000 7f2dda8aa740 1 -- 172.21.15.181:0/775461647 >> 172.21.15.181:0/775461647 conn(0x2673e40 msgr2=0x26c03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- 172.21.15.181:0/775461647 shutdown_connections 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- 172.21.15.181:0/775461647 wait complete. 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- start start 2020-11-16T22:30:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 Processor -- start 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2437bb0 0x2713c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x27109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f06e0 con 0x2437bb0 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f0820 con 0x26c3fd0 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25f1a90 con 0x24a57b0 2020-11-16T22:30:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 10 client.0 ll_register_callbacks cb 0x25e6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x27109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60024/0 (socket says 172.21.15.181:60024) 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x27109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39632/0 (socket says 172.21.15.181:39632) 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3883631698 learned_addr learned my addr 172.21.15.181:0/3883631698 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2437bb0 0x2713c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 msgr2=0x27109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x27109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3883631698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2437bb0 msgr2=0x2713c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2437bb0 0x2713c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3883631698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25e1280 con 0x26c3fd0 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2dc4062210 tx=0x7f2dbc006040).ready entity=mon.1 client_cookie=80336fe9e125250a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc014740 con 0x26c3fd0 2020-11-16T22:30:21.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26a46c0 con 0x26c3fd0 2020-11-16T22:30:21.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.581+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2dbc0148a0 con 0x26c3fd0 2020-11-16T22:30:21.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dbc014c70 con 0x26c3fd0 2020-11-16T22:30:21.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 10 client.6010 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:21.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x2416fd0 con 0x26c3fd0 2020-11-16T22:30:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26a46c0 con 0x26c3fd0 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2dbc01c020 con 0x26c3fd0 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 10 client.6010 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 10 client.6010 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 20 client.6010 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26dfbe0 0x2726500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 4 client.6010 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x24e1c50 con 0x26dfbe0 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26dfbe0 0x2726500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26dfbe0 0x2726500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=d7b16745212396ae server_cookie=6ea4dc1a584ae8e6 in_seq=0 out_seq=0 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 10 client.6010 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 1 -- 172.21.15.181:0/3883631698 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x24e1c50 con 0x26dfbe0 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 10 client.6010 handle_command_reply: tid=0 2020-11-16T22:30:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 client.6010 shutdown 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 msgr2=0x26be5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2dc4062210 tx=0x7f2dbc006040).stop 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 shutdown_connections 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26dfbe0 0x2726500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26c3fd0 0x26be5b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2d97fff700 0 client.6010 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:21.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2437bb0 0x2713c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3883631698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a57b0 0x27109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 >> 172.21.15.181:0/3883631698 conn(0x2673e40 msgr2=0x26f6430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 shutdown_connections 2020-11-16T22:30:21.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3883631698 wait complete. 2020-11-16T22:30:21.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.585+0000 7f2dda8aa740 20 client.6010 trim_cache size 0 max 16384 2020-11-16T22:30:21.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 Processor -- start 2020-11-16T22:30:21.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- start start 2020-11-16T22:30:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x26c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c8e40 con 0x24a57b0 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f96f0 con 0x2673e40 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26b86c0 con 0x2437bb0 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x26c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x26c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60030/0 (socket says 172.21.15.181:60030) 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39638/0 (socket says 172.21.15.181:39638) 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc9bb5700 1 -- 172.21.15.181:0/3026430247 learned_addr learned my addr 172.21.15.181:0/3026430247 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 msgr2=0x26c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x26c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3026430247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 msgr2=0x26f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 -- 172.21.15.181:0/3026430247 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90008160 con 0x2437bb0 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x26f93b0 tx=0x7f2db4002000).ready entity=mon.2 client_cookie=340ca316031c1f0c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2da37fe700 1 -- 172.21.15.181:0/3026430247 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40453c0 con 0x2437bb0 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2da37fe700 1 -- 172.21.15.181:0/3026430247 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4047e20 con 0x2437bb0 2020-11-16T22:30:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2da37fe700 1 -- 172.21.15.181:0/3026430247 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40483b0 con 0x2437bb0 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 msgr2=0x26b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x26f93b0 tx=0x7f2db4002000).stop 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3026430247 shutdown_connections 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x26c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3026430247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x26b8f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3026430247 >> 172.21.15.181:0/3026430247 conn(0x7f2d90002320 msgr2=0x26c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3026430247 shutdown_connections 2020-11-16T22:30:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3026430247 wait complete. 2020-11-16T22:30:21.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 -- start start 2020-11-16T22:30:21.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.609+0000 7f2dda8aa740 1 Processor -- start 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x27760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26be8f0 con 0x24a57b0 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f78a0 con 0x2673e40 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26f7b40 con 0x2437bb0 2020-11-16T22:30:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 10 client.0 ll_register_callbacks cb 0x270f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x27760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60038/0 (socket says 172.21.15.181:60038) 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3952199743 learned_addr learned my addr 172.21.15.181:0/3952199743 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dc9bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x27760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39644/0 (socket says 172.21.15.181:39644) 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dc93b4700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 msgr2=0x27760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x27760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3952199743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 msgr2=0x26be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26be400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 -- 172.21.15.181:0/3952199743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2455720 con 0x2673e40 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dca3b6700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f2dbc0139f0 tx=0x7f2dbc01f000).ready entity=mon.1 client_cookie=720fee341f813446 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc02b040 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2db4045520 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2dbc000f00 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dbc014740 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 10 client.6016 fetch_fsmap learned FSMap version 54 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x2460e00 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4045520 con 0x2673e40 2020-11-16T22:30:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2dbc02e070 con 0x2673e40 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 10 client.6016 fetch_fsmap finished waiting for FSMap version 54 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 10 client.6016 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 20 client.6016 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26c3fd0 0x2726f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 4 client.6016 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x24e8f50 con 0x26c3fd0 2020-11-16T22:30:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26c3fd0 0x2726f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 10 client.6016 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dc9bb5700 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26c3fd0 0x2726f00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=3bd411b6d05e84a server_cookie=99f77ac2255d6d33 in_seq=0 out_seq=0 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 1 -- 172.21.15.181:0/3952199743 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x24e8f50 con 0x26c3fd0 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2d97fff700 10 client.6016 handle_command_reply: tid=0 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.613+0000 7f2dda8aa740 1 client.6016 shutdown 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 msgr2=0x2728f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f2dbc0139f0 tx=0x7f2dbc01f000).stop 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 shutdown_connections 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x26c3fd0 0x2726f00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2673e40 0x2728f30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2d97fff700 0 client.6016 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2437bb0 0x27760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 --2- 172.21.15.181:0/3952199743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a57b0 0x26be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 >> 172.21.15.181:0/3952199743 conn(0x7f2d90002320 msgr2=0x26b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 shutdown_connections 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 1 -- 172.21.15.181:0/3952199743 wait complete. 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dda8aa740 20 client.6016 trim_cache size 0 max 16384 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 msgr2=0x7f2d9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 0x7f2d9c057520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2dc0006d00 tx=0x7f2dc0006df0).stop 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 msgr2=0x7f2dc415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 0x7f2dc415ca50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f2db40018d0 tx=0x7f2db4016000).stop 2020-11-16T22:30:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 shutdown_connections 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d9c055060 0x7f2d9c057520 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415cf40 0x7f2dc4171450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac003480 0x7f2dc4140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 --2- 172.21.15.181:0/443015446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc41410a0 0x7f2dc415ca50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 >> 172.21.15.181:0/443015446 conn(0x7f2dc40cfe90 msgr2=0x7f2dc4161100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 shutdown_connections 2020-11-16T22:30:21.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:21.617+0000 7f2dcabb7700 1 -- 172.21.15.181:0/443015446 wait complete. 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout: "e4baabdf-e471-437f-a951-916e78873b52": { 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:30:21.610 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e4baabdf-e471-437f-a951-916e78873b52", 2020-11-16T22:30:21.611 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:30:21.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:21.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:21.611 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:21.613 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:30:25.668 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:30:25.688+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:30:26.614 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:26.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.761+0000 7fb3f61b1700 1 Processor -- start 2020-11-16T22:30:26.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.761+0000 7fb3f61b1700 1 -- start start 2020-11-16T22:30:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb3f0145a30 0x7fb3f0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3f00816e0 0x7fb3f00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f0061b50 con 0x7fb3f01492c0 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00618b0 con 0x7fb3f0145a30 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00cb930 con 0x7fb3f00816e0 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f49ae700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb3f01492c0 0x7fb3f0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46114/0 (socket says 172.21.15.181:46114) 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3f00816e0 0x7fb3f00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f49ae700 1 -- 172.21.15.181:0/3844857649 learned_addr learned my addr 172.21.15.181:0/3844857649 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f51af700 1 --2- 172.21.15.181:0/3844857649 >> v2:172.21.15.175:3300/0 conn(0x7fb3f0145a30 0x7fb3f0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3844857649 >> v2:172.21.15.175:3300/0 conn(0x7fb3f0145a30 msgr2=0x7fb3f0145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 --2- 172.21.15.181:0/3844857649 >> v2:172.21.15.175:3300/0 conn(0x7fb3f0145a30 0x7fb3f0145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3844857649 >> v1:172.21.15.181:6789/0 conn(0x7fb3f01492c0 legacy=0x7fb3f0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3e7fff700 1 -- 172.21.15.181:0/3844857649 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834001361 0 0) 0x7fb3f0061b50 con 0x7fb3f01492c0 2020-11-16T22:30:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3844857649 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00cbbd0 con 0x7fb3f00816e0 2020-11-16T22:30:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f59b0700 1 --2- 172.21.15.181:0/3844857649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f00816e0 0x7fb3f00cc5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb3f00d78a0 tx=0x7fb3e80091d0).ready entity=mon.1 client_cookie=4650e02fb61dc07 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3e7fff700 1 -- 172.21.15.181:0/3844857649 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800ba70 con 0x7fb3f00816e0 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3e7fff700 1 -- 172.21.15.181:0/3844857649 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e800bbd0 con 0x7fb3f00816e0 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3e7fff700 1 -- 172.21.15.181:0/3844857649 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8011400 con 0x7fb3f00816e0 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3844857649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f00816e0 msgr2=0x7fb3f00cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3844857649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f00816e0 0x7fb3f00cc5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb3f00d78a0 tx=0x7fb3e80091d0).stop 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3844857649 shutdown_connections 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3844857649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f00816e0 0x7fb3f00cc5b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3844857649 >> v2:172.21.15.175:3300/0 conn(0x7fb3f0145a30 0x7fb3f0145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3844857649 >> 172.21.15.181:0/3844857649 conn(0x7fb3f00d68c0 msgr2=0x7fb3f00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3844857649 shutdown_connections 2020-11-16T22:30:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.765+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3844857649 wait complete. 2020-11-16T22:30:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.769+0000 7fb3f61b1700 1 Processor -- start 2020-11-16T22:30:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.769+0000 7fb3f61b1700 1 -- start start 2020-11-16T22:30:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f00816e0 0x7fb3f0172900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 0x7fb3f013e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 0x7fb3f015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015de20 con 0x7fb3f00816e0 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0172eb0 con 0x7fb3f01492c0 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0173150 con 0x7fb3f0145a30 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f49ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 0x7fb3f013e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 0x7fb3f015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f51af700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f00816e0 0x7fb3f0172900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f49ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 0x7fb3f013e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39660/0 (socket says 172.21.15.181:39660) 2020-11-16T22:30:26.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f49ae700 1 -- 172.21.15.181:0/3434726388 learned_addr learned my addr 172.21.15.181:0/3434726388 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 msgr2=0x7fb3f013e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 0x7fb3f013e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3434726388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f00816e0 msgr2=0x7fb3f0172900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f00816e0 0x7fb3f0172900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 -- 172.21.15.181:0/3434726388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f01733f0 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f59b0700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 0x7fb3f015e4f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fb3f0081be0 tx=0x7fb3e8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e801c5f0 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e801c750 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0173690 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e801cad0 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.773+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0161900 con 0x7fb3f01492c0 2020-11-16T22:30:26.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0173820 con 0x7fb3f01492c0 2020-11-16T22:30:26.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb3e8024030 con 0x7fb3f01492c0 2020-11-16T22:30:26.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3e57fa700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 0x7fb3c8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:26.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb3e806f8d0 con 0x7fb3f01492c0 2020-11-16T22:30:26.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3f51af700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 0x7fb3c8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:26.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.777+0000 7fb3f51af700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 0x7fb3c8057550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb3e0006fd0 tx=0x7fb3e0006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:26.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.785+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb3f0173820 con 0x7fb3f01492c0 2020-11-16T22:30:26.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.989+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3f0173820 con 0x7fb3f01492c0 2020-11-16T22:30:26.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3e57fa700 1 -- 172.21.15.181:0/3434726388 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7fb3f0173820 con 0x7fb3f01492c0 2020-11-16T22:30:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:26.974 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:26.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 msgr2=0x7fb3c8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 0x7fb3c8057550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb3e0006fd0 tx=0x7fb3e0006df0).stop 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 msgr2=0x7fb3f015e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 0x7fb3f015e4f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fb3f0081be0 tx=0x7fb3e8016000).stop 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 shutdown_connections 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3c8055090 0x7fb3c8057550 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f01492c0 0x7fb3f015e4f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f00816e0 0x7fb3f0172900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 --2- 172.21.15.181:0/3434726388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3f0145a30 0x7fb3f013e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:26.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 >> 172.21.15.181:0/3434726388 conn(0x7fb3f00d68c0 msgr2=0x7fb3f00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:26.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 shutdown_connections 2020-11-16T22:30:26.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:26.993+0000 7fb3f61b1700 1 -- 172.21.15.181:0/3434726388 wait complete. 2020-11-16T22:30:26.978 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:26.989 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:30:27.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 Processor -- start 2020-11-16T22:30:27.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 -- start start 2020-11-16T22:30:27.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f19c4145dc0 0x7f19c40c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c40c8420 con 0x7f19c41461a0 2020-11-16T22:30:27.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40c86c0 con 0x7f19c4145dc0 2020-11-16T22:30:27.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40c8960 con 0x7f19c4061810 2020-11-16T22:30:27.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19b9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f19c41461a0 0x7f19c4061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46130/0 (socket says 172.21.15.181:46130) 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19b9d9b700 1 -- 172.21.15.181:0/2051554642 learned_addr learned my addr 172.21.15.181:0/2051554642 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c259c700 1 --2- 172.21.15.181:0/2051554642 >> v2:172.21.15.175:3300/0 conn(0x7f19c4145dc0 0x7f19c40c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 -- 172.21.15.181:0/2051554642 >> v2:172.21.15.175:3300/0 conn(0x7f19c4145dc0 msgr2=0x7f19c40c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/2051554642 >> v2:172.21.15.175:3300/0 conn(0x7f19c4145dc0 0x7f19c40c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c1d9b700 1 -- 172.21.15.181:0/2051554642 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601268424 0 0) 0x7f19c40c8420 con 0x7f19c41461a0 2020-11-16T22:30:27.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 -- 172.21.15.181:0/2051554642 >> v1:172.21.15.181:6789/0 conn(0x7f19c41461a0 legacy=0x7f19c4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:27.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 -- 172.21.15.181:0/2051554642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40c8c00 con 0x7f19c4061810 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.133+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/2051554642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c40cdc70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f19bc009c90 tx=0x7f19bc0091d0).ready entity=mon.2 client_cookie=55b3636211b73ed5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c1d9b700 1 -- 172.21.15.181:0/2051554642 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00ba70 con 0x7f19c4061810 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c1d9b700 1 -- 172.21.15.181:0/2051554642 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00bbd0 con 0x7f19c4061810 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c1d9b700 1 -- 172.21.15.181:0/2051554642 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc011400 con 0x7f19c4061810 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 -- 172.21.15.181:0/2051554642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 msgr2=0x7f19c40cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:27.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 --2- 172.21.15.181:0/2051554642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c40cdc70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f19bc009c90 tx=0x7f19bc0091d0).stop 2020-11-16T22:30:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 -- 172.21.15.181:0/2051554642 shutdown_connections 2020-11-16T22:30:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 --2- 172.21.15.181:0/2051554642 >> v2:172.21.15.175:3300/0 conn(0x7f19c4145dc0 0x7f19c40c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 --2- 172.21.15.181:0/2051554642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c40cdc70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 -- 172.21.15.181:0/2051554642 >> 172.21.15.181:0/2051554642 conn(0x7f19c40cf630 msgr2=0x7f19c40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 -- 172.21.15.181:0/2051554642 shutdown_connections 2020-11-16T22:30:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.137+0000 7f19c359e700 1 -- 172.21.15.181:0/2051554642 wait complete. 2020-11-16T22:30:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 Processor -- start 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 -- start start 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c4163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c41461a0 0x7f19c4173050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4164350 con 0x7f19c41461a0 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c416f4f0 con 0x7f19c4145dc0 2020-11-16T22:30:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c416f790 con 0x7f19c4061810 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c4163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c4163e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39672/0 (socket says 172.21.15.181:39672) 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c259c700 1 -- 172.21.15.181:0/1579461595 learned_addr learned my addr 172.21.15.181:0/1579461595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60064/0 (socket says 172.21.15.181:60064) 2020-11-16T22:30:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c41461a0 0x7f19c4173050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 msgr2=0x7f19c4163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c4163e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1579461595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c41461a0 msgr2=0x7f19c4173050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c41461a0 0x7f19c4173050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1579461595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c416fa30 con 0x7f19c4145dc0 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f19ac007fd0 tx=0x7f19ac007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19baffd700 1 -- 172.21.15.181:0/1579461595 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0072e0 con 0x7f19c4145dc0 2020-11-16T22:30:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19baffd700 1 -- 172.21.15.181:0/1579461595 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac007440 con 0x7f19c4145dc0 2020-11-16T22:30:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c416fcd0 con 0x7f19c4145dc0 2020-11-16T22:30:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.141+0000 7f19baffd700 1 -- 172.21.15.181:0/1579461595 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac005800 con 0x7f19c4145dc0 2020-11-16T22:30:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c4170440 con 0x7f19c4145dc0 2020-11-16T22:30:27.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19baffd700 1 -- 172.21.15.181:0/1579461595 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f19ac010070 con 0x7f19c4145dc0 2020-11-16T22:30:27.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19baffd700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 0x7f19a0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19baffd700 1 -- 172.21.15.181:0/1579461595 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19ac06d4e0 con 0x7f19c4145dc0 2020-11-16T22:30:27.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19c259c700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 0x7f19a0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.145+0000 7f19c259c700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 0x7f19a0057520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f19b4006a90 tx=0x7f19b4006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 Processor -- start 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- start start 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x22a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 0x229d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x212f4b0 con 0x22a3fd0 2020-11-16T22:30:27.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e3930 con 0x2017bb0 2020-11-16T22:30:27.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x20a36f0 con 0x20857b0 2020-11-16T22:30:27.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39680/0 (socket says 172.21.15.181:39680) 2020-11-16T22:30:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 -- 172.21.15.181:0/344755369 learned_addr learned my addr 172.21.15.181:0/344755369 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x22a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 0x229d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 -- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 msgr2=0x22a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x22a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 -- 172.21.15.181:0/344755369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 msgr2=0x229d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 0x229d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 -- 172.21.15.181:0/344755369 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x209e5a0 con 0x20857b0 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 0x229d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19c259c700 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f19b4013dd0 tx=0x7f19b4014000).ready entity=mon.2 client_cookie=3c26b422b15ebc1e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f199b7fe700 1 -- 172.21.15.181:0/344755369 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b400fb90 con 0x20857b0 2020-11-16T22:30:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f199b7fe700 1 -- 172.21.15.181:0/344755369 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b400fcf0 con 0x20857b0 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f199b7fe700 1 -- 172.21.15.181:0/344755369 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b401e610 con 0x20857b0 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 msgr2=0x22a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f19b4013dd0 tx=0x7f19b4014000).stop 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- 172.21.15.181:0/344755369 shutdown_connections 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22a3fd0 0x229d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22a6720 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 --2- 172.21.15.181:0/344755369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x22a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- 172.21.15.181:0/344755369 >> 172.21.15.181:0/344755369 conn(0x2253e40 msgr2=0x22a03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.149+0000 7f19d7599740 1 -- 172.21.15.181:0/344755369 shutdown_connections 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 -- 172.21.15.181:0/344755369 wait complete. 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 -- start start 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 Processor -- start 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x22f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d06e0 con 0x2017bb0 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d0820 con 0x22a3fd0 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21d1a90 con 0x20857b0 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19d7599740 10 client.0 ll_register_callbacks cb 0x21c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39688/0 (socket says 172.21.15.181:39688) 2020-11-16T22:30:27.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 -- 172.21.15.181:0/2872544440 learned_addr learned my addr 172.21.15.181:0/2872544440 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60080/0 (socket says 172.21.15.181:60080) 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19c259c700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x22f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 -- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 msgr2=0x22a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 -- 172.21.15.181:0/2872544440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 msgr2=0x22f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x22f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19b9d9b700 1 -- 172.21.15.181:0/2872544440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21c1280 con 0x20857b0 2020-11-16T22:30:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.153+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f19ac047330 tx=0x7f19ac049040).ready entity=mon.2 client_cookie=68c35556bd23a235 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:28.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac06d2e0 con 0x20857b0 2020-11-16T22:30:28.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19ac0431f0 con 0x20857b0 2020-11-16T22:30:28.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22846c0 con 0x20857b0 2020-11-16T22:30:28.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19ac046360 con 0x20857b0 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 10 client.5639 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x1ff6fd0 con 0x20857b0 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22846c0 con 0x20857b0 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f19ac01b020 con 0x20857b0 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 10 client.5639 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 10 client.5639 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:28.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 20 client.5639 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x2307390 0x22c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 4 client.5639 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x20c1c50 con 0x2307390 2020-11-16T22:30:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19c259c700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x2307390 0x22c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19c259c700 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x2307390 0x22c2e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=a9a71423099c7654 server_cookie=4124a5a4155c16d3 in_seq=0 out_seq=0 2020-11-16T22:30:28.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 10 client.5639 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:28.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 1 -- 172.21.15.181:0/2872544440 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20c1c50 con 0x2307390 2020-11-16T22:30:28.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 10 client.5639 handle_command_reply: tid=0 2020-11-16T22:30:28.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 client.5639 shutdown 2020-11-16T22:30:28.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 msgr2=0x22f09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f19ac047330 tx=0x7f19ac049040).stop 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 shutdown_connections 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x2307390 0x22c2e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22a3fd0 0x22a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x22f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f1998ff9700 0 client.5639 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 --2- 172.21.15.181:0/2872544440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x22f09e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 >> 172.21.15.181:0/2872544440 conn(0x2253e40 msgr2=0x22a2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 shutdown_connections 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 1 -- 172.21.15.181:0/2872544440 wait complete. 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.157+0000 7f19d7599740 20 client.5639 trim_cache size 0 max 16384 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 Processor -- start 2020-11-16T22:30:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 -- start start 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x229f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20857b0 0x23066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a8e40 con 0x20857b0 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x22a4860 con 0x2017bb0 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22a6270 con 0x2253e40 2020-11-16T22:30:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19c259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39694/0 (socket says 172.21.15.181:39694) 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19c259c700 1 -- 172.21.15.181:0/3826916950 learned_addr learned my addr 172.21.15.181:0/3826916950 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x229f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.181+0000 7f19c2d9d700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20857b0 0x23066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 -- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 msgr2=0x229f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x229f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 -- 172.21.15.181:0/3826916950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20857b0 msgr2=0x23066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20857b0 0x23066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 -- 172.21.15.181:0/3826916950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0081d0 con 0x2253e40 2020-11-16T22:30:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x229f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19c259c700 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f19b4013da0 tx=0x7f19b401e000).ready entity=mon.2 client_cookie=f3c096a802c750ba server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f199bfff700 1 -- 172.21.15.181:0/3826916950 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4013610 con 0x2253e40 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f199bfff700 1 -- 172.21.15.181:0/3826916950 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b402e0b0 con 0x2253e40 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f199bfff700 1 -- 172.21.15.181:0/3826916950 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b401e610 con 0x2253e40 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 msgr2=0x22f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f19b4013da0 tx=0x7f19b401e000).stop 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- 172.21.15.181:0/3826916950 shutdown_connections 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20857b0 0x23066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2253e40 0x22f26d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 --2- 172.21.15.181:0/3826916950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2017bb0 0x229f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- 172.21.15.181:0/3826916950 >> 172.21.15.181:0/3826916950 conn(0x7f198c002320 msgr2=0x22a31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- 172.21.15.181:0/3826916950 shutdown_connections 2020-11-16T22:30:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- 172.21.15.181:0/3826916950 wait complete. 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 -- start start 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.185+0000 7f19d7599740 1 Processor -- start 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x23066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2253e40 0x2302370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:28.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2362b00 con 0x2017bb0 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x235f4c0 con 0x2253e40 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2361be0 con 0x20857b0 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 10 client.0 ll_register_callbacks cb 0x22a49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39702/0 (socket says 172.21.15.181:39702) 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2253e40 0x2302370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1040996851 learned_addr learned my addr 172.21.15.181:0/1040996851 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:28.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19c259c700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x23066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:28.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2253e40 msgr2=0x2302370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:28.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2253e40 0x2302370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1040996851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 msgr2=0x23066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x23066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 -- 172.21.15.181:0/1040996851 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2035720 con 0x20857b0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19b9d9b700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f19ac0437a0 tx=0x7f19ac046cb0).ready entity=mon.2 client_cookie=24ece6b7e68f198 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac04c070 con 0x20857b0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19ac0431f0 con 0x20857b0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2284530 con 0x20857b0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19ac046f40 con 0x20857b0 2020-11-16T22:30:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 10 client.5645 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x2040e00 con 0x20857b0 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2284530 con 0x20857b0 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f19ac01b040 con 0x20857b0 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 10 client.5645 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 10 client.5645 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 20 client.5645 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x22a3fd0 0x2302860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:29.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 4 client.5645 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:30:29.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x7f19b400fcf0 con 0x22a3fd0 2020-11-16T22:30:29.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f19c259c700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x22a3fd0 0x2302860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.189+0000 7f1998ff9700 10 client.5645 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c259c700 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x22a3fd0 0x2302860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=a635a524ff066360 server_cookie=323ee84db70fc763 in_seq=0 out_seq=0 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f1998ff9700 1 -- 172.21.15.181:0/1040996851 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7f19b400fcf0 con 0x22a3fd0 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f1998ff9700 10 client.5645 handle_command_reply: tid=0 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 client.5645 shutdown 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 msgr2=0x2306bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f19ac0437a0 tx=0x7f19ac046cb0).stop 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 shutdown_connections 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x22a3fd0 0x2302860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2253e40 0x2302370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2017bb0 0x23066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 --2- 172.21.15.181:0/1040996851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20857b0 0x2306bc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f1998ff9700 0 client.5645 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 >> 172.21.15.181:0/1040996851 conn(0x7f198c002320 msgr2=0x22d6190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 shutdown_connections 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 1 -- 172.21.15.181:0/1040996851 wait complete. 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19d7599740 20 client.5645 trim_cache size 0 max 16384 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 msgr2=0x7f19a0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 0x7f19a0057520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f19b4006a90 tx=0x7f19b4006df0).stop 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 msgr2=0x7f19c4172b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f19ac007fd0 tx=0x7f19ac007960).stop 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 shutdown_connections 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f19a0055060 0x7f19a0057520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c41461a0 0x7f19c4173050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4061810 0x7f19c4163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 --2- 172.21.15.181:0/1579461595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c4145dc0 0x7f19c4172b60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 >> 172.21.15.181:0/1579461595 conn(0x7f19c40cf630 msgr2=0x7f19c40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 shutdown_connections 2020-11-16T22:30:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:27.193+0000 7f19c359e700 1 -- 172.21.15.181:0/1579461595 wait complete. 2020-11-16T22:30:29.386 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:29.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:30:29.386 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: "e4baabdf-e471-437f-a951-916e78873b52": { 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e4baabdf-e471-437f-a951-916e78873b52", 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:29.387 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:29.388 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:30:29.390 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:29.396 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:29.398 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:34.388 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:34.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 Processor -- start 2020-11-16T22:30:34.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 -- start start 2020-11-16T22:30:34.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f816c0b2900 0x7f816c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:34.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:34.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c0b31d0 con 0x7f816c15d2e0 2020-11-16T22:30:34.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c0c8910 con 0x7f816c0b2900 2020-11-16T22:30:34.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8172000700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0b3460 con 0x7f816c0cf6b0 2020-11-16T22:30:34.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:34.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39712/0 (socket says 172.21.15.181:39712) 2020-11-16T22:30:34.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 -- 172.21.15.181:0/1570755496 learned_addr learned my addr 172.21.15.181:0/1570755496 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:34.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f8170ffe700 1 --2- 172.21.15.181:0/1570755496 >> v2:172.21.15.175:3300/0 conn(0x7f816c0b2900 0x7f816c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:34.725 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:30:34.688+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:30:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 -- 172.21.15.181:0/1570755496 >> v2:172.21.15.175:3300/0 conn(0x7f816c0b2900 msgr2=0x7f816c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 --2- 172.21.15.181:0/1570755496 >> v2:172.21.15.175:3300/0 conn(0x7f816c0b2900 0x7f816c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 -- 172.21.15.181:0/1570755496 >> v1:172.21.15.175:6789/0 conn(0x7f816c15d2e0 legacy=0x7f816c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:34.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 -- 172.21.15.181:0/1570755496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0c8130 con 0x7f816c0cf6b0 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816bfff700 1 --2- 172.21.15.181:0/1570755496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8154006fd0 tx=0x7f8154006eb0).ready entity=mon.2 client_cookie=f856a96c21c34fbc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.537+0000 7f816b7fe700 1 -- 172.21.15.181:0/1570755496 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154009810 con 0x7f816c0cf6b0 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f816b7fe700 1 -- 172.21.15.181:0/1570755496 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8154009970 con 0x7f816c0cf6b0 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f816b7fe700 1 -- 172.21.15.181:0/1570755496 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154009cf0 con 0x7f816c0cf6b0 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- 172.21.15.181:0/1570755496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 msgr2=0x7f816c15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:34.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 --2- 172.21.15.181:0/1570755496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8154006fd0 tx=0x7f8154006eb0).stop 2020-11-16T22:30:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- 172.21.15.181:0/1570755496 shutdown_connections 2020-11-16T22:30:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 --2- 172.21.15.181:0/1570755496 >> v2:172.21.15.175:3300/0 conn(0x7f816c0b2900 0x7f816c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:34.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 --2- 172.21.15.181:0/1570755496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c0cf6b0 0x7f816c15cdf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- 172.21.15.181:0/1570755496 >> 172.21.15.181:0/1570755496 conn(0x7f816c0d2dc0 msgr2=0x7f816c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- 172.21.15.181:0/1570755496 shutdown_connections 2020-11-16T22:30:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- 172.21.15.181:0/1570755496 wait complete. 2020-11-16T22:30:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 Processor -- start 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.541+0000 7f8172000700 1 -- start start 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f816c0b2900 0x7f816c0648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c15d2e0 0x7f816c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c065300 con 0x7f816c0b2900 2020-11-16T22:30:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0655a0 con 0x7f816c0cf6b0 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c065840 con 0x7f816c15d2e0 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60106/0 (socket says 172.21.15.181:60106) 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 -- 172.21.15.181:0/2710968455 learned_addr learned my addr 172.21.15.181:0/2710968455 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f81717ff700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c15d2e0 0x7f816c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8170ffe700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f816c0b2900 0x7f816c0648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 -- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c15d2e0 msgr2=0x7f816c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c15d2e0 0x7f816c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 -- 172.21.15.181:0/2710968455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f816c0b2900 msgr2=0x7f816c0648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f816c0b2900 0x7f816c0648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 -- 172.21.15.181:0/2710968455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0ccd20 con 0x7f816c0cf6b0 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f816bfff700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f8154009fd0 tx=0x7f8154015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154011de0 con 0x7f816c0cf6b0 2020-11-16T22:30:35.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815401e460 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815401e820 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c0ccfc0 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c064f50 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c065a00 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8154025030 con 0x7f816c0cf6b0 2020-11-16T22:30:35.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 0x7f813c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f815406f590 con 0x7f816c0cf6b0 2020-11-16T22:30:35.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.545+0000 7f8170ffe700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 0x7f813c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.549+0000 7f8170ffe700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 0x7f813c057520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f8164006fd0 tx=0x7f8164006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.553+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f81540457f0 con 0x7f816c0cf6b0 2020-11-16T22:30:35.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.757+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f816c0cc6d0 con 0x7f816c0cf6b0 2020-11-16T22:30:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8168ff9700 1 -- 172.21.15.181:0/2710968455 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7f8154023020 con 0x7f816c0cf6b0 2020-11-16T22:30:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 msgr2=0x7f813c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 0x7f813c057520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f8164006fd0 tx=0x7f8164006df0).stop 2020-11-16T22:30:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 msgr2=0x7f816c1402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f8154009fd0 tx=0x7f8154015000).stop 2020-11-16T22:30:35.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 shutdown_connections 2020-11-16T22:30:35.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f813c055060 0x7f813c057520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f816c15d2e0 0x7f816c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f816c0b2900 0x7f816c0648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 --2- 172.21.15.181:0/2710968455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f816c0cf6b0 0x7f816c1402f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 >> 172.21.15.181:0/2710968455 conn(0x7f816c0d2dc0 msgr2=0x7f816c156470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 shutdown_connections 2020-11-16T22:30:35.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:34.761+0000 7f8172000700 1 -- 172.21.15.181:0/2710968455 wait complete. 2020-11-16T22:30:35.551 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:35.551 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:35.552 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:30:35.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.701+0000 7f04967c1700 1 Processor -- start 2020-11-16T22:30:35.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.701+0000 7f04967c1700 1 -- start start 2020-11-16T22:30:35.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f049015d8f0 0x7f04900b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04900b79f0 0x7f04900b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04900ceb10 con 0x7f049015d510 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04900cedb0 con 0x7f049015d8f0 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04900cf050 con 0x7f04900b79f0 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04900b79f0 0x7f04900b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f049015d510 0x7f04900ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46186/0 (socket says 172.21.15.181:46186) 2020-11-16T22:30:35.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048ffff700 1 -- 172.21.15.181:0/3688175149 learned_addr learned my addr 172.21.15.181:0/3688175149 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:35.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3688175149 >> v2:172.21.15.175:3300/0 conn(0x7f049015d8f0 0x7f04900b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 -- 172.21.15.181:0/3688175149 >> v2:172.21.15.175:3300/0 conn(0x7f049015d8f0 msgr2=0x7f04900b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3688175149 >> v2:172.21.15.175:3300/0 conn(0x7f049015d8f0 0x7f04900b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 -- 172.21.15.181:0/3688175149 >> v1:172.21.15.181:6789/0 conn(0x7f049015d510 legacy=0x7f04900ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048effd700 1 -- 172.21.15.181:0/3688175149 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45467144 0 0) 0x7f04900ceb10 con 0x7f049015d510 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 -- 172.21.15.181:0/3688175149 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04900cf2f0 con 0x7f04900b79f0 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3688175149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900b89c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f0490067ce0 tx=0x7f0480006df0).ready entity=mon.1 client_cookie=3fc553fbe904ff8e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048effd700 1 -- 172.21.15.181:0/3688175149 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800097b0 con 0x7f04900b79f0 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048effd700 1 -- 172.21.15.181:0/3688175149 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480009910 con 0x7f04900b79f0 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f048effd700 1 -- 172.21.15.181:0/3688175149 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480009c90 con 0x7f04900b79f0 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- 172.21.15.181:0/3688175149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 msgr2=0x7f04900b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 --2- 172.21.15.181:0/3688175149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900b89c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f0490067ce0 tx=0x7f0480006df0).stop 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- 172.21.15.181:0/3688175149 shutdown_connections 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 --2- 172.21.15.181:0/3688175149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900b89c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 --2- 172.21.15.181:0/3688175149 >> v2:172.21.15.175:3300/0 conn(0x7f049015d8f0 0x7f04900b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- 172.21.15.181:0/3688175149 >> 172.21.15.181:0/3688175149 conn(0x7f04900d3680 msgr2=0x7f04900d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- 172.21.15.181:0/3688175149 shutdown_connections 2020-11-16T22:30:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.705+0000 7f04967c1700 1 -- 172.21.15.181:0/3688175149 wait complete. 2020-11-16T22:30:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 Processor -- start 2020-11-16T22:30:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 -- start start 2020-11-16T22:30:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f049015d8f0 0x7f04900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0490160110 con 0x7f049015d8f0 2020-11-16T22:30:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04901478d0 con 0x7f04900b79f0 2020-11-16T22:30:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f04967c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0490147b70 con 0x7f049015d510 2020-11-16T22:30:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39730/0 (socket says 172.21.15.181:39730) 2020-11-16T22:30:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048f7fe700 1 -- 172.21.15.181:0/3087608715 learned_addr learned my addr 172.21.15.181:0/3087608715 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f049015d8f0 0x7f04900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 -- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 msgr2=0x7f04900cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 -- 172.21.15.181:0/3087608715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f049015d8f0 msgr2=0x7f04900b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f049015d8f0 0x7f04900b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 -- 172.21.15.181:0/3087608715 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490147e10 con 0x7f04900b79f0 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.709+0000 7f048ffff700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900cd0c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f048800cae0 tx=0x7f0488007ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 -- 172.21.15.181:0/3087608715 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488005b80 con 0x7f04900b79f0 2020-11-16T22:30:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 -- 172.21.15.181:0/3087608715 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488005ce0 con 0x7f04900b79f0 2020-11-16T22:30:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04900ca9d0 con 0x7f04900b79f0 2020-11-16T22:30:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 -- 172.21.15.181:0/3087608715 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488007340 con 0x7f04900b79f0 2020-11-16T22:30:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04900caf10 con 0x7f04900b79f0 2020-11-16T22:30:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 -- 172.21.15.181:0/3087608715 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0488010030 con 0x7f04900b79f0 2020-11-16T22:30:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 0x7f046c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f0487fff700 1 -- 172.21.15.181:0/3087608715 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f048806dc20 con 0x7f04900b79f0 2020-11-16T22:30:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 0x7f046c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.713+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 0x7f046c057520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f0490153510 tx=0x7f0478006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 Processor -- start 2020-11-16T22:30:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 -- start start 2020-11-16T22:30:35.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1474cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13074b0 con 0x1476470 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13bb930 con 0x11efbb0 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x127b6f0 con 0x125d7b0 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1474cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60130/0 (socket says 172.21.15.181:60130) 2020-11-16T22:30:35.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1474cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39736/0 (socket says 172.21.15.181:39736) 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f048f7fe700 1 -- 172.21.15.181:0/3274815156 learned_addr learned my addr 172.21.15.181:0/3274815156 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.717+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 msgr2=0x1474cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1474cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3274815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 msgr2=0x14d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3274815156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12765a0 con 0x11efbb0 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f0478013610 tx=0x7f0478014000).ready entity=mon.1 client_cookie=3cf9bec86cd2814d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04857fa700 1 -- 172.21.15.181:0/3274815156 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478013de0 con 0x11efbb0 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04857fa700 1 -- 172.21.15.181:0/3274815156 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047801e400 con 0x11efbb0 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04857fa700 1 -- 172.21.15.181:0/3274815156 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047801e780 con 0x11efbb0 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 msgr2=0x14b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f0478013610 tx=0x7f0478014000).stop 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- 172.21.15.181:0/3274815156 shutdown_connections 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1474cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- 172.21.15.181:0/3274815156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14b3530 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- 172.21.15.181:0/3274815156 >> 172.21.15.181:0/3274815156 conn(0x142be40 msgr2=0x14d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- 172.21.15.181:0/3274815156 shutdown_connections 2020-11-16T22:30:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- 172.21.15.181:0/3274815156 wait complete. 2020-11-16T22:30:35.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- start start 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 Processor -- start 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11efbb0 0x149f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13a86e0 con 0x1476470 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13a8820 con 0x125d7b0 2020-11-16T22:30:35.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13a9a90 con 0x11efbb0 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04a5252740 10 client.0 ll_register_callbacks cb 0x139ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11efbb0 0x149f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60132/0 (socket says 172.21.15.181:60132) 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3890686397 learned_addr learned my addr 172.21.15.181:0/3890686397 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11efbb0 msgr2=0x149f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11efbb0 0x149f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3890686397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 msgr2=0x14dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14dec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 -- 172.21.15.181:0/3890686397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1399280 con 0x125d7b0 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f0494d5e700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0478013530 tx=0x7f0478021000).ready entity=mon.1 client_cookie=dfa78a0ee2170b3d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.721+0000 7f04667fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047802d070 con 0x125d7b0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0478029630 con 0x125d7b0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1470e10 con 0x125d7b0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047801fe80 con 0x125d7b0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 10 client.6061 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x11cefd0 con 0x125d7b0 2020-11-16T22:30:35.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1470e10 con 0x125d7b0 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f0478027040 con 0x125d7b0 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 10 client.6061 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 10 client.6061 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 20 client.6061 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x14daed0 0x14db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 4 client.6061 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:35.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1299c50 con 0x14daed0 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f048ffff700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x14daed0 0x14db2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f048ffff700 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x14daed0 0x14db2b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=af2aa15c72e7bb1b server_cookie=c9e6609fc00cec31 in_seq=0 out_seq=0 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 10 client.6061 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 1 -- 172.21.15.181:0/3890686397 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1299c50 con 0x14daed0 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 10 client.6061 handle_command_reply: tid=0 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 client.6061 shutdown 2020-11-16T22:30:35.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 msgr2=0x14de780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0478013530 tx=0x7f0478021000).stop 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 shutdown_connections 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x14daed0 0x14db2b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1476470 0x14dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11efbb0 0x149f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04667fc700 0 client.6061 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:35.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 --2- 172.21.15.181:0/3890686397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x125d7b0 0x14de780 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 >> 172.21.15.181:0/3890686397 conn(0x142be40 msgr2=0x14b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 shutdown_connections 2020-11-16T22:30:35.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 1 -- 172.21.15.181:0/3890686397 wait complete. 2020-11-16T22:30:35.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.725+0000 7f04a5252740 20 client.6061 trim_cache size 0 max 16384 2020-11-16T22:30:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.745+0000 7f04a5252740 1 Processor -- start 2020-11-16T22:30:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.745+0000 7f04a5252740 1 -- start start 2020-11-16T22:30:35.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x142be40 0x1489460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x14a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14b4760 con 0x142be40 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1380e40 con 0x11efbb0 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14da9a0 con 0x125d7b0 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f0494d5e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x14a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60142/0 (socket says 172.21.15.181:60142) 2020-11-16T22:30:35.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 -- 172.21.15.181:0/592234555 learned_addr learned my addr 172.21.15.181:0/592234555 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:35.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048ffff700 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x142be40 0x1489460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 -- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 msgr2=0x14a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x14a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 -- 172.21.15.181:0/592234555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x142be40 msgr2=0x1489460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x142be40 0x1489460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 -- 172.21.15.181:0/592234555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0081d0 con 0x11efbb0 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f048f7fe700 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f047800b260 tx=0x7f0478029000).ready entity=mon.1 client_cookie=3e987f5a2143321 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f0485ffb700 1 -- 172.21.15.181:0/592234555 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047802d070 con 0x11efbb0 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f0485ffb700 1 -- 172.21.15.181:0/592234555 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478029a90 con 0x11efbb0 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f0485ffb700 1 -- 172.21.15.181:0/592234555 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478029e10 con 0x11efbb0 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 msgr2=0x14a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f047800b260 tx=0x7f0478029000).stop 2020-11-16T22:30:35.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- 172.21.15.181:0/592234555 shutdown_connections 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x14a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x142be40 0x1489460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 --2- 172.21.15.181:0/592234555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11efbb0 0x14a0150 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- 172.21.15.181:0/592234555 >> 172.21.15.181:0/592234555 conn(0x7f045c002320 msgr2=0x14d9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- 172.21.15.181:0/592234555 shutdown_connections 2020-11-16T22:30:35.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- 172.21.15.181:0/592234555 wait complete. 2020-11-16T22:30:35.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 -- start start 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.749+0000 7f04a5252740 1 Processor -- start 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11efbb0 0x1535840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 0x1476f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14814e0 con 0x11efbb0 2020-11-16T22:30:35.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1535ef0 con 0x142be40 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1536d50 con 0x125d7b0 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04a5252740 10 client.0 ll_register_callbacks cb 0x14b5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f048f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39758/0 (socket says 172.21.15.181:39758) 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f048ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11efbb0 0x1535840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f048f7fe700 1 -- 172.21.15.181:0/1225723468 learned_addr learned my addr 172.21.15.181:0/1225723468 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:35.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 0x1476f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 -- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 msgr2=0x1536860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 -- 172.21.15.181:0/1225723468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11efbb0 msgr2=0x1535840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:35.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11efbb0 0x1535840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 -- 172.21.15.181:0/1225723468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x120d720 con 0x142be40 2020-11-16T22:30:35.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f048f7fe700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:35.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f0494d5e700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 0x1476f30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f04900681b0 tx=0x7f0480015000).ready entity=mon.1 client_cookie=f3829c9339e4890e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:35.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04667fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:35.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.753+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480009cd0 con 0x142be40 2020-11-16T22:30:35.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0480011c50 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1476bb0 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0480011e10 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 10 client.6067 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x1218e00 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1476bb0 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f048001f650 con 0x142be40 2020-11-16T22:30:35.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 10 client.6067 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 10 client.6067 resolve_mds: resolved ID 'b' to GID 5605 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 20 client.6067 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x1476470 0x14a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 4 client.6067 mds_command: new command op to 5605 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 --> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x7f047801e400 con 0x1476470 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f048ffff700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x1476470 0x14a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:35.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 10 client.6067 ms_handle_connect on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f048ffff700 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x1476470 0x14a0040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=3c5fe3c98548b5f9 server_cookie=cb5c71e8970647cb in_seq=0 out_seq=0 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 1 -- 172.21.15.181:0/1225723468 <== mds.1 v2:172.21.15.181:6836/2557321074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f047801e400 con 0x1476470 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 10 client.6067 handle_command_reply: tid=0 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 client.6067 shutdown 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 msgr2=0x1476f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 0x1476f30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f04900681b0 tx=0x7f0480015000).stop 2020-11-16T22:30:35.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 shutdown_connections 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:6836/2557321074,v1:172.21.15.181:6837/2557321074] conn(0x1476470 0x14a0040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x142be40 0x1476f30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11efbb0 0x1535840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 --2- 172.21.15.181:0/1225723468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125d7b0 0x1536860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 >> 172.21.15.181:0/1225723468 conn(0x7f045c002320 msgr2=0x14b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04667fc700 0 client.6067 ms_handle_reset on v2:172.21.15.181:6836/2557321074 2020-11-16T22:30:35.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 shutdown_connections 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 1 -- 172.21.15.181:0/1225723468 wait complete. 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04a5252740 20 client.6067 trim_cache size 0 max 16384 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 msgr2=0x7f046c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 0x7f046c057520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f0490153510 tx=0x7f0478006df0).stop 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 msgr2=0x7f04900cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900cd0c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f048800cae0 tx=0x7f0488007ce0).stop 2020-11-16T22:30:35.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 shutdown_connections 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f046c055060 0x7f046c057520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.757+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f049015d8f0 0x7f04900b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.761+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900b79f0 0x7f04900cd0c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.761+0000 7f04967c1700 1 --2- 172.21.15.181:0/3087608715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f049015d510 0x7f04900cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.761+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 >> 172.21.15.181:0/3087608715 conn(0x7f04900d3680 msgr2=0x7f04901539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.761+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 shutdown_connections 2020-11-16T22:30:35.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:35.761+0000 7f04967c1700 1 -- 172.21.15.181:0/3087608715 wait complete. 2020-11-16T22:30:35.953 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:30:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:30:35.953 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:35.954 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:36.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 Processor -- start 2020-11-16T22:30:36.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- start start 2020-11-16T22:30:36.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02440b2900 0x7f02440b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f024415d500 0x7f02440c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024415d9b0 con 0x7f02440cf6b0 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440c8910 con 0x7f024415d500 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02440b3460 con 0x7f02440b2900 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0243fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02440b2900 0x7f02440b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f024415d500 0x7f02440c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f024415d500 0x7f02440c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60158/0 (socket says 172.21.15.181:60158) 2020-11-16T22:30:36.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 -- 172.21.15.181:0/492576270 learned_addr learned my addr 172.21.15.181:0/492576270 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 -- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02440b2900 msgr2=0x7f02440b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 --2- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02440b2900 0x7f02440b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 -- 172.21.15.181:0/492576270 >> v1:172.21.15.175:6789/0 conn(0x7f02440cf6b0 legacy=0x7f024415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 -- 172.21.15.181:0/492576270 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440b31d0 con 0x7f024415d500 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0248879700 1 --2- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f024415d500 0x7f02440c7250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f023c009db0 tx=0x7f023c009290).ready entity=mon.1 client_cookie=f7736ef59929836c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0242ffd700 1 -- 172.21.15.181:0/492576270 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c00bb40 con 0x7f024415d500 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0242ffd700 1 -- 172.21.15.181:0/492576270 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c00bca0 con 0x7f024415d500 2020-11-16T22:30:36.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f0242ffd700 1 -- 172.21.15.181:0/492576270 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c011610 con 0x7f024415d500 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f024415d500 msgr2=0x7f02440c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 --2- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f024415d500 0x7f02440c7250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f023c009db0 tx=0x7f023c009290).stop 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- 172.21.15.181:0/492576270 shutdown_connections 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 --2- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f024415d500 0x7f02440c7250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 --2- 172.21.15.181:0/492576270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02440b2900 0x7f02440b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- 172.21.15.181:0/492576270 >> 172.21.15.181:0/492576270 conn(0x7f02440d2dc0 msgr2=0x7f02440ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- 172.21.15.181:0/492576270 shutdown_connections 2020-11-16T22:30:36.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.105+0000 7f024907a700 1 -- 172.21.15.181:0/492576270 wait complete. 2020-11-16T22:30:36.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 Processor -- start 2020-11-16T22:30:36.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 -- start start 2020-11-16T22:30:36.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02440b2900 0x7f024416d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0244171290 con 0x7f02440b2900 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0244154de0 con 0x7f02440cf6b0 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.109+0000 7f024907a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0244155080 con 0x7f024415d500 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0248879700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60162/0 (socket says 172.21.15.181:60162) 2020-11-16T22:30:36.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 -- 172.21.15.181:0/4052986991 learned_addr learned my addr 172.21.15.181:0/4052986991 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0248879700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39774/0 (socket says 172.21.15.181:39774) 2020-11-16T22:30:36.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0243fff700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02440b2900 0x7f024416d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 -- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 msgr2=0x7f02440c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 -- 172.21.15.181:0/4052986991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02440b2900 msgr2=0x7f024416d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02440b2900 0x7f024416d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 -- 172.21.15.181:0/4052986991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244155320 con 0x7f02440cf6b0 2020-11-16T22:30:36.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0248879700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f02437fe700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f023800c710 tx=0x7f0238007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238007020 con 0x7f02440cf6b0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238007180 con 0x7f02440cf6b0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02441555c0 con 0x7f02440cf6b0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800e560 con 0x7f02440cf6b0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02440c7960 con 0x7f02440cf6b0 2020-11-16T22:30:36.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0238010070 con 0x7f02440cf6b0 2020-11-16T22:30:36.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 0x7f021c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f023806ce80 con 0x7f02440cf6b0 2020-11-16T22:30:36.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0243fff700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 0x7f021c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024415d9b0 con 0x7f02440cf6b0 2020-11-16T22:30:36.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.113+0000 7f0243fff700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 0x7f021c057550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0234006fd0 tx=0x7f0234006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.121+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f024415d9b0 con 0x7f02440cf6b0 2020-11-16T22:30:36.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.321+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f024415d9b0 con 0x7f02440cf6b0 2020-11-16T22:30:36.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f0223fff700 1 -- 172.21.15.181:0/4052986991 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7f024415d9b0 con 0x7f02440cf6b0 2020-11-16T22:30:36.305 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:36.305 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:36.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 msgr2=0x7f021c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 0x7f021c057550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0234006fd0 tx=0x7f0234006df0).stop 2020-11-16T22:30:36.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 msgr2=0x7f024416d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f023800c710 tx=0x7f0238007c20).stop 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 shutdown_connections 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f021c055090 0x7f021c057550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f024415d500 0x7f02440c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02440b2900 0x7f024416d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 --2- 172.21.15.181:0/4052986991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02440cf6b0 0x7f024416d8b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 >> 172.21.15.181:0/4052986991 conn(0x7f02440d2dc0 msgr2=0x7f02440649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 shutdown_connections 2020-11-16T22:30:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.325+0000 7f024907a700 1 -- 172.21.15.181:0/4052986991 wait complete. 2020-11-16T22:30:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:36.320 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:30:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 Processor -- start 2020-11-16T22:30:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 -- start start 2020-11-16T22:30:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38280cf390 0x7f38280cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3828163970 0x7f3828163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 0x7f3828060580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38280cfc60 con 0x7f3828163970 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3828065190 con 0x7f38280cf390 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.465+0000 7f382e71c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3828065430 con 0x7f38280600f0 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38280cf390 0x7f38280cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 0x7f3828060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38280cf390 0x7f38280cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60168/0 (socket says 172.21.15.181:60168) 2020-11-16T22:30:36.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 -- 172.21.15.181:0/1971887878 learned_addr learned my addr 172.21.15.181:0/1971887878 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382cf19700 1 --2- 172.21.15.181:0/1971887878 >> v2:172.21.15.175:3300/0 conn(0x7f3828163970 0x7f3828163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 -- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 msgr2=0x7f3828060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 0x7f3828060580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 -- 172.21.15.181:0/1971887878 >> v2:172.21.15.175:3300/0 conn(0x7f3828163970 msgr2=0x7f3828163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 --2- 172.21.15.181:0/1971887878 >> v2:172.21.15.175:3300/0 conn(0x7f3828163970 0x7f3828163df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 -- 172.21.15.181:0/1971887878 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280656d0 con 0x7f38280cf390 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382df1b700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 0x7f3828060580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382d71a700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f38280cf770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f38280622a0 tx=0x7f381c006eb0).ready entity=mon.1 client_cookie=976640231662d9f5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f3813fff700 1 -- 172.21.15.181:0/1971887878 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c009720 con 0x7f38280cf390 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f3813fff700 1 -- 172.21.15.181:0/1971887878 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c009880 con 0x7f38280cf390 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f3813fff700 1 -- 172.21.15.181:0/1971887878 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c009c00 con 0x7f38280cf390 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 msgr2=0x7f38280cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f38280cf770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f38280622a0 tx=0x7f381c006eb0).stop 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- 172.21.15.181:0/1971887878 shutdown_connections 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280600f0 0x7f3828060580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 --2- 172.21.15.181:0/1971887878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f38280cf770 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 --2- 172.21.15.181:0/1971887878 >> v2:172.21.15.175:3300/0 conn(0x7f3828163970 0x7f3828163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- 172.21.15.181:0/1971887878 >> 172.21.15.181:0/1971887878 conn(0x7f38280cfe90 msgr2=0x7f38280d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- 172.21.15.181:0/1971887878 shutdown_connections 2020-11-16T22:30:36.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- 172.21.15.181:0/1971887878 wait complete. 2020-11-16T22:30:36.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 Processor -- start 2020-11-16T22:30:36.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.469+0000 7f382e71c700 1 -- start start 2020-11-16T22:30:36.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38280600f0 0x7f38280b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3828163970 0x7f382816f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382815d330 con 0x7f38280600f0 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828162f40 con 0x7f38280cf390 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3828172970 con 0x7f3828163970 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60172/0 (socket says 172.21.15.181:60172) 2020-11-16T22:30:36.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3828163970 0x7f382816f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 -- 172.21.15.181:0/1884330420 learned_addr learned my addr 172.21.15.181:0/1884330420 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382d71a700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38280600f0 0x7f38280b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 -- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3828163970 msgr2=0x7f382816f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3828163970 0x7f382816f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 -- 172.21.15.181:0/1884330420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38280600f0 msgr2=0x7f38280b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38280600f0 0x7f38280b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 -- 172.21.15.181:0/1884330420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828162ca0 con 0x7f38280cf390 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382cf19700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f382400eba0 tx=0x7f3824006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f3819ffb700 1 -- 172.21.15.181:0/1884330420 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382400b3a0 con 0x7f38280cf390 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f3819ffb700 1 -- 172.21.15.181:0/1884330420 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382400c8e0 con 0x7f38280cf390 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f3819ffb700 1 -- 172.21.15.181:0/1884330420 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382400cca0 con 0x7f38280cf390 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3828172bd0 con 0x7f38280cf390 2020-11-16T22:30:36.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.473+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281735b0 con 0x7f38280cf390 2020-11-16T22:30:36.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f3819ffb700 1 -- 172.21.15.181:0/1884330420 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3824012600 con 0x7f38280cf390 2020-11-16T22:30:36.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 Processor -- start 2020-11-16T22:30:36.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 -- start start 2020-11-16T22:30:36.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f3819ffb700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 0x7f37fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f3819ffb700 1 -- 172.21.15.181:0/1884330420 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f382406e0b0 con 0x7f38280cf390 2020-11-16T22:30:36.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382df1b700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 0x7f37fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382df1b700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 0x7f37fc057520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f38280cd2c0 tx=0x7f38200091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 0xe18700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xcaa4b0 con 0xb92bb0 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xd5e930 con 0xe1efd0 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xc1e6f0 con 0xc007b0 2020-11-16T22:30:36.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 0xe18700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39790/0 (socket says 172.21.15.181:39790) 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 -- 172.21.15.181:0/1985646348 learned_addr learned my addr 172.21.15.181:0/1985646348 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382cf19700 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 -- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 msgr2=0xe18700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 0xe18700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 -- 172.21.15.181:0/1985646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 msgr2=0xe1f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.477+0000 7f382d71a700 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe1f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f382d71a700 1 -- 172.21.15.181:0/1985646348 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc195a0 con 0xc007b0 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f382df1b700 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 0xe18700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f382d71a700 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f381c010000 tx=0x7f381c015000).ready entity=mon.2 client_cookie=1fedb2b1dd271fdb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f38127fc700 1 -- 172.21.15.181:0/1985646348 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c00b600 con 0xc007b0 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f38127fc700 1 -- 172.21.15.181:0/1985646348 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c00b760 con 0xc007b0 2020-11-16T22:30:36.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f38127fc700 1 -- 172.21.15.181:0/1985646348 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c009c00 con 0xc007b0 2020-11-16T22:30:36.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 msgr2=0xe21720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f381c010000 tx=0x7f381c015000).stop 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- 172.21.15.181:0/1985646348 shutdown_connections 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe1efd0 0xe18700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe21720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 --2- 172.21.15.181:0/1985646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe1f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- 172.21.15.181:0/1985646348 >> 172.21.15.181:0/1985646348 conn(0xdcee40 msgr2=0xe1b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- 172.21.15.181:0/1985646348 shutdown_connections 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- 172.21.15.181:0/1985646348 wait complete. 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 -- start start 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.481+0000 7f383e40f740 1 Processor -- start 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe4ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe1efd0 0xe6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4b6e0 con 0xe1efd0 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4b820 con 0xb92bb0 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xd4ca90 con 0xc007b0 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 10 client.0 ll_register_callbacks cb 0xd41ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39796/0 (socket says 172.21.15.181:39796) 2020-11-16T22:30:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 -- 172.21.15.181:0/1228687334 learned_addr learned my addr 172.21.15.181:0/1228687334 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382df1b700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe1efd0 0xe6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382d71a700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe4ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 -- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 msgr2=0xe4ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe4ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 -- 172.21.15.181:0/1228687334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe1efd0 msgr2=0xe6cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe1efd0 0xe6cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 -- 172.21.15.181:0/1228687334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd3c280 con 0xc007b0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382cf19700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3824047ea0 tx=0x7f382401e040).ready entity=mon.2 client_cookie=74713aa019bc91ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240463f0 con 0xc007b0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3824044680 con 0xc007b0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xdff6c0 con 0xc007b0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3824044840 con 0xc007b0 2020-11-16T22:30:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 10 client.5684 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0xb71fd0 con 0xc007b0 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xdff6c0 con 0xc007b0 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f3824043f70 con 0xc007b0 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 10 client.5684 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 10 client.5684 resolve_mds: resolved ID 'a' to GID 5599 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 20 client.5684 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:36.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1b900 0xe4f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 4 client.5684 mds_command: new command op to 5599 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:36.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 --> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xc3cc50 con 0xe1b900 2020-11-16T22:30:36.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382d71a700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1b900 0xe4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f37fb7fe700 10 client.5684 ms_handle_connect on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:36.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.485+0000 7f382d71a700 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1b900 0xe4f600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=e2fb6f6b46b5b0e4 server_cookie=6133c0c1949c75a in_seq=0 out_seq=0 2020-11-16T22:30:36.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f37fb7fe700 1 -- 172.21.15.181:0/1228687334 <== mds.2 v2:172.21.15.181:6834/3960101400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xc3cc50 con 0xe1b900 2020-11-16T22:30:36.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f37fb7fe700 10 client.5684 handle_command_reply: tid=0 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 client.5684 shutdown 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 msgr2=0xe6c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3824047ea0 tx=0x7f382401e040).stop 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 shutdown_connections 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1b900 0xe4f600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe1efd0 0xe6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe4ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 --2- 172.21.15.181:0/1228687334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe6c770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 >> 172.21.15.181:0/1228687334 conn(0xdcee40 msgr2=0xe54a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f37fb7fe700 0 client.5684 ms_handle_reset on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 shutdown_connections 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 1 -- 172.21.15.181:0/1228687334 wait complete. 2020-11-16T22:30:36.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.489+0000 7f383e40f740 20 client.5684 trim_cache size 0 max 16384 2020-11-16T22:30:36.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 Processor -- start 2020-11-16T22:30:36.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- start start 2020-11-16T22:30:36.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 0xe18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe1b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xd23e40 con 0xdcee40 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe707b0 con 0xb92bb0 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xe51d60 con 0xc007b0 2020-11-16T22:30:36.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39804/0 (socket says 172.21.15.181:39804) 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382cf19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe1b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 -- 172.21.15.181:0/2326967133 learned_addr learned my addr 172.21.15.181:0/2326967133 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382d71a700 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 0xe18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 -- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 msgr2=0xe1b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe1b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 -- 172.21.15.181:0/2326967133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 msgr2=0xe18910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 0xe18910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 -- 172.21.15.181:0/2326967133 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f4008290 con 0xc007b0 2020-11-16T22:30:36.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382d71a700 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 0xe18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:36.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f382df1b700 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f3820013730 tx=0x7f3820015000).ready entity=mon.2 client_cookie=dec9c2c5a5011440 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f3812ffd700 1 -- 172.21.15.181:0/2326967133 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000f840 con 0xc007b0 2020-11-16T22:30:36.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f3812ffd700 1 -- 172.21.15.181:0/2326967133 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820013e80 con 0xc007b0 2020-11-16T22:30:36.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f3812ffd700 1 -- 172.21.15.181:0/2326967133 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382001f690 con 0xc007b0 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 msgr2=0xe1b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f3820013730 tx=0x7f3820015000).stop 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- 172.21.15.181:0/2326967133 shutdown_connections 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe1b5c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xdcee40 0xe18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 --2- 172.21.15.181:0/2326967133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xb92bb0 0xe1b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- 172.21.15.181:0/2326967133 >> 172.21.15.181:0/2326967133 conn(0x7f37f4002320 msgr2=0xe6b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- 172.21.15.181:0/2326967133 shutdown_connections 2020-11-16T22:30:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.509+0000 7f383e40f740 1 -- 172.21.15.181:0/2326967133 wait complete. 2020-11-16T22:30:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- start start 2020-11-16T22:30:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 Processor -- start 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe50300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7f690 con 0xb92bb0 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe6f6e0 con 0xdcee40 2020-11-16T22:30:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xe6f980 con 0xc007b0 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60204/0 (socket says 172.21.15.181:60204) 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 10 client.0 ll_register_callbacks cb 0xe4fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 -- 172.21.15.181:0/2004972134 learned_addr learned my addr 172.21.15.181:0/2004972134 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382cf19700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382d71a700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe50300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 -- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 msgr2=0xe51230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe51230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 -- 172.21.15.181:0/2004972134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 msgr2=0xe50300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe50300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 -- 172.21.15.181:0/2004972134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xbb0720 con 0xdcee40 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382d71a700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe50300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f382df1b700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f38200092c0 tx=0x7f3820024000).ready entity=mon.1 client_cookie=130b1d753149b801 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f37fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820028600 con 0xdcee40 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3820018040 con 0xdcee40 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc43f50 con 0xdcee40 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3820034e80 con 0xdcee40 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 10 client.6091 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0xbbbe00 con 0xdcee40 2020-11-16T22:30:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.513+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc43f50 con 0xdcee40 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f382003a020 con 0xdcee40 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 10 client.6091 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 10 client.6091 resolve_mds: resolved ID 'a' to GID 5599 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 20 client.6091 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1efd0 0xe136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 4 client.6091 mds_command: new command op to 5599 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:30:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 --> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xc211d0 con 0xe1efd0 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382d71a700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1efd0 0xe136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f37fb7fe700 10 client.6091 ms_handle_connect on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382d71a700 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1efd0 0xe136c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=56d69242e6dff00a server_cookie=742cf152947cb8c2 in_seq=0 out_seq=0 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f37fb7fe700 1 -- 172.21.15.181:0/2004972134 <== mds.2 v2:172.21.15.181:6834/3960101400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xc211d0 con 0xe1efd0 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f37fb7fe700 10 client.6091 handle_command_reply: tid=0 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 client.6091 shutdown 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 msgr2=0xe21570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f38200092c0 tx=0x7f3820024000).stop 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 shutdown_connections 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:6834/3960101400,v1:172.21.15.181:6835/3960101400] conn(0xe1efd0 0xe136c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xdcee40 0xe21570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xb92bb0 0xe50300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f37fb7fe700 0 client.6091 ms_handle_reset on v2:172.21.15.181:6834/3960101400 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 --2- 172.21.15.181:0/2004972134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc007b0 0xe51230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 >> 172.21.15.181:0/2004972134 conn(0x7f37f4002320 msgr2=0xe6b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 shutdown_connections 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 1 -- 172.21.15.181:0/2004972134 wait complete. 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f383e40f740 20 client.6091 trim_cache size 0 max 16384 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:30:36.506 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:30:36.507 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:36.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 msgr2=0x7f37fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 0x7f37fc057520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f38280cd2c0 tx=0x7f38200091d0).stop 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 msgr2=0x7f3828162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f382400eba0 tx=0x7f3824006df0).stop 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 shutdown_connections 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f37fc055060 0x7f37fc057520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3828163970 0x7f382816f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38280600f0 0x7f38280b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 --2- 172.21.15.181:0/1884330420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280cf390 0x7f3828162230 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 >> 172.21.15.181:0/1884330420 conn(0x7f38280cfe90 msgr2=0x7f38280d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 shutdown_connections 2020-11-16T22:30:36.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.517+0000 7f382e71c700 1 -- 172.21.15.181:0/1884330420 wait complete. 2020-11-16T22:30:36.513 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.653+0000 7fc81d80a700 1 Processor -- start 2020-11-16T22:30:36.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.653+0000 7fc81d80a700 1 -- start start 2020-11-16T22:30:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8180612a0 0x7fc818061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818062b70 con 0x7fc818065fa0 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818061b70 con 0x7fc818066420 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81815db10 con 0x7fc8180612a0 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc816ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8180612a0 0x7fc818061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc8177fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc818066420 0x7fc8180ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46284/0 (socket says 172.21.15.181:46284) 2020-11-16T22:30:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc8177fe700 1 -- 172.21.15.181:0/2564846091 learned_addr learned my addr 172.21.15.181:0/2564846091 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc816ffd700 1 -- 172.21.15.181:0/2564846091 >> v1:172.21.15.181:6789/0 conn(0x7fc818066420 legacy=0x7fc8180ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc816ffd700 1 -- 172.21.15.181:0/2564846091 >> v1:172.21.15.175:6789/0 conn(0x7fc818065fa0 legacy=0x7fc81815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc816ffd700 1 -- 172.21.15.181:0/2564846091 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180cd790 con 0x7fc8180612a0 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc816ffd700 1 --2- 172.21.15.181:0/2564846091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8180612a0 0x7fc818061680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc808009c90 tx=0x7fc8080091d0).ready entity=mon.1 client_cookie=b767fde13afc9101 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc815ffb700 1 -- 172.21.15.181:0/2564846091 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800bbc0 con 0x7fc8180612a0 2020-11-16T22:30:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc815ffb700 1 -- 172.21.15.181:0/2564846091 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800bd20 con 0x7fc8180612a0 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc815ffb700 1 -- 172.21.15.181:0/2564846091 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808011610 con 0x7fc8180612a0 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- 172.21.15.181:0/2564846091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8180612a0 msgr2=0x7fc818061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 --2- 172.21.15.181:0/2564846091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8180612a0 0x7fc818061680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc808009c90 tx=0x7fc8080091d0).stop 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- 172.21.15.181:0/2564846091 shutdown_connections 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 --2- 172.21.15.181:0/2564846091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8180612a0 0x7fc818061680 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.657+0000 7fc81d80a700 1 -- 172.21.15.181:0/2564846091 >> 172.21.15.181:0/2564846091 conn(0x7fc8180cf890 msgr2=0x7fc8180cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.661+0000 7fc81d80a700 1 -- 172.21.15.181:0/2564846091 shutdown_connections 2020-11-16T22:30:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.661+0000 7fc81d80a700 1 -- 172.21.15.181:0/2564846091 wait complete. 2020-11-16T22:30:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.661+0000 7fc81d80a700 1 Processor -- start 2020-11-16T22:30:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.661+0000 7fc81d80a700 1 -- start start 2020-11-16T22:30:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818065fa0 0x7fc81814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 0x7fc8180a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180618b0 con 0x7fc818065fa0 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181737d0 con 0x7fc818066420 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81816f4c0 con 0x7fc8180612a0 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39824/0 (socket says 172.21.15.181:39824) 2020-11-16T22:30:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 -- 172.21.15.181:0/3086137120 learned_addr learned my addr 172.21.15.181:0/3086137120 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc8177fe700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 0x7fc8180a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc8167fc700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818065fa0 0x7fc81814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 -- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 msgr2=0x7fc8180a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 0x7fc8180a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 -- 172.21.15.181:0/3086137120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818065fa0 msgr2=0x7fc81814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818065fa0 0x7fc81814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 -- 172.21.15.181:0/3086137120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81816f760 con 0x7fc8180612a0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc8177fe700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 0x7fc8180a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc816ffd700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc808009fd0 tx=0x7fc808016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801c6c0 con 0x7fc8180612a0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80801c820 con 0x7fc8180612a0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81816fa00 con 0x7fc8180612a0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801cba0 con 0x7fc8180612a0 2020-11-16T22:30:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81816ff40 con 0x7fc8180612a0 2020-11-16T22:30:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.665+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818069eb0 con 0x7fc8180612a0 2020-11-16T22:30:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.669+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc808017070 con 0x7fc8180612a0 2020-11-16T22:30:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.669+0000 7fc7feffd700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 0x7fc7f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.669+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc808070200 con 0x7fc8180612a0 2020-11-16T22:30:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.669+0000 7fc8167fc700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 0x7fc7f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.669+0000 7fc8167fc700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 0x7fc7f4057520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fc80c003300 tx=0x7fc80c007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:36.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.677+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc8080464a0 con 0x7fc8180612a0 2020-11-16T22:30:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.877+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc81805d4e0 con 0x7fc8180612a0 2020-11-16T22:30:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc7feffd700 1 -- 172.21.15.181:0/3086137120 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7fc808015090 con 0x7fc8180612a0 2020-11-16T22:30:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:36.862 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 msgr2=0x7fc7f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 0x7fc7f4057520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fc80c003300 tx=0x7fc80c007b90).stop 2020-11-16T22:30:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 msgr2=0x7fc81814e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc808009fd0 tx=0x7fc808016000).stop 2020-11-16T22:30:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 shutdown_connections 2020-11-16T22:30:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7f4055060 0x7fc7f4057520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc818066420 0x7fc8180a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.881+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8180612a0 0x7fc81814e800 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.885+0000 7fc81d80a700 1 --2- 172.21.15.181:0/3086137120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc818065fa0 0x7fc81814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.885+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 >> 172.21.15.181:0/3086137120 conn(0x7fc8180cf890 msgr2=0x7fc8180d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.885+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 shutdown_connections 2020-11-16T22:30:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:36.885+0000 7fc81d80a700 1 -- 172.21.15.181:0/3086137120 wait complete. 2020-11-16T22:30:36.872 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:36.883 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:30:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.029+0000 7ff261318700 1 Processor -- start 2020-11-16T22:30:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.029+0000 7ff261318700 1 -- start start 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff25c0612a0 0x7ff25c061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff25c15fa10 0x7ff25c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c061b70 con 0x7ff25c0ce750 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c15cda0 con 0x7ff25c0612a0 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c15d040 con 0x7ff25c15fa10 2020-11-16T22:30:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff25c15fa10 0x7ff25c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff25c15fa10 0x7ff25c15fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60226/0 (socket says 172.21.15.181:60226) 2020-11-16T22:30:37.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 -- 172.21.15.181:0/1937000934 learned_addr learned my addr 172.21.15.181:0/1937000934 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25affd700 1 --2- 172.21.15.181:0/1937000934 >> v2:172.21.15.175:3300/0 conn(0x7ff25c0612a0 0x7ff25c061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 -- 172.21.15.181:0/1937000934 >> v2:172.21.15.175:3300/0 conn(0x7ff25c0612a0 msgr2=0x7ff25c061680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/1937000934 >> v2:172.21.15.175:3300/0 conn(0x7ff25c0612a0 0x7ff25c061680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 -- 172.21.15.181:0/1937000934 >> v1:172.21.15.175:6789/0 conn(0x7ff25c0ce750 legacy=0x7ff25c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 -- 172.21.15.181:0/1937000934 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c15d2e0 con 0x7ff25c15fa10 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/1937000934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c15fea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7ff25c063560 tx=0x7ff254006eb0).ready entity=mon.1 client_cookie=a02b62737fee415f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff259ffb700 1 -- 172.21.15.181:0/1937000934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254014070 con 0x7ff25c15fa10 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff259ffb700 1 -- 172.21.15.181:0/1937000934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254009810 con 0x7ff25c15fa10 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff259ffb700 1 -- 172.21.15.181:0/1937000934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254009b90 con 0x7ff25c15fa10 2020-11-16T22:30:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.033+0000 7ff261318700 1 -- 172.21.15.181:0/1937000934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 msgr2=0x7ff25c15fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 --2- 172.21.15.181:0/1937000934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c15fea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7ff25c063560 tx=0x7ff254006eb0).stop 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 -- 172.21.15.181:0/1937000934 shutdown_connections 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 --2- 172.21.15.181:0/1937000934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c15fea0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 --2- 172.21.15.181:0/1937000934 >> v2:172.21.15.175:3300/0 conn(0x7ff25c0612a0 0x7ff25c061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 -- 172.21.15.181:0/1937000934 >> 172.21.15.181:0/1937000934 conn(0x7ff25c0cf890 msgr2=0x7ff25c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 -- 172.21.15.181:0/1937000934 shutdown_connections 2020-11-16T22:30:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 -- 172.21.15.181:0/1937000934 wait complete. 2020-11-16T22:30:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 Processor -- start 2020-11-16T22:30:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.037+0000 7ff261318700 1 -- start start 2020-11-16T22:30:37.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 0x7ff25c16d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 0x7ff25c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c09e4a0 con 0x7ff25c0ce750 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c16d970 con 0x7ff25c15fa10 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c16dc10 con 0x7ff25c0612a0 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 0x7ff25c16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 0x7ff25c16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39838/0 (socket says 172.21.15.181:39838) 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25affd700 1 -- 172.21.15.181:0/630325884 learned_addr learned my addr 172.21.15.181:0/630325884 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 0x7ff25c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 -- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 msgr2=0x7ff25c16d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 0x7ff25c16d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 -- 172.21.15.181:0/630325884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 msgr2=0x7ff25c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 0x7ff25c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 -- 172.21.15.181:0/630325884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c16deb0 con 0x7ff25c15fa10 2020-11-16T22:30:37.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 0x7ff25c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c09ee10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7ff254013000 tx=0x7ff254016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 -- 172.21.15.181:0/630325884 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254010e10 con 0x7ff25c15fa10 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c09e6d0 con 0x7ff25c15fa10 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 -- 172.21.15.181:0/630325884 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25401c650 con 0x7ff25c15fa10 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 -- 172.21.15.181:0/630325884 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25401c9f0 con 0x7ff25c15fa10 2020-11-16T22:30:37.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c09fa20 con 0x7ff25c15fa10 2020-11-16T22:30:37.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 -- 172.21.15.181:0/630325884 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff25400e040 con 0x7ff25c15fa10 2020-11-16T22:30:37.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 0x7ff238057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff252ffd700 1 -- 172.21.15.181:0/630325884 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff25406fb70 con 0x7ff25c15fa10 2020-11-16T22:30:37.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25affd700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 0x7ff238057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.041+0000 7ff25affd700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 0x7ff238057520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7ff25c160990 tx=0x7ff24c0091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.045+0000 7ff26fda9740 1 Processor -- start 2020-11-16T22:30:37.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.045+0000 7ff26fda9740 1 -- start start 2020-11-16T22:30:37.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2ace530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x29224b0 con 0x280abb0 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x29d6930 con 0x2a91470 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x28966f0 con 0x28787b0 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60240/0 (socket says 172.21.15.181:60240) 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 -- 172.21.15.181:0/3275889004 learned_addr learned my addr 172.21.15.181:0/3275889004 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2ace530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 -- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 msgr2=0x2a8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 -- 172.21.15.181:0/3275889004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 msgr2=0x2ace530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2ace530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 -- 172.21.15.181:0/3275889004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28915a0 con 0x2a91470 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2ace530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7ff254049760 tx=0x7ff254012000).ready entity=mon.1 client_cookie=ea282499ba58e331 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff233fff700 1 -- 172.21.15.181:0/3275889004 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254035070 con 0x2a91470 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff233fff700 1 -- 172.21.15.181:0/3275889004 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25401cda0 con 0x2a91470 2020-11-16T22:30:37.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff233fff700 1 -- 172.21.15.181:0/3275889004 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25406de60 con 0x2a91470 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 msgr2=0x2aebc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7ff254049760 tx=0x7ff254012000).stop 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- 172.21.15.181:0/3275889004 shutdown_connections 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a91470 0x2aebc90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3275889004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2ace530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- 172.21.15.181:0/3275889004 >> 172.21.15.181:0/3275889004 conn(0x2a46e40 msgr2=0x2aeb580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- 172.21.15.181:0/3275889004 shutdown_connections 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- 172.21.15.181:0/3275889004 wait complete. 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 -- start start 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.049+0000 7ff26fda9740 1 Processor -- start 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2af6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 0x2ae6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c36e0 con 0x280abb0 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c3820 con 0x28787b0 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x29c4a90 con 0x2a91470 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 10 client.0 ll_register_callbacks cb 0x29b9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60244/0 (socket says 172.21.15.181:60244) 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25a7fc700 1 -- 172.21.15.181:0/2786883849 learned_addr learned my addr 172.21.15.181:0/2786883849 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 0x2ae6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25affd700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2af6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 -- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 msgr2=0x2ab8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 -- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 msgr2=0x2af6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2af6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 -- 172.21.15.181:0/2786883849 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x29b4280 con 0x2a91470 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 0x2ae6cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ff254032000 tx=0x7ff254031000).ready entity=mon.2 client_cookie=484ee374c533adf1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254035040 con 0x2a91470 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff2540461c0 con 0x2a91470 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2a8be10 con 0x2a91470 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff254046380 con 0x2a91470 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 10 client.5702 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x27e9fd0 con 0x2a91470 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2a8be10 con 0x2a91470 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7ff2540458a0 con 0x2a91470 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 10 client.5702 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 10 client.5702 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 20 client.5702 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2af8b90 0x2ae7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 4 client.5702 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x28b4c50 con 0x2af8b90 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25affd700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2af8b90 0x2ae7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff2317fa700 10 client.5702 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.053+0000 7ff25affd700 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2af8b90 0x2ae7dd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9a3b4af96c83c25e server_cookie=7b16758712a3d486 in_seq=0 out_seq=0 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff2317fa700 1 -- 172.21.15.181:0/2786883849 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x28b4c50 con 0x2af8b90 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff2317fa700 10 client.5702 handle_command_reply: tid=0 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 client.5702 shutdown 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 msgr2=0x2ae6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 0x2ae6cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ff254032000 tx=0x7ff254031000).stop 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 shutdown_connections 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2af8b90 0x2ae7dd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a91470 0x2ae6cf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2af6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 --2- 172.21.15.181:0/2786883849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28787b0 0x2ab8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 >> 172.21.15.181:0/2786883849 conn(0x2a46e40 msgr2=0x2abc340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff2317fa700 0 client.5702 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 shutdown_connections 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 1 -- 172.21.15.181:0/2786883849 wait complete. 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.057+0000 7ff26fda9740 20 client.5702 trim_cache size 0 max 16384 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 Processor -- start 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- start start 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 0x2ae81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x299be40 con 0x2a46e40 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2af5fe0 con 0x280abb0 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2af9bf0 con 0x28787b0 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60250/0 (socket says 172.21.15.181:60250) 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39862/0 (socket says 172.21.15.181:39862) 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 -- 172.21.15.181:0/3117996537 learned_addr learned my addr 172.21.15.181:0/3117996537 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25affd700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 0x2ae81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 -- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 msgr2=0x2a8ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 -- 172.21.15.181:0/3117996537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 msgr2=0x2ae81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 0x2ae81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 -- 172.21.15.181:0/3117996537 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280081d0 con 0x280abb0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25affd700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 0x2ae81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ff244003960 tx=0x7ff244009040).ready entity=mon.1 client_cookie=498470f40617db3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff250ff9700 1 -- 172.21.15.181:0/3117996537 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244007030 con 0x280abb0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff250ff9700 1 -- 172.21.15.181:0/3117996537 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244003030 con 0x280abb0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff250ff9700 1 -- 172.21.15.181:0/3117996537 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440073a0 con 0x280abb0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 msgr2=0x2af9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ff244003960 tx=0x7ff244009040).stop 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- 172.21.15.181:0/3117996537 shutdown_connections 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2a8ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2a46e40 0x2ae81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 --2- 172.21.15.181:0/3117996537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x280abb0 0x2af9700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- 172.21.15.181:0/3117996537 >> 172.21.15.181:0/3117996537 conn(0x7ff228002320 msgr2=0x2af7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- 172.21.15.181:0/3117996537 shutdown_connections 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.077+0000 7ff26fda9740 1 -- 172.21.15.181:0/3117996537 wait complete. 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 -- start start 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 Processor -- start 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2a90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a46e40 0x2ae5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2afa000 con 0x280abb0 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae6c50 con 0x2a46e40 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ae5f50 con 0x28787b0 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff26fda9740 10 client.0 ll_register_callbacks cb 0x2ad0430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39868/0 (socket says 172.21.15.181:39868) 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 -- 172.21.15.181:0/830710294 learned_addr learned my addr 172.21.15.181:0/830710294 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25affd700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2a90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25b7fe700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a46e40 0x2ae5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 -- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a46e40 msgr2=0x2ae5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a46e40 0x2ae5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 -- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 msgr2=0x2a90950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2a90950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.081+0000 7ff25a7fc700 1 -- 172.21.15.181:0/830710294 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2828720 con 0x28787b0 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff25a7fc700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ff244019000 tx=0x7ff244009040).ready entity=mon.2 client_cookie=e0d9459e2d71b272 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244003d80 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff24401eae0 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff25401cda0 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff244006700 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 10 client.5708 fetch_fsmap learned FSMap version 56 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x2833e00 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25401cda0 con 0x28787b0 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 56) v1 ==== 2611+0+0 (secure 0 0 0) 0x7ff244020070 con 0x28787b0 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 10 client.5708 fetch_fsmap finished waiting for FSMap version 56 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 10 client.5708 resolve_mds: resolved ID 'i' to GID 5614 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 20 client.5708 populate_metadata read hostname 'smithi181' 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2a91470 0x2acc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 4 client.5708 mds_command: new command op to 5614 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 --> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x28bbf50 con 0x2a91470 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff25affd700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2a91470 0x2acc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff25affd700 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2a91470 0x2acc990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d77c1c804c796c2b server_cookie=9f4dfc0374831276 in_seq=0 out_seq=0 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 10 client.5708 ms_handle_connect on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 1 -- 172.21.15.181:0/830710294 <== mds.0 v2:172.21.15.175:6836/2753478724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x28bbf50 con 0x2a91470 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 10 client.5708 handle_command_reply: tid=0 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 client.5708 shutdown 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 msgr2=0x2ae5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ff244019000 tx=0x7ff244009040).stop 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 shutdown_connections 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:6836/2753478724,v1:172.21.15.175:6837/2753478724] conn(0x2a91470 0x2acc990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a46e40 0x2ae5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff2317fa700 0 client.5708 ms_handle_reset on v2:172.21.15.175:6836/2753478724 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x280abb0 0x2a90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 --2- 172.21.15.181:0/830710294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x28787b0 0x2ae5570 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 >> 172.21.15.181:0/830710294 conn(0x7ff228002320 msgr2=0x2ace430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 shutdown_connections 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 1 -- 172.21.15.181:0/830710294 wait complete. 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff26fda9740 20 client.5708 trim_cache size 0 max 16384 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 msgr2=0x7ff238057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 0x7ff238057520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7ff25c160990 tx=0x7ff24c0091d0).stop 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 msgr2=0x7ff25c09ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c09ee10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7ff254013000 tx=0x7ff254016000).stop 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 shutdown_connections 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff238055060 0x7ff238057520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c15fa10 0x7ff25c09ee10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c0612a0 0x7ff25c16d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 --2- 172.21.15.181:0/630325884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c0ce750 0x7ff25c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.085+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 >> 172.21.15.181:0/630325884 conn(0x7ff25c0cf890 msgr2=0x7ff25c063590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.089+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 shutdown_connections 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.089+0000 7ff261318700 1 -- 172.21.15.181:0/630325884 wait complete. 2020-11-16T22:30:37.087 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 Processor -- start 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 -- start start 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 0x7f99e8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99e8161860 0x7f99e8062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e80c8db0 con 0x7f99e8164e10 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e80c8330 con 0x7f99e8161860 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ef56a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e80c85d0 con 0x7f99e8161480 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99e8161860 0x7f99e8062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99edd67700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f99e8164e10 0x7f99e8062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46336/0 (socket says 172.21.15.181:46336) 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99edd67700 1 -- 172.21.15.181:0/1091395975 learned_addr learned my addr 172.21.15.181:0/1091395975 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 --2- 172.21.15.181:0/1091395975 >> v2:172.21.15.181:3300/0 conn(0x7f99e8161860 0x7f99e8062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60268/0 (socket says 172.21.15.181:60268) 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ee568700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 0x7f99e8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 -- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 msgr2=0x7f99e8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 0x7f99e8164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ed566700 1 -- 172.21.15.181:0/1091395975 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261553505 0 0) 0x7f99e80c8db0 con 0x7f99e8164e10 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 -- 172.21.15.181:0/1091395975 >> v1:172.21.15.181:6789/0 conn(0x7f99e8164e10 legacy=0x7f99e8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 -- 172.21.15.181:0/1091395975 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80c8870 con 0x7f99e8161860 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ee568700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 0x7f99e8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99eed69700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161860 0x7f99e8062960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f99e000b9a0 tx=0x7f99e0006df0).ready entity=mon.1 client_cookie=8671f6af2e2f3109 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ed566700 1 -- 172.21.15.181:0/1091395975 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00097b0 con 0x7f99e8161860 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ed566700 1 -- 172.21.15.181:0/1091395975 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0009910 con 0x7f99e8161860 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.233+0000 7f99ed566700 1 -- 172.21.15.181:0/1091395975 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0009c90 con 0x7f99e8161860 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161860 msgr2=0x7f99e8062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161860 0x7f99e8062960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f99e000b9a0 tx=0x7f99e0006df0).stop 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- 172.21.15.181:0/1091395975 shutdown_connections 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161860 0x7f99e8062960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 --2- 172.21.15.181:0/1091395975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161480 0x7f99e8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- 172.21.15.181:0/1091395975 >> 172.21.15.181:0/1091395975 conn(0x7f99e80d2dc0 msgr2=0x7f99e80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- 172.21.15.181:0/1091395975 shutdown_connections 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- 172.21.15.181:0/1091395975 wait complete. 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 Processor -- start 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.237+0000 7f99ef56a700 1 -- start start 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161860 0x7f99e81605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 0x7f99e8160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8149fd0 con 0x7f99e8177140 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8174b50 con 0x7f99e8161480 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8177b60 con 0x7f99e8161860 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99edd67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161860 0x7f99e81605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60272/0 (socket says 172.21.15.181:60272) 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 -- 172.21.15.181:0/3022672656 learned_addr learned my addr 172.21.15.181:0/3022672656 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99eed69700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 0x7f99e8160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 -- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161860 msgr2=0x7f99e81605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161860 0x7f99e81605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 -- 172.21.15.181:0/3022672656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 msgr2=0x7f99e8160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 0x7f99e8160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 -- 172.21.15.181:0/3022672656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8177620 con 0x7f99e8161480 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99eed69700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 0x7f99e8160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ee568700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f99d800ef50 tx=0x7f99d8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800cb30 con 0x7f99e8161480 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d800cc90 con 0x7f99e8161480 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8012600 con 0x7f99e8161480 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e81778c0 con 0x7f99e8161480 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e80b7130 con 0x7f99e8161480 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e80b6cd0 con 0x7f99e8161480 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.241+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f99d8012970 con 0x7f99e8161480 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.245+0000 7f99e67fc700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 0x7f99c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.245+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99d806e3e0 con 0x7f99e8161480 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.245+0000 7f99edd67700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 0x7f99c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.245+0000 7f99edd67700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 0x7f99c4057520 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f99e8164430 tx=0x7f99d0007ce0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.249+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f99d8044640 con 0x7f99e8161480 2020-11-16T22:30:37.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.449+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e8044f60 con 0x7f99e8161480 2020-11-16T22:30:37.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99e67fc700 1 -- 172.21.15.181:0/3022672656 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7f99d8018270 con 0x7f99e8161480 2020-11-16T22:30:37.433 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:37.433 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 msgr2=0x7f99c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 0x7f99c4057520 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f99e8164430 tx=0x7f99d0007ce0).stop 2020-11-16T22:30:37.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 msgr2=0x7f99e8176c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f99d800ef50 tx=0x7f99d8006df0).stop 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 shutdown_connections 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c4055060 0x7f99c4057520 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e8177140 0x7f99e8160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99e8161480 0x7f99e8176c50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 --2- 172.21.15.181:0/3022672656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99e8161860 0x7f99e81605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 >> 172.21.15.181:0/3022672656 conn(0x7f99e80d2dc0 msgr2=0x7f99e8063220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 shutdown_connections 2020-11-16T22:30:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.453+0000 7f99ef56a700 1 -- 172.21.15.181:0/3022672656 wait complete. 2020-11-16T22:30:37.438 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:37.452 DEBUG:tasks.cephfs.test_multimds_misc:current active=['i', 'b', 'a'] 2020-11-16T22:30:37.453 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:30:37.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 Processor -- start 2020-11-16T22:30:37.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 -- start start 2020-11-16T22:30:37.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 0x7f9128062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 0x7f91280644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280c8db0 con 0x7f9128161480 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9128165030 con 0x7f9128062740 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912ec85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280c8330 con 0x7f9128161860 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912dc83700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9128161480 0x7f9128164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46350/0 (socket says 172.21.15.181:46350) 2020-11-16T22:30:37.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912dc83700 1 -- 172.21.15.181:0/124201689 learned_addr learned my addr 172.21.15.181:0/124201689 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912e484700 1 --2- 172.21.15.181:0/124201689 >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 0x7f91280644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 --2- 172.21.15.181:0/124201689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 0x7f9128062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 -- 172.21.15.181:0/124201689 >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 msgr2=0x7f91280644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 --2- 172.21.15.181:0/124201689 >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 0x7f91280644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 -- 172.21.15.181:0/124201689 >> v1:172.21.15.181:6789/0 conn(0x7f9128161480 legacy=0x7f9128164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 -- 172.21.15.181:0/124201689 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280c85d0 con 0x7f9128161860 2020-11-16T22:30:37.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912e484700 1 --2- 172.21.15.181:0/124201689 >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 0x7f91280644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:37.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912d482700 1 --2- 172.21.15.181:0/124201689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 0x7f9128062250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f911c00b9a0 tx=0x7f911c006df0).ready entity=mon.2 client_cookie=5aaca0208bdb20b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.593+0000 7f912cc81700 1 -- 172.21.15.181:0/124201689 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0097b0 con 0x7f9128161860 2020-11-16T22:30:37.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912cc81700 1 -- 172.21.15.181:0/124201689 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c009910 con 0x7f9128161860 2020-11-16T22:30:37.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912cc81700 1 -- 172.21.15.181:0/124201689 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c009c90 con 0x7f9128161860 2020-11-16T22:30:37.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 -- 172.21.15.181:0/124201689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 msgr2=0x7f9128062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 --2- 172.21.15.181:0/124201689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 0x7f9128062250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f911c00b9a0 tx=0x7f911c006df0).stop 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 -- 172.21.15.181:0/124201689 shutdown_connections 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 --2- 172.21.15.181:0/124201689 >> v2:172.21.15.181:3300/0 conn(0x7f9128062740 0x7f91280644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 --2- 172.21.15.181:0/124201689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128161860 0x7f9128062250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 -- 172.21.15.181:0/124201689 >> 172.21.15.181:0/124201689 conn(0x7f91280d2dc0 msgr2=0x7f91280ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 -- 172.21.15.181:0/124201689 shutdown_connections 2020-11-16T22:30:37.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.597+0000 7f912ec85700 1 -- 172.21.15.181:0/124201689 wait complete. 2020-11-16T22:30:37.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 Processor -- start 2020-11-16T22:30:37.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 -- start start 2020-11-16T22:30:37.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 0x7f91280ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 0x7f91280ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b82a0 con 0x7f9128161480 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b6e90 con 0x7f9128161860 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912ec85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b7130 con 0x7f9128062740 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912dc83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 0x7f91280ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912dc83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39894/0 (socket says 172.21.15.181:39894) 2020-11-16T22:30:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912dc83700 1 -- 172.21.15.181:0/3126705499 learned_addr learned my addr 172.21.15.181:0/3126705499 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912d482700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 0x7f91280ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 -- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 msgr2=0x7f91280b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 -- 172.21.15.181:0/3126705499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 msgr2=0x7f91280ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 0x7f91280ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 -- 172.21.15.181:0/3126705499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280b73d0 con 0x7f9128161860 2020-11-16T22:30:37.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912d482700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 0x7f91280ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:37.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912dc83700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f912e484700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 0x7f91280ce1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f91200092c0 tx=0x7f91200091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.601+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000cc70 con 0x7f9128161860 2020-11-16T22:30:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912000cdd0 con 0x7f9128161860 2020-11-16T22:30:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280b7670 con 0x7f9128161860 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120012620 con 0x7f9128161860 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280cad90 con 0x7f9128161860 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9120022070 con 0x7f9128161860 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128044f60 con 0x7f9128161860 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f9125ffb700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 0x7f9100057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f912006e5a0 con 0x7f9128161860 2020-11-16T22:30:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f912dc83700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 0x7f9100057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.605+0000 7f912dc83700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 0x7f9100057520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9118007310 tx=0x7f9118007c30).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.613+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9120044800 con 0x7f9128161860 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.813+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f91280671e0 con 0x7f9128161860 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f9125ffb700 1 -- 172.21.15.181:0/3126705499 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1783 (secure 0 0 0) 0x7f9120016710 con 0x7f9128161860 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":242,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":115880,"bytes_used":49594368,"bytes_avail":773044518912,"bytes_total":773094113280,"write_bytes_sec":426,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":56,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"a","status":"up:active","gid":5599},{"filesystem_id":3,"rank":1,"name":"b","status":"up:active","gid":5605},{"filesystem_id":3,"rank":0,"name":"i","status":"up:active","gid":5614}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2020-11-16T22:30:36.741875+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 msgr2=0x7f9100057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 0x7f9100057520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9118007310 tx=0x7f9118007c30).stop 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 msgr2=0x7f91280ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 0x7f91280ce1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f91200092c0 tx=0x7f91200091d0).stop 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 shutdown_connections 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9100055060 0x7f9100057520 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9128161860 0x7f91280ce1a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9128062740 0x7f91280b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 --2- 172.21.15.181:0/3126705499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9128161480 0x7f91280ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 >> 172.21.15.181:0/3126705499 conn(0x7f91280d2dc0 msgr2=0x7f91280cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 shutdown_connections 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:37.817+0000 7f912ec85700 1 -- 172.21.15.181:0/3126705499 wait complete. 2020-11-16T22:30:37.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]', 'i': 'idle+waiting paths [/d1/d2/d3]'} 2020-11-16T22:30:38.813 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:38.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 Processor -- start 2020-11-16T22:30:38.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 -- start start 2020-11-16T22:30:38.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f23400629b0 0x7f2340080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:38.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2340160e40 con 0x7f234015f5b0 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2340160ba0 con 0x7f23400615f0 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f23449f7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23401609a0 con 0x7f23400629b0 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f2335d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f23400615f0 0x7f23400622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46364/0 (socket says 172.21.15.181:46364) 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f2335d9b700 1 -- 172.21.15.181:0/1647968248 learned_addr learned my addr 172.21.15.181:0/1647968248 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.957+0000 7f233ed9d700 1 --2- 172.21.15.181:0/1647968248 >> v2:172.21.15.175:3300/0 conn(0x7f23400629b0 0x7f2340080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233dd9b700 1 -- 172.21.15.181:0/1647968248 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942699730 0 0) 0x7f2340160ba0 con 0x7f23400615f0 2020-11-16T22:30:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233dd9b700 1 -- 172.21.15.181:0/1647968248 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2320003240 con 0x7f23400615f0 2020-11-16T22:30:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233ed9d700 1 -- 172.21.15.181:0/1647968248 >> v1:172.21.15.181:6789/0 conn(0x7f23400615f0 legacy=0x7f23400622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233ed9d700 1 -- 172.21.15.181:0/1647968248 >> v1:172.21.15.175:6789/0 conn(0x7f234015f5b0 legacy=0x7f23400d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233ed9d700 1 -- 172.21.15.181:0/1647968248 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340060970 con 0x7f23400629b0 2020-11-16T22:30:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233ed9d700 1 --2- 172.21.15.181:0/1647968248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23400629b0 0x7f2340080fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2338009c90 tx=0x7f23380091d0).ready entity=mon.0 client_cookie=d2b1e77c70e50819 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:38.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233dd9b700 1 -- 172.21.15.181:0/1647968248 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233800ba70 con 0x7f23400629b0 2020-11-16T22:30:38.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233dd9b700 1 -- 172.21.15.181:0/1647968248 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233800bbd0 con 0x7f23400629b0 2020-11-16T22:30:38.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f233dd9b700 1 -- 172.21.15.181:0/1647968248 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338011400 con 0x7f23400629b0 2020-11-16T22:30:38.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- 172.21.15.181:0/1647968248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23400629b0 msgr2=0x7f2340080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:38.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 --2- 172.21.15.181:0/1647968248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23400629b0 0x7f2340080fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2338009c90 tx=0x7f23380091d0).stop 2020-11-16T22:30:38.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- 172.21.15.181:0/1647968248 shutdown_connections 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 --2- 172.21.15.181:0/1647968248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23400629b0 0x7f2340080fd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- 172.21.15.181:0/1647968248 >> 172.21.15.181:0/1647968248 conn(0x7f23400d69a0 msgr2=0x7f23400d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- 172.21.15.181:0/1647968248 shutdown_connections 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- 172.21.15.181:0/1647968248 wait complete. 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 Processor -- start 2020-11-16T22:30:38.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.961+0000 7f23449f7700 1 -- start start 2020-11-16T22:30:38.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:38.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23400629b0 0x7f2340145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:38.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 0x7f23401703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:38.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400d1360 con 0x7f234015f5b0 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400d1600 con 0x7f23400615f0 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23400d18a0 con 0x7f23400629b0 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2335d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23400629b0 0x7f2340145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60298/0 (socket says 172.21.15.181:60298) 2020-11-16T22:30:38.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 -- 172.21.15.181:0/2944610768 learned_addr learned my addr 172.21.15.181:0/2944610768 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233ed9d700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 0x7f23401703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 -- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23400629b0 msgr2=0x7f2340145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23400629b0 0x7f2340145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 -- 172.21.15.181:0/2944610768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 msgr2=0x7f23401703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 0x7f23401703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:38.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 -- 172.21.15.181:0/2944610768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400d1b40 con 0x7f23400615f0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233ed9d700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 0x7f23401703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f233e59c700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f233000c6e0 tx=0x7f2330007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2330007020 con 0x7f23400615f0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23400d1de0 con 0x7f23400615f0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2330007180 con 0x7f23400615f0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23400d0f80 con 0x7f23400615f0 2020-11-16T22:30:38.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233000e560 con 0x7f23400615f0 2020-11-16T22:30:38.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2330010070 con 0x7f23400615f0 2020-11-16T22:30:38.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23400d11d0 con 0x7f23400615f0 2020-11-16T22:30:38.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2336ffd700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 0x7f231c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:38.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.965+0000 7f2335d9b700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 0x7f231c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:38.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.969+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f233006d2e0 con 0x7f23400615f0 2020-11-16T22:30:38.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.969+0000 7f2335d9b700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 0x7f231c057520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f2340081be0 tx=0x7f2328009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:38.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:38.973+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f23300434b0 con 0x7f23400615f0 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2340069eb0 con 0x7f23400615f0 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f2336ffd700 1 -- 172.21.15.181:0/2944610768 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7f2330046ad0 con 0x7f23400615f0 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 msgr2=0x7f231c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 0x7f231c057520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f2340081be0 tx=0x7f2328009040).stop 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 msgr2=0x7f234015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f233000c6e0 tx=0x7f2330007bb0).stop 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 shutdown_connections 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f231c055060 0x7f231c057520 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f234015f5b0 0x7f23401703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23400615f0 0x7f234015d030 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 --2- 172.21.15.181:0/2944610768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23400629b0 0x7f2340145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 >> 172.21.15.181:0/2944610768 conn(0x7f23400d69a0 msgr2=0x7f23400622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 shutdown_connections 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.173+0000 7f23449f7700 1 -- 172.21.15.181:0/2944610768 wait complete. 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:39.169 DEBUG:tasks.cephfs.test_multimds_misc:current active=['i', 'b', 'a'] 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:30:39.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 Processor -- start 2020-11-16T22:30:39.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 -- start start 2020-11-16T22:30:39.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbcec065040 0x7fbcec1634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec160840 con 0x7fbcec060670 2020-11-16T22:30:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec160ae0 con 0x7fbcec060290 2020-11-16T22:30:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbcf15c2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0ced30 con 0x7fbcec065040 2020-11-16T22:30:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbceaffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbcec060290 0x7fbcec0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46380/0 (socket says 172.21.15.181:46380) 2020-11-16T22:30:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbceaffd700 1 -- 172.21.15.181:0/3177509881 learned_addr learned my addr 172.21.15.181:0/3177509881 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbceb7fe700 1 --2- 172.21.15.181:0/3177509881 >> v2:172.21.15.175:3300/0 conn(0x7fbcec065040 0x7fbcec1634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158844978 0 0) 0x7fbcec160ae0 con 0x7fbcec060290 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd8003240 con 0x7fbcec060290 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321763208 0 0) 0x7fbcec160840 con 0x7fbcec060670 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcec160ae0 con 0x7fbcec060670 2020-11-16T22:30:39.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.309+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2953580975 0 0) 0x7fbcd8003240 con 0x7fbcec060290 2020-11-16T22:30:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbceb7fe700 1 -- 172.21.15.181:0/3177509881 >> v1:172.21.15.181:6789/0 conn(0x7fbcec060290 legacy=0x7fbcec0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbceb7fe700 1 -- 172.21.15.181:0/3177509881 >> v1:172.21.15.175:6789/0 conn(0x7fbcec060670 legacy=0x7fbcec15fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbceb7fe700 1 -- 172.21.15.181:0/3177509881 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0cefd0 con 0x7fbcec065040 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbceb7fe700 1 --2- 172.21.15.181:0/3177509881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec065040 0x7fbcec1634b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbce4009c10 tx=0x7fbce40091d0).ready entity=mon.0 client_cookie=86d5ed77f25d628a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00014e0 con 0x7fbcec065040 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400b740 con 0x7fbcec065040 2020-11-16T22:30:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbce9ffb700 1 -- 172.21.15.181:0/3177509881 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0014e0 con 0x7fbcec065040 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- 172.21.15.181:0/3177509881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec065040 msgr2=0x7fbcec1634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/3177509881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec065040 0x7fbcec1634b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbce4009c10 tx=0x7fbce40091d0).stop 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- 172.21.15.181:0/3177509881 shutdown_connections 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/3177509881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec065040 0x7fbcec1634b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- 172.21.15.181:0/3177509881 >> 172.21.15.181:0/3177509881 conn(0x7fbcec0d3430 msgr2=0x7fbcec0d3810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- 172.21.15.181:0/3177509881 shutdown_connections 2020-11-16T22:30:39.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- 172.21.15.181:0/3177509881 wait complete. 2020-11-16T22:30:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 Processor -- start 2020-11-16T22:30:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.313+0000 7fbcf15c2700 1 -- start start 2020-11-16T22:30:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec060290 0x7fbcec145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 0x7fbcec0d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b7700 con 0x7fbcec060290 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b79a0 con 0x7fbcec060670 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec144e90 con 0x7fbcec065040 2020-11-16T22:30:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 0x7fbcec0d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbceaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec060290 0x7fbcec145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60314/0 (socket says 172.21.15.181:60314) 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 -- 172.21.15.181:0/309432470 learned_addr learned my addr 172.21.15.181:0/309432470 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 0x7fbcec0d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39926/0 (socket says 172.21.15.181:39926) 2020-11-16T22:30:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 -- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 msgr2=0x7fbcec0d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 0x7fbcec0d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 -- 172.21.15.181:0/309432470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec060290 msgr2=0x7fbcec145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec060290 0x7fbcec145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 -- 172.21.15.181:0/309432470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec145130 con 0x7fbcec060670 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcea7fc700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbce0001690 tx=0x7fbce00093e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0001bb0 con 0x7fbcec060670 2020-11-16T22:30:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0010070 con 0x7fbcec060670 2020-11-16T22:30:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0001d70 con 0x7fbcec060670 2020-11-16T22:30:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec1453d0 con 0x7fbcec060670 2020-11-16T22:30:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec1707d0 con 0x7fbcec060670 2020-11-16T22:30:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec160600 con 0x7fbcec060670 2020-11-16T22:30:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.317+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbce001c070 con 0x7fbcec060670 2020-11-16T22:30:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.321+0000 7fbcd2ffd700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 0x7fbcc8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.321+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbce006d1d0 con 0x7fbcec060670 2020-11-16T22:30:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.321+0000 7fbceaffd700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 0x7fbcc8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.321+0000 7fbceaffd700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 0x7fbcc8057550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fbcdc001610 tx=0x7fbcdc000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.329+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbcec160600 con 0x7fbcec060670 2020-11-16T22:30:39.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.533+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbcec160600 con 0x7fbcec060670 2020-11-16T22:30:39.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.533+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/309432470 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1736 (secure 0 0 0) 0x7fbcec160600 con 0x7fbcec060670 2020-11-16T22:30:39.516 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":244,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":117477,"bytes_used":49659904,"bytes_avail":773044453376,"bytes_total":773094113280,"write_bytes_sec":597,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":56,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"a","status":"up:active","gid":5599},{"filesystem_id":3,"rank":1,"name":"b","status":"up:active","gid":5605},{"filesystem_id":3,"rank":0,"name":"i","status":"up:active","gid":5614}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2020-11-16T22:30:38.742526+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:30:39.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 msgr2=0x7fbcc8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 0x7fbcc8057550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fbcdc001610 tx=0x7fbcdc000b80).stop 2020-11-16T22:30:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 msgr2=0x7fbcec1702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbce0001690 tx=0x7fbce00093e0).stop 2020-11-16T22:30:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 shutdown_connections 2020-11-16T22:30:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcc8055090 0x7fbcc8057550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec065040 0x7fbcec0d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec060290 0x7fbcec145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 --2- 172.21.15.181:0/309432470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec060670 0x7fbcec1702e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 >> 172.21.15.181:0/309432470 conn(0x7fbcec0d3430 msgr2=0x7fbcec0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 shutdown_connections 2020-11-16T22:30:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.537+0000 7fbcf15c2700 1 -- 172.21.15.181:0/309432470 wait complete. 2020-11-16T22:30:39.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/d1/d2/d3]'} 2020-11-16T22:30:39.531 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:30:39.531 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:39.569 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:39.586 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:39.601 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:30:39.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:30:39.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:30:39.602 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:30:39.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:30:39.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:39.676 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:30:39.694 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:30:39.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 Processor -- start 2020-11-16T22:30:39.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 -- start start 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10f4149d20 0x7f10f4065f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f414a580 con 0x7f10f4063d50 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f40cf620 con 0x7f10f414a100 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10fa8ab700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f4064d60 con 0x7f10f4149d20 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f90a8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f10f414a100 0x7f10f40cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46392/0 (socket says 172.21.15.181:46392) 2020-11-16T22:30:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f90a8700 1 -- 172.21.15.181:0/33562186 learned_addr learned my addr 172.21.15.181:0/33562186 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f98a9700 1 --2- 172.21.15.181:0/33562186 >> v2:172.21.15.175:3300/0 conn(0x7f10f4149d20 0x7f10f4065f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56552650 0 0) 0x7f10f40cf620 con 0x7f10f414a100 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e0003240 con 0x7f10f414a100 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048590019 0 0) 0x7f10f414a580 con 0x7f10f4063d50 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f40cf620 con 0x7f10f4063d50 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1769701196 0 0) 0x7f10e0003240 con 0x7f10f414a100 2020-11-16T22:30:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f98a9700 1 -- 172.21.15.181:0/33562186 >> v1:172.21.15.181:6789/0 conn(0x7f10f414a100 legacy=0x7f10f40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f98a9700 1 -- 172.21.15.181:0/33562186 >> v1:172.21.15.175:6789/0 conn(0x7f10f4063d50 legacy=0x7f10f40641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f98a9700 1 -- 172.21.15.181:0/33562186 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4065000 con 0x7f10f4149d20 2020-11-16T22:30:39.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f98a9700 1 --2- 172.21.15.181:0/33562186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4149d20 0x7f10f4065f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f10e400b920 tx=0x7f10e4006df0).ready entity=mon.0 client_cookie=695df1f207f87a47 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0001520 con 0x7f10f4149d20 2020-11-16T22:30:39.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4011400 con 0x7f10f4149d20 2020-11-16T22:30:39.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.849+0000 7f10f88a7700 1 -- 172.21.15.181:0/33562186 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4011780 con 0x7f10f4149d20 2020-11-16T22:30:39.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- 172.21.15.181:0/33562186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4149d20 msgr2=0x7f10f4065f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/33562186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4149d20 0x7f10f4065f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f10e400b920 tx=0x7f10e4006df0).stop 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- 172.21.15.181:0/33562186 shutdown_connections 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/33562186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4149d20 0x7f10f4065f20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- 172.21.15.181:0/33562186 >> 172.21.15.181:0/33562186 conn(0x7f10f40d4570 msgr2=0x7f10f40d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- 172.21.15.181:0/33562186 shutdown_connections 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- 172.21.15.181:0/33562186 wait complete. 2020-11-16T22:30:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 Processor -- start 2020-11-16T22:30:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- start start 2020-11-16T22:30:39.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 0x7f10f415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10f414a100 0x7f10f414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4160420 con 0x7f10f4063d50 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f415ed70 con 0x7f10f4149d20 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa8ab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f415f010 con 0x7f10f414a100 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60328/0 (socket says 172.21.15.181:60328) 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 -- 172.21.15.181:0/1214727861 learned_addr learned my addr 172.21.15.181:0/1214727861 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:39.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10fa0aa700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10f414a100 0x7f10f414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f98a9700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 0x7f10f415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 -- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10f414a100 msgr2=0x7f10f414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10f414a100 0x7f10f414a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 -- 172.21.15.181:0/1214727861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 msgr2=0x7f10f415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 0x7f10f415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 -- 172.21.15.181:0/1214727861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f415f2b0 con 0x7f10f4149d20 2020-11-16T22:30:39.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f98a9700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 0x7f10f415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10f90a8700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f10f000ca90 tx=0x7f10f0001aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.853+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0007170 con 0x7f10f4149d20 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00072d0 con 0x7f10f4149d20 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000a9c0 con 0x7f10f4149d20 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f415f550 con 0x7f10f4149d20 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4174f10 con 0x7f10f4149d20 2020-11-16T22:30:39.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4044f60 con 0x7f10f4149d20 2020-11-16T22:30:39.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f10f0010070 con 0x7f10f4149d20 2020-11-16T22:30:39.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10e9ffb700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 0x7f10cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:39.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10f98a9700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 0x7f10cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:39.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10f006d810 con 0x7f10f4149d20 2020-11-16T22:30:39.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.857+0000 7f10f98a9700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 0x7f10cc057520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f10e400b150 tx=0x7f10e4013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:39.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:39.861+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f10f0043a70 con 0x7f10f4149d20 2020-11-16T22:30:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.017+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f10f405d4e0 con 0x7f10f4149d20 2020-11-16T22:30:40.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.029+0000 7f10e9ffb700 1 -- 172.21.15.181:0/1214727861 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10f0047090 con 0x7f10f4149d20 2020-11-16T22:30:40.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 msgr2=0x7f10cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 0x7f10cc057520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f10e400b150 tx=0x7f10e4013000).stop 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 msgr2=0x7f10f4174a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f10f000ca90 tx=0x7f10f0001aa0).stop 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 shutdown_connections 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10cc055060 0x7f10cc057520 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10f414a100 0x7f10f414a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f4063d50 0x7f10f415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 --2- 172.21.15.181:0/1214727861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f4149d20 0x7f10f4174a20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 >> 172.21.15.181:0/1214727861 conn(0x7f10f40d4570 msgr2=0x7f10f40cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:40.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 shutdown_connections 2020-11-16T22:30:40.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.033+0000 7f10fa8ab700 1 -- 172.21.15.181:0/1214727861 wait complete. 2020-11-16T22:30:40.027 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2020-11-16T22:30:40.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.165+0000 7f74ec9d1700 1 Processor -- start 2020-11-16T22:30:40.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.165+0000 7f74ec9d1700 1 -- start start 2020-11-16T22:30:40.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e80cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e8065190 con 0x7f74e8060800 2020-11-16T22:30:40.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e8065430 con 0x7f74e8163970 2020-11-16T22:30:40.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80cfc60 con 0x7f74e80cf390 2020-11-16T22:30:40.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74de59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f74e8163970 0x7f74e80600f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46406/0 (socket says 172.21.15.181:46406) 2020-11-16T22:30:40.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74de59c700 1 -- 172.21.15.181:0/2776965312 learned_addr learned my addr 172.21.15.181:0/2776965312 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:40.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e6d9d700 1 --2- 172.21.15.181:0/2776965312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e80cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127194900 0 0) 0x7f74e8065430 con 0x7f74e8163970 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c8003240 con 0x7f74e8163970 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730660547 0 0) 0x7f74e8065190 con 0x7f74e8060800 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e6d9d700 1 -- 172.21.15.181:0/2776965312 >> v1:172.21.15.181:6789/0 conn(0x7f74e8163970 legacy=0x7f74e80600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e6d9d700 1 -- 172.21.15.181:0/2776965312 >> v1:172.21.15.175:6789/0 conn(0x7f74e8060800 legacy=0x7f74e80660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e6d9d700 1 -- 172.21.15.181:0/2776965312 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80656d0 con 0x7f74e80cf390 2020-11-16T22:30:40.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e6d9d700 1 --2- 172.21.15.181:0/2776965312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e80cf770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f74d800ba00 tx=0x7f74d8006df0).ready entity=mon.2 client_cookie=dca336e0cc4672d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8009770 con 0x7f74e80cf390 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d80098d0 con 0x7f74e80cf390 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74e659c700 1 -- 172.21.15.181:0/2776965312 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8009c50 con 0x7f74e80cf390 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- 172.21.15.181:0/2776965312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 msgr2=0x7f74e80cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/2776965312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e80cf770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f74d800ba00 tx=0x7f74d8006df0).stop 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- 172.21.15.181:0/2776965312 shutdown_connections 2020-11-16T22:30:40.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/2776965312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e80cf770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- 172.21.15.181:0/2776965312 >> 172.21.15.181:0/2776965312 conn(0x7f74e80cfe90 msgr2=0x7f74e80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:40.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- 172.21.15.181:0/2776965312 shutdown_connections 2020-11-16T22:30:40.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.169+0000 7f74ec9d1700 1 -- 172.21.15.181:0/2776965312 wait complete. 2020-11-16T22:30:40.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 Processor -- start 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- start start 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74e8060800 0x7f74e815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 0x7f74e8066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8160590 con 0x7f74e8060800 2020-11-16T22:30:40.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815f3d0 con 0x7f74e8163970 2020-11-16T22:30:40.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815f670 con 0x7f74e80cf390 2020-11-16T22:30:40.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74e759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 0x7f74e8066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39952/0 (socket says 172.21.15.181:39952) 2020-11-16T22:30:40.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74e759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 0x7f74e8066660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60344/0 (socket says 172.21.15.181:60344) 2020-11-16T22:30:40.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 -- 172.21.15.181:0/4266045907 learned_addr learned my addr 172.21.15.181:0/4266045907 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74e6d9d700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74e8060800 0x7f74e815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 -- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 msgr2=0x7f74e8066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 0x7f74e8066660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 -- 172.21.15.181:0/4266045907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74e8060800 msgr2=0x7f74e815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74e8060800 0x7f74e815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 -- 172.21.15.181:0/4266045907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e815f910 con 0x7f74e80cf390 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74de59c700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f74d000c6e0 tx=0x7f74d0007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d00072e0 con 0x7f74e80cf390 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d0007440 con 0x7f74e80cf390 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e815fbb0 con 0x7f74e80cf390 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0005820 con 0x7f74e80cf390 2020-11-16T22:30:40.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.173+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e815f000 con 0x7f74e80cf390 2020-11-16T22:30:40.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8044f60 con 0x7f74e80cf390 2020-11-16T22:30:40.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f74d0010070 con 0x7f74e80cf390 2020-11-16T22:30:40.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74df7fe700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 0x7f74c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f74d006d2f0 con 0x7f74e80cf390 2020-11-16T22:30:40.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74e6d9d700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 0x7f74c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.177+0000 7f74e6d9d700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 0x7f74c4057520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f74d800bfd0 tx=0x7f74d8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.185+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f74d0043550 con 0x7f74e80cf390 2020-11-16T22:30:40.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.393+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7f74e8163410 con 0x7f74e80cf390 2020-11-16T22:30:40.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.853+0000 7f74df7fe700 1 -- 172.21.15.181:0/4266045907 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v195) v1 ==== 143+0+0 (secure 0 0 0) 0x7f74d0046b70 con 0x7f74e80cf390 2020-11-16T22:30:40.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 msgr2=0x7f74c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 0x7f74c4057520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f74d800bfd0 tx=0x7f74d8015000).stop 2020-11-16T22:30:40.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 msgr2=0x7f74e8066170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f74d000c6e0 tx=0x7f74d0007a00).stop 2020-11-16T22:30:40.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 shutdown_connections 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c4055060 0x7f74c4057520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e8163970 0x7f74e8066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74e8060800 0x7f74e815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 --2- 172.21.15.181:0/4266045907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e80cf390 0x7f74e8066170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 >> 172.21.15.181:0/4266045907 conn(0x7f74e80cfe90 msgr2=0x7f74e80600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 shutdown_connections 2020-11-16T22:30:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.857+0000 7f74ec9d1700 1 -- 172.21.15.181:0/4266045907 wait complete. 2020-11-16T22:30:40.852 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:30:40.852 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:30:40.853 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2020-11-16T22:30:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 Processor -- start 2020-11-16T22:30:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 -- start start 2020-11-16T22:30:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe494062740 0x7fe4940644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940c8db0 con 0x7fe494161480 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe494165030 con 0x7fe494062740 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe49359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940c8330 con 0x7fe494161860 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:40.997+0000 7fe491d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39958/0 (socket says 172.21.15.181:39958) 2020-11-16T22:30:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 -- 172.21.15.181:0/3801214295 learned_addr learned my addr 172.21.15.181:0/3801214295 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe492d9d700 1 --2- 172.21.15.181:0/3801214295 >> v2:172.21.15.175:3300/0 conn(0x7fe494062740 0x7fe4940644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 -- 172.21.15.181:0/3801214295 >> v2:172.21.15.175:3300/0 conn(0x7fe494062740 msgr2=0x7fe4940644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 --2- 172.21.15.181:0/3801214295 >> v2:172.21.15.175:3300/0 conn(0x7fe494062740 0x7fe4940644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49159a700 1 -- 172.21.15.181:0/3801214295 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985265430 0 0) 0x7fe4940c8db0 con 0x7fe494161480 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 -- 172.21.15.181:0/3801214295 >> v1:172.21.15.181:6789/0 conn(0x7fe494161480 legacy=0x7fe494164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 -- 172.21.15.181:0/3801214295 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940c85d0 con 0x7fe494161860 2020-11-16T22:30:40.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe491d9b700 1 --2- 172.21.15.181:0/3801214295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fe47c006fd0 tx=0x7fe47c006eb0).ready entity=mon.2 client_cookie=a79ae540b618ff41 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49159a700 1 -- 172.21.15.181:0/3801214295 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c009810 con 0x7fe494161860 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49159a700 1 -- 172.21.15.181:0/3801214295 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c009970 con 0x7fe494161860 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49159a700 1 -- 172.21.15.181:0/3801214295 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c009cf0 con 0x7fe494161860 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- 172.21.15.181:0/3801214295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 msgr2=0x7fe494062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 --2- 172.21.15.181:0/3801214295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fe47c006fd0 tx=0x7fe47c006eb0).stop 2020-11-16T22:30:40.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- 172.21.15.181:0/3801214295 shutdown_connections 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 --2- 172.21.15.181:0/3801214295 >> v2:172.21.15.175:3300/0 conn(0x7fe494062740 0x7fe4940644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 --2- 172.21.15.181:0/3801214295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494161860 0x7fe494062250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- 172.21.15.181:0/3801214295 >> 172.21.15.181:0/3801214295 conn(0x7fe4940d2dc0 msgr2=0x7fe4940ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- 172.21.15.181:0/3801214295 shutdown_connections 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- 172.21.15.181:0/3801214295 wait complete. 2020-11-16T22:30:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 Processor -- start 2020-11-16T22:30:40.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.001+0000 7fe49359e700 1 -- start start 2020-11-16T22:30:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 0x7fe4940b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe494161480 0x7fe4940cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 0x7fe4940ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494177560 con 0x7fe494161480 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494177800 con 0x7fe494161860 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe494177aa0 con 0x7fe494062740 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 0x7fe4940b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe492d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 0x7fe4940ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe492d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 0x7fe4940ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60358/0 (socket says 172.21.15.181:60358) 2020-11-16T22:30:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe492d9d700 1 -- 172.21.15.181:0/3949276371 learned_addr learned my addr 172.21.15.181:0/3949276371 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe491d9b700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe494161480 0x7fe4940cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 -- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 msgr2=0x7fe4940ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 0x7fe4940ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 -- 172.21.15.181:0/3949276371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe494161480 msgr2=0x7fe4940cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe494161480 0x7fe4940cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 -- 172.21.15.181:0/3949276371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494177d40 con 0x7fe494062740 2020-11-16T22:30:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49259c700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 0x7fe4940b79e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fe4940c91e0 tx=0x7fe484007da0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400ce50 con 0x7fe494062740 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48400a840 con 0x7fe494062740 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe494177fe0 con 0x7fe494062740 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400ac00 con 0x7fe494062740 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4940b5640 con 0x7fe494062740 2020-11-16T22:30:40.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe484019030 con 0x7fe494062740 2020-11-16T22:30:40.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.005+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4940b58f0 con 0x7fe494062740 2020-11-16T22:30:40.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.009+0000 7fe48a7fc700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 0x7fe464057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:40.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.009+0000 7fe491d9b700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 0x7fe464057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:40.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.009+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe48406d5a0 con 0x7fe494062740 2020-11-16T22:30:40.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.009+0000 7fe491d9b700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 0x7fe464057520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fe47c00b1b0 tx=0x7fe47c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:40.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.013+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe4840438b0 con 0x7fe494062740 2020-11-16T22:30:41.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.217+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7fe4940cf7c0 con 0x7fe494062740 2020-11-16T22:30:41.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.853+0000 7fe48a7fc700 1 -- 172.21.15.181:0/3949276371 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v196) v1 ==== 145+0+0 (secure 0 0 0) 0x7fe4940cf7c0 con 0x7fe494062740 2020-11-16T22:30:41.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 msgr2=0x7fe464057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 0x7fe464057520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fe47c00b1b0 tx=0x7fe47c015000).stop 2020-11-16T22:30:41.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 msgr2=0x7fe4940b79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 0x7fe4940b79e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fe4940c91e0 tx=0x7fe484007da0).stop 2020-11-16T22:30:41.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 shutdown_connections 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe464055060 0x7fe464057520 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe494161860 0x7fe4940ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe494062740 0x7fe4940b79e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 --2- 172.21.15.181:0/3949276371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe494161480 0x7fe4940cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 >> 172.21.15.181:0/3949276371 conn(0x7fe4940d2dc0 msgr2=0x7fe494062f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 shutdown_connections 2020-11-16T22:30:41.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.857+0000 7fe49359e700 1 -- 172.21.15.181:0/3949276371 wait complete. 2020-11-16T22:30:41.853 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:30:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.997+0000 7f74bf59e700 1 Processor -- start 2020-11-16T22:30:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:41.997+0000 7f74bf59e700 1 -- start start 2020-11-16T22:30:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74c0161480 0x7f74c0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161860 0x7f74c0062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00c8db0 con 0x7f74c0164e10 2020-11-16T22:30:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c00c8330 con 0x7f74c0161480 2020-11-16T22:30:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00c85d0 con 0x7f74c0161860 2020-11-16T22:30:41.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161860 0x7f74c0062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74c0161480 0x7f74c0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bdd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f74c0164e10 0x7f74c0062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46436/0 (socket says 172.21.15.181:46436) 2020-11-16T22:30:41.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bdd9b700 1 -- 172.21.15.181:0/1209147238 learned_addr learned my addr 172.21.15.181:0/1209147238 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:41.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bd59a700 1 -- 172.21.15.181:0/1209147238 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619990192 0 0) 0x7f74c00c8db0 con 0x7f74c0164e10 2020-11-16T22:30:41.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bd59a700 1 -- 172.21.15.181:0/1209147238 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a4003240 con 0x7f74c0164e10 2020-11-16T22:30:41.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 -- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161860 msgr2=0x7f74c0062960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 --2- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161860 0x7f74c0062960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 -- 172.21.15.181:0/1209147238 >> v1:172.21.15.181:6789/0 conn(0x7f74c0164e10 legacy=0x7f74c0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 -- 172.21.15.181:0/1209147238 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00c8870 con 0x7f74c0161480 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74be59c700 1 --2- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161480 0x7f74c0164920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f74b000b9a0 tx=0x7f74b0006df0).ready entity=mon.1 client_cookie=1fe57e4727f5798a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bd59a700 1 -- 172.21.15.181:0/1209147238 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0009680 con 0x7f74c0161480 2020-11-16T22:30:41.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bd59a700 1 -- 172.21.15.181:0/1209147238 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00097e0 con 0x7f74c0161480 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bd59a700 1 -- 172.21.15.181:0/1209147238 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0009b60 con 0x7f74c0161480 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161480 msgr2=0x7f74c0164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161480 0x7f74c0164920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f74b000b9a0 tx=0x7f74b0006df0).stop 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- 172.21.15.181:0/1209147238 shutdown_connections 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161480 0x7f74c0164920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1209147238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161860 0x7f74c0062960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- 172.21.15.181:0/1209147238 >> 172.21.15.181:0/1209147238 conn(0x7f74c00d2dc0 msgr2=0x7f74c00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- 172.21.15.181:0/1209147238 shutdown_connections 2020-11-16T22:30:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.001+0000 7f74bf59e700 1 -- 172.21.15.181:0/1209147238 wait complete. 2020-11-16T22:30:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 Processor -- start 2020-11-16T22:30:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- start start 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 0x7f74c00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 0x7f74c0174b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c01736b0 0x7f74c0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0174680 con 0x7f74c01736b0 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00c9e30 con 0x7f74c0161860 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0173ea0 con 0x7f74c0161480 2020-11-16T22:30:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74be59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 0x7f74c00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 0x7f74c0174b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74be59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 0x7f74c00cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39978/0 (socket says 172.21.15.181:39978) 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74be59c700 1 -- 172.21.15.181:0/1276935018 learned_addr learned my addr 172.21.15.181:0/1276935018 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bed9d700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c01736b0 0x7f74c0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 -- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 msgr2=0x7f74c00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 0x7f74c00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 -- 172.21.15.181:0/1276935018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c01736b0 msgr2=0x7f74c0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c01736b0 0x7f74c0160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 -- 172.21.15.181:0/1276935018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0174140 con 0x7f74c0161860 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bdd9b700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 0x7f74c0174b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f74a800c6e0 tx=0x7f74a80079c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80072e0 con 0x7f74c0161860 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8007440 con 0x7f74c0161860 2020-11-16T22:30:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8005820 con 0x7f74c0161860 2020-11-16T22:30:41.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c01743e0 con 0x7f74c0161860 2020-11-16T22:30:41.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.005+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c0146440 con 0x7f74c0161860 2020-11-16T22:30:41.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f74a8010070 con 0x7f74c0161860 2020-11-16T22:30:41.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c0146600 con 0x7f74c0161860 2020-11-16T22:30:41.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74b67fc700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 0x7f7490057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:41.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f74a806d360 con 0x7f74c0161860 2020-11-16T22:30:41.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74be59c700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 0x7f7490057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:41.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.009+0000 7f74be59c700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 0x7f7490057520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f74b0009380 tx=0x7f74b0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:41.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.017+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f74a80435c0 con 0x7f74c0161860 2020-11-16T22:30:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.169+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f74c0044f60 con 0x7f74c0161860 2020-11-16T22:30:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74b67fc700 1 -- 172.21.15.181:0/1276935018 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74a8018020 con 0x7f74c0161860 2020-11-16T22:30:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 msgr2=0x7f7490057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 0x7f7490057520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f74b0009380 tx=0x7f74b0015000).stop 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 msgr2=0x7f74c0174b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 0x7f74c0174b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f74a800c6e0 tx=0x7f74a80079c0).stop 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 shutdown_connections 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7490055060 0x7f7490057520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c01736b0 0x7f74c0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74c0161480 0x7f74c00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 --2- 172.21.15.181:0/1276935018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c0161860 0x7f74c0174b50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 >> 172.21.15.181:0/1276935018 conn(0x7f74c00d2dc0 msgr2=0x7f74c0147470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:42.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 shutdown_connections 2020-11-16T22:30:42.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.181+0000 7f74bf59e700 1 -- 172.21.15.181:0/1276935018 wait complete. 2020-11-16T22:30:42.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:30:42.177 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:42.215 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:42.231 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:30:42.247 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:30:42.247 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:30:42.248 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:42.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 Processor -- start 2020-11-16T22:30:42.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 -- start start 2020-11-16T22:30:42.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffb74064ae0 0x7ffb74160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb74165250 con 0x7ffb741642d0 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb74161d60 con 0x7ffb74064ae0 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb7b138700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74067220 con 0x7ffb74066950 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb78ed4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb78ed4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39988/0 (socket says 172.21.15.181:39988) 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb78ed4700 1 -- 172.21.15.181:0/1049517101 learned_addr learned my addr 172.21.15.181:0/1049517101 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:42.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.433+0000 7ffb796d5700 1 --2- 172.21.15.181:0/1049517101 >> v2:172.21.15.175:3300/0 conn(0x7ffb74064ae0 0x7ffb74160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb78ed4700 1 -- 172.21.15.181:0/1049517101 >> v2:172.21.15.175:3300/0 conn(0x7ffb74064ae0 msgr2=0x7ffb74160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb78ed4700 1 --2- 172.21.15.181:0/1049517101 >> v2:172.21.15.175:3300/0 conn(0x7ffb74064ae0 0x7ffb74160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb78ed4700 1 -- 172.21.15.181:0/1049517101 >> v1:172.21.15.175:6789/0 conn(0x7ffb741642d0 legacy=0x7ffb740643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb78ed4700 1 -- 172.21.15.181:0/1049517101 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74164a70 con 0x7ffb74066950 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb78ed4700 1 --2- 172.21.15.181:0/1049517101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7ffb64006fd0 tx=0x7ffb64006eb0).ready entity=mon.2 client_cookie=76828dc7104ee7d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb737fe700 1 -- 172.21.15.181:0/1049517101 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640098a0 con 0x7ffb74066950 2020-11-16T22:30:42.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb737fe700 1 -- 172.21.15.181:0/1049517101 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64009a00 con 0x7ffb74066950 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb737fe700 1 -- 172.21.15.181:0/1049517101 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64009d80 con 0x7ffb74066950 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- 172.21.15.181:0/1049517101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 msgr2=0x7ffb74066d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1049517101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7ffb64006fd0 tx=0x7ffb64006eb0).stop 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- 172.21.15.181:0/1049517101 shutdown_connections 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1049517101 >> v2:172.21.15.175:3300/0 conn(0x7ffb74064ae0 0x7ffb74160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1049517101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb74066d30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- 172.21.15.181:0/1049517101 >> 172.21.15.181:0/1049517101 conn(0x7ffb740d4790 msgr2=0x7ffb740d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:42.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- 172.21.15.181:0/1049517101 shutdown_connections 2020-11-16T22:30:42.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- 172.21.15.181:0/1049517101 wait complete. 2020-11-16T22:30:42.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 Processor -- start 2020-11-16T22:30:42.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- start start 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb74064ae0 0x7ffb740d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb741642d0 0x7ffb740b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74173950 con 0x7ffb74064ae0 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74173bf0 con 0x7ffb741642d0 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb7b138700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74173e90 con 0x7ffb74066950 2020-11-16T22:30:42.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb73fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.437+0000 7ffb73fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:39994/0 (socket says 172.21.15.181:39994) 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 -- 172.21.15.181:0/1587187103 learned_addr learned my addr 172.21.15.181:0/1587187103 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb796d5700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb741642d0 0x7ffb740b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb78ed4700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb74064ae0 0x7ffb740d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 -- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb741642d0 msgr2=0x7ffb740b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb741642d0 0x7ffb740b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 -- 172.21.15.181:0/1587187103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb74064ae0 msgr2=0x7ffb740d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb74064ae0 0x7ffb740d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 -- 172.21.15.181:0/1587187103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74174130 con 0x7ffb74066950 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb73fff700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7ffb5c00c710 tx=0x7ffb5c007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00e490 con 0x7ffb74066950 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c00e5f0 con 0x7ffb74066950 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb741743d0 con 0x7ffb74066950 2020-11-16T22:30:42.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00e990 con 0x7ffb74066950 2020-11-16T22:30:42.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb740d2b80 con 0x7ffb74066950 2020-11-16T22:30:42.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb74069eb0 con 0x7ffb74066950 2020-11-16T22:30:42.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ffb5c019030 con 0x7ffb74066950 2020-11-16T22:30:42.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 0x7ffb50057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffb5c06d120 con 0x7ffb74066950 2020-11-16T22:30:42.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb78ed4700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 0x7ffb50057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.441+0000 7ffb78ed4700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 0x7ffb50057520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ffb74067180 tx=0x7ffb64015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.445+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ffb5c043380 con 0x7ffb74066950 2020-11-16T22:30:42.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.645+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb74063410 con 0x7ffb74066950 2020-11-16T22:30:42.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.645+0000 7ffb70ff9700 1 -- 172.21.15.181:0/1587187103 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7ffb5c017020 con 0x7ffb74066950 2020-11-16T22:30:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:42.629 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:42.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 msgr2=0x7ffb50057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 0x7ffb50057520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7ffb74067180 tx=0x7ffb64015000).stop 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 msgr2=0x7ffb740b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7ffb5c00c710 tx=0x7ffb5c007c10).stop 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 shutdown_connections 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffb50055060 0x7ffb50057520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb741642d0 0x7ffb740b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb74064ae0 0x7ffb740d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 --2- 172.21.15.181:0/1587187103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb74066950 0x7ffb740b78c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 >> 172.21.15.181:0/1587187103 conn(0x7ffb740d4790 msgr2=0x7ffb740b8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:42.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 shutdown_connections 2020-11-16T22:30:42.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.649+0000 7ffb7b138700 1 -- 172.21.15.181:0/1587187103 wait complete. 2020-11-16T22:30:42.633 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:42.648 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 Processor -- start 2020-11-16T22:30:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 -- start start 2020-11-16T22:30:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50e0164e10 0x7f50e0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e0161860 0x7f50e0062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00c8db0 con 0x7f50e0164e10 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00c8330 con 0x7f50e0161860 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e54c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c85d0 con 0x7f50e0161480 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e0161860 0x7f50e0062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e0161860 0x7f50e0062740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60390/0 (socket says 172.21.15.181:60390) 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 -- 172.21.15.181:0/1489192193 learned_addr learned my addr 172.21.15.181:0/1489192193 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:42.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50dffff700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50df7fe700 1 --2- 172.21.15.181:0/1489192193 >> v2:172.21.15.175:3300/0 conn(0x7f50e0164e10 0x7f50e0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 -- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 msgr2=0x7f50e0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 -- 172.21.15.181:0/1489192193 >> v2:172.21.15.175:3300/0 conn(0x7f50e0164e10 msgr2=0x7f50e0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/1489192193 >> v2:172.21.15.175:3300/0 conn(0x7f50e0164e10 0x7f50e0062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50e4cbf700 1 -- 172.21.15.181:0/1489192193 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00c8870 con 0x7f50e0161860 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.789+0000 7f50dffff700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0161860 0x7f50e0062740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f50e00b8580 tx=0x7f50c8006eb0).ready entity=mon.1 client_cookie=32f7e9df18576f0a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50deffd700 1 -- 172.21.15.181:0/1489192193 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8009720 con 0x7f50e0161860 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50deffd700 1 -- 172.21.15.181:0/1489192193 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8009880 con 0x7f50e0161860 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50deffd700 1 -- 172.21.15.181:0/1489192193 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8009c00 con 0x7f50e0161860 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0161860 msgr2=0x7f50e0062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0161860 0x7f50e0062740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f50e00b8580 tx=0x7f50c8006eb0).stop 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- 172.21.15.181:0/1489192193 shutdown_connections 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0161860 0x7f50e0062740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- 172.21.15.181:0/1489192193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- 172.21.15.181:0/1489192193 >> v2:172.21.15.175:3300/0 conn(0x7f50e0164e10 0x7f50e0062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- 172.21.15.181:0/1489192193 >> 172.21.15.181:0/1489192193 conn(0x7f50e00d2dc0 msgr2=0x7f50e00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- 172.21.15.181:0/1489192193 shutdown_connections 2020-11-16T22:30:42.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- 172.21.15.181:0/1489192193 wait complete. 2020-11-16T22:30:42.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 Processor -- start 2020-11-16T22:30:42.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- start start 2020-11-16T22:30:42.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 0x7f50e0060780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 0x7f50e0173fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e01744a0 con 0x7f50e0161860 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0174f80 con 0x7f50e0173b30 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e54c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0174690 con 0x7f50e0161480 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50dffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50e4cbf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 0x7f50e0173fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50dffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40008/0 (socket says 172.21.15.181:40008) 2020-11-16T22:30:42.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50dffff700 1 -- 172.21.15.181:0/103423044 learned_addr learned my addr 172.21.15.181:0/103423044 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.793+0000 7f50df7fe700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 0x7f50e0060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 -- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 msgr2=0x7f50e0060290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 -- 172.21.15.181:0/103423044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 msgr2=0x7f50e0060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 0x7f50e0060780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 -- 172.21.15.181:0/103423044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0174930 con 0x7f50e0173b30 2020-11-16T22:30:42.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50df7fe700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 0x7f50e0060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50dffff700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e4cbf700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 0x7f50e0173fb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f50e0045680 tx=0x7f50c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c801e460 con 0x7f50e0173b30 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c801e5c0 con 0x7f50e0173b30 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e01751b0 con 0x7f50e0173b30 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c801e940 con 0x7f50e0173b30 2020-11-16T22:30:42.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e01756f0 con 0x7f50e0173b30 2020-11-16T22:30:42.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e0044f60 con 0x7f50e0173b30 2020-11-16T22:30:42.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f50c8025030 con 0x7f50e0173b30 2020-11-16T22:30:42.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 0x7f50b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:42.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f50c806f160 con 0x7f50e0173b30 2020-11-16T22:30:42.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50dffff700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 0x7f50b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:42.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.797+0000 7f50dffff700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 0x7f50b0057520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f50d800f440 tx=0x7f50d80091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:42.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.805+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f50c8045450 con 0x7f50e0173b30 2020-11-16T22:30:42.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.957+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50e00b8360 con 0x7f50e0173b30 2020-11-16T22:30:42.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50d7fff700 1 -- 172.21.15.181:0/103423044 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f50c8031200 con 0x7f50e0173b30 2020-11-16T22:30:42.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:42.943 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:31.359585+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:42.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 msgr2=0x7f50b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 0x7f50b0057520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f50d800f440 tx=0x7f50d80091d0).stop 2020-11-16T22:30:42.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 msgr2=0x7f50e0173fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 0x7f50e0173fb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f50e0045680 tx=0x7f50c8015000).stop 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 shutdown_connections 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f50b0055060 0x7f50b0057520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0173b30 0x7f50e0173fb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50e0161480 0x7f50e0060290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 --2- 172.21.15.181:0/103423044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e0161860 0x7f50e0060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:42.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 >> 172.21.15.181:0/103423044 conn(0x7f50e00d2dc0 msgr2=0x7f50e00cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:42.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 shutdown_connections 2020-11-16T22:30:42.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:42.961+0000 7f50e54c0700 1 -- 172.21.15.181:0/103423044 wait complete. 2020-11-16T22:30:42.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:29:28.703069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:29:29.511094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:30:42.958 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:30:43.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 Processor -- start 2020-11-16T22:30:43.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 -- start start 2020-11-16T22:30:43.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff714161860 0x7ff714062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff714062740 0x7ff7140644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7140c8db0 con 0x7ff714161480 2020-11-16T22:30:43.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff714165030 con 0x7ff714062740 2020-11-16T22:30:43.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff713fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7140c8330 con 0x7ff714161860 2020-11-16T22:30:43.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff712ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff714161480 0x7ff714164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46476/0 (socket says 172.21.15.181:46476) 2020-11-16T22:30:43.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff712ffd700 1 -- 172.21.15.181:0/1046441425 learned_addr learned my addr 172.21.15.181:0/1046441425 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:43.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 --2- 172.21.15.181:0/1046441425 >> v2:172.21.15.181:3300/0 conn(0x7ff714161860 0x7ff714062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7137fe700 1 --2- 172.21.15.181:0/1046441425 >> v2:172.21.15.175:3300/0 conn(0x7ff714062740 0x7ff7140644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff711ffb700 1 -- 172.21.15.181:0/1046441425 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047264124 0 0) 0x7ff7140c8db0 con 0x7ff714161480 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff711ffb700 1 -- 172.21.15.181:0/1046441425 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f4003240 con 0x7ff714161480 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 -- 172.21.15.181:0/1046441425 >> v2:172.21.15.175:3300/0 conn(0x7ff714062740 msgr2=0x7ff7140644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 --2- 172.21.15.181:0/1046441425 >> v2:172.21.15.175:3300/0 conn(0x7ff714062740 0x7ff7140644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 -- 172.21.15.181:0/1046441425 >> v1:172.21.15.181:6789/0 conn(0x7ff714161480 legacy=0x7ff714164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 -- 172.21.15.181:0/1046441425 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140c85d0 con 0x7ff714161860 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff7127fc700 1 --2- 172.21.15.181:0/1046441425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161860 0x7ff714062250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff7140ced60 tx=0x7ff6fc006df0).ready entity=mon.1 client_cookie=a48469c4d068d63e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff711ffb700 1 -- 172.21.15.181:0/1046441425 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc009640 con 0x7ff714161860 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff711ffb700 1 -- 172.21.15.181:0/1046441425 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0097a0 con 0x7ff714161860 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.105+0000 7ff711ffb700 1 -- 172.21.15.181:0/1046441425 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc009b20 con 0x7ff714161860 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- 172.21.15.181:0/1046441425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161860 msgr2=0x7ff714062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- 172.21.15.181:0/1046441425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161860 0x7ff714062250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff7140ced60 tx=0x7ff6fc006df0).stop 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- 172.21.15.181:0/1046441425 shutdown_connections 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- 172.21.15.181:0/1046441425 >> v2:172.21.15.175:3300/0 conn(0x7ff714062740 0x7ff7140644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- 172.21.15.181:0/1046441425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161860 0x7ff714062250 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- 172.21.15.181:0/1046441425 >> 172.21.15.181:0/1046441425 conn(0x7ff7140d2dc0 msgr2=0x7ff7140ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- 172.21.15.181:0/1046441425 shutdown_connections 2020-11-16T22:30:43.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- 172.21.15.181:0/1046441425 wait complete. 2020-11-16T22:30:43.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 Processor -- start 2020-11-16T22:30:43.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- start start 2020-11-16T22:30:43.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff714062740 0x7ff714164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff714161860 0x7ff7140ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714169700 con 0x7ff714062740 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff7127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7141698e0 con 0x7ff714161480 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff713fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff714169b80 con 0x7ff714161860 2020-11-16T22:30:43.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff7127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60412/0 (socket says 172.21.15.181:60412) 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.109+0000 7ff7127fc700 1 -- 172.21.15.181:0/633307541 learned_addr learned my addr 172.21.15.181:0/633307541 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff712ffd700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff714062740 0x7ff714164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7137fe700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff714161860 0x7ff7140ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 -- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff714161860 msgr2=0x7ff7140ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff714161860 0x7ff7140ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 -- 172.21.15.181:0/633307541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff714062740 msgr2=0x7ff714164530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff714062740 0x7ff714164530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 -- 172.21.15.181:0/633307541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff714169e20 con 0x7ff714161480 2020-11-16T22:30:43.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff7127fc700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7ff6fc010000 tx=0x7ff6fc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc011b40 con 0x7ff714161480 2020-11-16T22:30:43.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc011ca0 con 0x7ff714161480 2020-11-16T22:30:43.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71416a0c0 con 0x7ff714161480 2020-11-16T22:30:43.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01e670 con 0x7ff714161480 2020-11-16T22:30:43.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7140ce570 con 0x7ff714161480 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff6fc025030 con 0x7ff714161480 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff714068fb0 con 0x7ff714161480 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 0x7ff6f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff6fc06f320 con 0x7ff714161480 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff712ffd700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 0x7ff6f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.113+0000 7ff712ffd700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 0x7ff6f0057520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7ff714148fb0 tx=0x7ff704009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.117+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff714068fb0 con 0x7ff714161480 2020-11-16T22:30:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff714068fb0 con 0x7ff714161480 2020-11-16T22:30:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff70affd700 1 -- 172.21.15.181:0/633307541 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v49) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff714068fb0 con 0x7ff714161480 2020-11-16T22:30:43.255 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 msgr2=0x7ff6f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 0x7ff6f0057520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7ff714148fb0 tx=0x7ff704009040).stop 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 msgr2=0x7ff714164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.273+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7ff6fc010000 tx=0x7ff6fc015000).stop 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 shutdown_connections 2020-11-16T22:30:43.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff6f0055060 0x7ff6f0057520 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff714161860 0x7ff7140ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff714062740 0x7ff714164530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 --2- 172.21.15.181:0/633307541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff714161480 0x7ff714164a20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 >> 172.21.15.181:0/633307541 conn(0x7ff7140d2dc0 msgr2=0x7ff7140cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 shutdown_connections 2020-11-16T22:30:43.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.277+0000 7ff713fff700 1 -- 172.21.15.181:0/633307541 wait complete. 2020-11-16T22:30:43.269 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 Processor -- start 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 -- start start 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 0x7f3fe80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3fe80d1830 0x7f3fe80629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe8160e40 con 0x7f3fe80615f0 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe81609a0 con 0x7f3fe80d1830 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fed120700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8160ba0 con 0x7f3fe815f5b0 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fe659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3fe80615f0 0x7f3fe80622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46490/0 (socket says 172.21.15.181:46490) 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fe6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 0x7f3fe80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fe659c700 1 -- 172.21.15.181:0/2988855265 learned_addr learned my addr 172.21.15.181:0/2988855265 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.413+0000 7f3fe759e700 1 --2- 172.21.15.181:0/2988855265 >> v2:172.21.15.175:3300/0 conn(0x7f3fe80d1830 0x7f3fe80629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe5d9b700 1 -- 172.21.15.181:0/2988855265 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692865737 0 0) 0x7f3fe8160e40 con 0x7f3fe80615f0 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe5d9b700 1 -- 172.21.15.181:0/2988855265 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003240 con 0x7f3fe80615f0 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/2988855265 >> v2:172.21.15.175:3300/0 conn(0x7f3fe80d1830 msgr2=0x7f3fe80629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe6d9d700 1 --2- 172.21.15.181:0/2988855265 >> v2:172.21.15.175:3300/0 conn(0x7f3fe80d1830 0x7f3fe80629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/2988855265 >> v1:172.21.15.181:6789/0 conn(0x7f3fe80615f0 legacy=0x7f3fe80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/2988855265 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8060970 con 0x7f3fe815f5b0 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe6d9d700 1 --2- 172.21.15.181:0/2988855265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 0x7f3fe80d1340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3fdc00b990 tx=0x7f3fdc006d90).ready entity=mon.2 client_cookie=74c090c2c9bf615f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe5d9b700 1 -- 172.21.15.181:0/2988855265 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc006030 con 0x7f3fe815f5b0 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe5d9b700 1 -- 172.21.15.181:0/2988855265 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc006190 con 0x7f3fe815f5b0 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fe5d9b700 1 -- 172.21.15.181:0/2988855265 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc006510 con 0x7f3fe815f5b0 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 -- 172.21.15.181:0/2988855265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 msgr2=0x7f3fe80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 --2- 172.21.15.181:0/2988855265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 0x7f3fe80d1340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3fdc00b990 tx=0x7f3fdc006d90).stop 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 -- 172.21.15.181:0/2988855265 shutdown_connections 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 --2- 172.21.15.181:0/2988855265 >> v2:172.21.15.175:3300/0 conn(0x7f3fe80d1830 0x7f3fe80629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 --2- 172.21.15.181:0/2988855265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe815f5b0 0x7f3fe80d1340 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 -- 172.21.15.181:0/2988855265 >> 172.21.15.181:0/2988855265 conn(0x7f3fe80d69a0 msgr2=0x7f3fe80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 -- 172.21.15.181:0/2988855265 shutdown_connections 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 -- 172.21.15.181:0/2988855265 wait complete. 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.417+0000 7f3fed120700 1 Processor -- start 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- start start 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80d1830 0x7f3fe8162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 0x7f3fe8162930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8162e20 con 0x7f3fe80d1830 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81700d0 con 0x7f3fe815f5b0 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80bb960 con 0x7f3fe80615f0 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40034/0 (socket says 172.21.15.181:40034) 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/510737011 learned_addr learned my addr 172.21.15.181:0/510737011 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe759e700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 0x7f3fe8162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe659c700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80d1830 0x7f3fe8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 msgr2=0x7f3fe8162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 0x7f3fe8162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/510737011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80d1830 msgr2=0x7f3fe8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80d1830 0x7f3fe8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 -- 172.21.15.181:0/510737011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80bbc00 con 0x7f3fe80615f0 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe759e700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 0x7f3fe8162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fe6d9d700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3fe8160ce0 tx=0x7f3fdc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc009850 con 0x7f3fe80615f0 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0099b0 con 0x7f3fe80615f0 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80bbea0 con 0x7f3fe80615f0 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc009d50 con 0x7f3fe80615f0 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.421+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80bc3e0 con 0x7f3fe80615f0 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80bc5a0 con 0x7f3fe80615f0 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3fdc025030 con 0x7f3fe80615f0 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fceffd700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 0x7f3fc4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3fdc06f460 con 0x7f3fe80615f0 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fe659c700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 0x7f3fc4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.425+0000 7f3fe659c700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 0x7f3fc4057550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3fe80b72a0 tx=0x7f3fd8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.433+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3fdc0455b0 con 0x7f3fe80615f0 2020-11-16T22:30:43.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3fe8069eb0 con 0x7f3fe80615f0 2020-11-16T22:30:43.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fceffd700 1 -- 172.21.15.181:0/510737011 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fdc048bd0 con 0x7f3fe80615f0 2020-11-16T22:30:43.562 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:30:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 msgr2=0x7f3fc4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 0x7f3fc4057550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3fe80b72a0 tx=0x7f3fd8009040).stop 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 msgr2=0x7f3fe814b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3fe8160ce0 tx=0x7f3fdc015000).stop 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 shutdown_connections 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fc4055090 0x7f3fc4057550 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe815f5b0 0x7f3fe8162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe80615f0 0x7f3fe814b500 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 --2- 172.21.15.181:0/510737011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fe80d1830 0x7f3fe8162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 >> 172.21.15.181:0/510737011 conn(0x7f3fe80d69a0 msgr2=0x7f3fe80d5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 shutdown_connections 2020-11-16T22:30:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.581+0000 7f3fed120700 1 -- 172.21.15.181:0/510737011 wait complete. 2020-11-16T22:30:43.576 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:30:43.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 Processor -- start 2020-11-16T22:30:43.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 -- start start 2020-11-16T22:30:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 0x7f2d7c0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2d7c0605e0 0x7f2d7c0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0ce340 con 0x7f2d7c0605e0 2020-11-16T22:30:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0ce5e0 con 0x7f2d7c149110 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0ce880 con 0x7f2d7c148d30 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.721+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 0x7f2d7c0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 0x7f2d7c0600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60434/0 (socket says 172.21.15.181:60434) 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7a7fc700 1 -- 172.21.15.181:0/26922987 learned_addr learned my addr 172.21.15.181:0/26922987 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40042/0 (socket says 172.21.15.181:40042) 2020-11-16T22:30:43.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7b7fe700 1 --2- 172.21.15.181:0/26922987 >> v2:172.21.15.175:3300/0 conn(0x7f2d7c0605e0 0x7f2d7c0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 -- 172.21.15.181:0/26922987 >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 msgr2=0x7f2d7c0600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 --2- 172.21.15.181:0/26922987 >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 0x7f2d7c0600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 -- 172.21.15.181:0/26922987 >> v2:172.21.15.175:3300/0 conn(0x7f2d7c0605e0 msgr2=0x7f2d7c0c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 --2- 172.21.15.181:0/26922987 >> v2:172.21.15.175:3300/0 conn(0x7f2d7c0605e0 0x7f2d7c0c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 -- 172.21.15.181:0/26922987 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0ceb20 con 0x7f2d7c148d30 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7affd700 1 --2- 172.21.15.181:0/26922987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2d7c067c80 tx=0x7f2d6c0091d0).ready entity=mon.2 client_cookie=c57474f72b878f6d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d79ffb700 1 -- 172.21.15.181:0/26922987 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00b9c0 con 0x7f2d7c148d30 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d79ffb700 1 -- 172.21.15.181:0/26922987 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c00bb20 con 0x7f2d7c148d30 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d79ffb700 1 -- 172.21.15.181:0/26922987 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00bea0 con 0x7f2d7c148d30 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 -- 172.21.15.181:0/26922987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 msgr2=0x7f2d7c0cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/26922987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2d7c067c80 tx=0x7f2d6c0091d0).stop 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 -- 172.21.15.181:0/26922987 shutdown_connections 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/26922987 >> v2:172.21.15.175:3300/0 conn(0x7f2d7c0605e0 0x7f2d7c0c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/26922987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c148d30 0x7f2d7c0cdd90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/26922987 >> v2:172.21.15.181:3300/0 conn(0x7f2d7c149110 0x7f2d7c0600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 -- 172.21.15.181:0/26922987 >> 172.21.15.181:0/26922987 conn(0x7f2d7c0d33f0 msgr2=0x7f2d7c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 -- 172.21.15.181:0/26922987 shutdown_connections 2020-11-16T22:30:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 -- 172.21.15.181:0/26922987 wait complete. 2020-11-16T22:30:43.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.725+0000 7f2d7bfff700 1 Processor -- start 2020-11-16T22:30:43.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- start start 2020-11-16T22:30:43.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c148d30 0x7f2d7c145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7c149110 0x7f2d7c0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c164e90 con 0x7f2d7c148d30 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cbe40 con 0x7f2d7c149110 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cc0e0 con 0x7f2d7c0605e0 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40048/0 (socket says 172.21.15.181:40048) 2020-11-16T22:30:43.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 -- 172.21.15.181:0/3673344802 learned_addr learned my addr 172.21.15.181:0/3673344802 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7b7fe700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7c149110 0x7f2d7c0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7a7fc700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c148d30 0x7f2d7c145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 -- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7c149110 msgr2=0x7f2d7c0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7c149110 0x7f2d7c0c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 -- 172.21.15.181:0/3673344802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c148d30 msgr2=0x7f2d7c145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c148d30 0x7f2d7c145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 -- 172.21.15.181:0/3673344802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0cc380 con 0x7f2d7c0605e0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7affd700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2d6c009440 tx=0x7f2d6c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c01d480 con 0x7f2d7c0605e0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c0cc620 con 0x7f2d7c0605e0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c01d5e0 con 0x7f2d7c0605e0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c01d9a0 con 0x7f2d7c0605e0 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.729+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c1704b0 con 0x7f2d7c0605e0 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c044f60 con 0x7f2d7c0605e0 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2d6c017070 con 0x7f2d7c0605e0 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d62ffd700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 0x7f2d58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d6c070040 con 0x7f2d7c0605e0 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d7a7fc700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 0x7f2d58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.733+0000 7f2d7a7fc700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 0x7f2d58057520 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f2d7c0cb8d0 tx=0x7f2d70008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.737+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2d6c046350 con 0x7f2d7c0605e0 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d7c068fb0 con 0x7f2d7c0605e0 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d62ffd700 1 -- 172.21.15.181:0/3673344802 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2d6c015070 con 0x7f2d7c0605e0 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 msgr2=0x7f2d58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 0x7f2d58057520 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f2d7c0cb8d0 tx=0x7f2d70008040).stop 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 msgr2=0x7f2d7c145510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2d6c009440 tx=0x7f2d6c016000).stop 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 shutdown_connections 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d58055060 0x7f2d58057520 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7c149110 0x7f2d7c0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d7c0605e0 0x7f2d7c145510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/3673344802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c148d30 0x7f2d7c145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 >> 172.21.15.181:0/3673344802 conn(0x7f2d7c0d33f0 msgr2=0x7f2d7c0d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 shutdown_connections 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:43.893+0000 7f2d7bfff700 1 -- 172.21.15.181:0/3673344802 wait complete. 2020-11-16T22:30:43.888 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:30:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.029+0000 7ff9aef01700 1 Processor -- start 2020-11-16T22:30:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.029+0000 7ff9aef01700 1 -- start start 2020-11-16T22:30:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff9a8062960 0x7ff9a80644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80c8db0 con 0x7ff9a8161860 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a8165030 con 0x7ff9a8161480 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80c8330 con 0x7ff9a8062960 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9adeff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9a8161480 0x7ff9a8164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46520/0 (socket says 172.21.15.181:46520) 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9adeff700 1 -- 172.21.15.181:0/2125516978 learned_addr learned my addr 172.21.15.181:0/2125516978 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9ae700700 1 --2- 172.21.15.181:0/2125516978 >> v2:172.21.15.175:3300/0 conn(0x7ff9a8062960 0x7ff9a80644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96456493 0 0) 0x7ff9a8165030 con 0x7ff9a8161480 2020-11-16T22:30:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff990003240 con 0x7ff9a8161480 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9ae700700 1 -- 172.21.15.181:0/2125516978 >> v1:172.21.15.181:6789/0 conn(0x7ff9a8161480 legacy=0x7ff9a8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9ae700700 1 -- 172.21.15.181:0/2125516978 >> v1:172.21.15.175:6789/0 conn(0x7ff9a8161860 legacy=0x7ff9a8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699589232 0 0) 0x7ff9a80c8db0 con 0x7ff9a8161860 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9ae700700 1 -- 172.21.15.181:0/2125516978 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80c85d0 con 0x7ff9a8062960 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9ae700700 1 --2- 172.21.15.181:0/2125516978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8062960 0x7ff9a80644e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ff9a0009c90 tx=0x7ff9a00091d0).ready entity=mon.0 client_cookie=5bb41bb62c3fcaeb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000ba70 con 0x7ff9a8062960 2020-11-16T22:30:44.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000bbd0 con 0x7ff9a8062960 2020-11-16T22:30:44.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9acefd700 1 -- 172.21.15.181:0/2125516978 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0011400 con 0x7ff9a8062960 2020-11-16T22:30:44.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- 172.21.15.181:0/2125516978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8062960 msgr2=0x7ff9a80644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 --2- 172.21.15.181:0/2125516978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8062960 0x7ff9a80644e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ff9a0009c90 tx=0x7ff9a00091d0).stop 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- 172.21.15.181:0/2125516978 shutdown_connections 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 --2- 172.21.15.181:0/2125516978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8062960 0x7ff9a80644e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- 172.21.15.181:0/2125516978 >> 172.21.15.181:0/2125516978 conn(0x7ff9a80d2dc0 msgr2=0x7ff9a80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- 172.21.15.181:0/2125516978 shutdown_connections 2020-11-16T22:30:44.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.033+0000 7ff9aef01700 1 -- 172.21.15.181:0/2125516978 wait complete. 2020-11-16T22:30:44.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.037+0000 7ff9aef01700 1 Processor -- start 2020-11-16T22:30:44.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.037+0000 7ff9aef01700 1 -- start start 2020-11-16T22:30:44.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8161480 0x7ff9a8169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a8161860 0x7ff9a816a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8174510 con 0x7ff9a8161480 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80cc4e0 con 0x7ff9a8161860 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81737f0 con 0x7ff9a8062960 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40062/0 (socket says 172.21.15.181:40062) 2020-11-16T22:30:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 -- 172.21.15.181:0/985848378 learned_addr learned my addr 172.21.15.181:0/985848378 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:44.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9ae700700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a8161860 0x7ff9a816a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9ad6fe700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8161480 0x7ff9a8169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 -- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a8161860 msgr2=0x7ff9a816a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a8161860 0x7ff9a816a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 -- 172.21.15.181:0/985848378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8161480 msgr2=0x7ff9a8169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8161480 0x7ff9a8169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 -- 172.21.15.181:0/985848378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8173a90 con 0x7ff9a8062960 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9adeff700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff9a4001e90 tx=0x7ff9a4007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40072e0 con 0x7ff9a8062960 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a4007440 con 0x7ff9a8062960 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4005820 con 0x7ff9a8062960 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a8173d30 con 0x7ff9a8062960 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.041+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a8173510 con 0x7ff9a8062960 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a8068740 con 0x7ff9a8062960 2020-11-16T22:30:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff9a4010070 con 0x7ff9a8062960 2020-11-16T22:30:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff995ffb700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 0x7ff980057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff9ad6fe700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 0x7ff980057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff9a406d4b0 con 0x7ff9a8062960 2020-11-16T22:30:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.045+0000 7ff9ad6fe700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 0x7ff980057520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff9a80b8530 tx=0x7ff9980079b0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.053+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff9a8068740 con 0x7ff9a8062960 2020-11-16T22:30:44.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9a8068740 con 0x7ff9a8062960 2020-11-16T22:30:44.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff995ffb700 1 -- 172.21.15.181:0/985848378 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff9a8068740 con 0x7ff9a8062960 2020-11-16T22:30:44.187 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 msgr2=0x7ff980057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 0x7ff980057520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff9a80b8530 tx=0x7ff9980079b0).stop 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 msgr2=0x7ff9a80cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7ff9a4001e90 tx=0x7ff9a4007ac0).stop 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.205+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 shutdown_connections 2020-11-16T22:30:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff980055060 0x7ff980057520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a8161860 0x7ff9a816a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9a8062960 0x7ff9a80cbff0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 --2- 172.21.15.181:0/985848378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9a8161480 0x7ff9a8169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 >> 172.21.15.181:0/985848378 conn(0x7ff9a80d2dc0 msgr2=0x7ff9a80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 shutdown_connections 2020-11-16T22:30:44.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.209+0000 7ff9aef01700 1 -- 172.21.15.181:0/985848378 wait complete. 2020-11-16T22:30:44.202 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:44.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.341+0000 7fbab7fff700 1 Processor -- start 2020-11-16T22:30:44.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 -- start start 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbab813ec50 0x7fbab8061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab80615f0 0x7fbab815e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab815d180 con 0x7fbab813e870 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab815d810 con 0x7fbab813ec50 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab815dab0 con 0x7fbab80615f0 2020-11-16T22:30:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbab813e870 0x7fbab815fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46532/0 (socket says 172.21.15.181:46532) 2020-11-16T22:30:44.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab6ffd700 1 -- 172.21.15.181:0/2277965649 learned_addr learned my addr 172.21.15.181:0/2277965649 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:44.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 --2- 172.21.15.181:0/2277965649 >> v2:172.21.15.181:3300/0 conn(0x7fbab813ec50 0x7fbab8061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab77fe700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab80615f0 0x7fbab815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 -- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab80615f0 msgr2=0x7fbab815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:44.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab80615f0 0x7fbab815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 -- 172.21.15.181:0/2277965649 >> v1:172.21.15.181:6789/0 conn(0x7fbab813e870 legacy=0x7fbab815fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 -- 172.21.15.181:0/2277965649 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab815dd50 con 0x7fbab813ec50 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab67fc700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813ec50 0x7fbab8061100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbaac00b9a0 tx=0x7fbaac006df0).ready entity=mon.1 client_cookie=f0c74379912476bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.345+0000 7fbab5ffb700 1 -- 172.21.15.181:0/2277965649 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac014070 con 0x7fbab813ec50 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab5ffb700 1 -- 172.21.15.181:0/2277965649 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac0097b0 con 0x7fbab813ec50 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab5ffb700 1 -- 172.21.15.181:0/2277965649 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac009b30 con 0x7fbab813ec50 2020-11-16T22:30:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813ec50 msgr2=0x7fbab8061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813ec50 0x7fbab8061100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbaac00b9a0 tx=0x7fbaac006df0).stop 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- 172.21.15.181:0/2277965649 shutdown_connections 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab80615f0 0x7fbab815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 --2- 172.21.15.181:0/2277965649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813ec50 0x7fbab8061100 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- 172.21.15.181:0/2277965649 >> 172.21.15.181:0/2277965649 conn(0x7fbab80cf630 msgr2=0x7fbab80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- 172.21.15.181:0/2277965649 shutdown_connections 2020-11-16T22:30:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- 172.21.15.181:0/2277965649 wait complete. 2020-11-16T22:30:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 Processor -- start 2020-11-16T22:30:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.349+0000 7fbab7fff700 1 -- start start 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbab80615f0 0x7fbab80c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 0x7fbab81390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80c9770 con 0x7fbab80615f0 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80c9a10 con 0x7fbab813e870 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab816b2d0 con 0x7fbab813ec50 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 0x7fbab81390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40080/0 (socket says 172.21.15.181:40080) 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 0x7fbab81390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60468/0 (socket says 172.21.15.181:60468) 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 -- 172.21.15.181:0/4069582454 learned_addr learned my addr 172.21.15.181:0/4069582454 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab6ffd700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbab80615f0 0x7fbab80c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 -- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 msgr2=0x7fbab81390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 0x7fbab81390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 -- 172.21.15.181:0/4069582454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbab80615f0 msgr2=0x7fbab80c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbab80615f0 0x7fbab80c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 -- 172.21.15.181:0/4069582454 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab816b570 con 0x7fbab813ec50 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab77fe700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbab000ef20 tx=0x7fbab00091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cb00 con 0x7fbab813ec50 2020-11-16T22:30:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab000cc60 con 0x7fbab813ec50 2020-11-16T22:30:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab816b810 con 0x7fbab813ec50 2020-11-16T22:30:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab00123f0 con 0x7fbab813ec50 2020-11-16T22:30:44.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.353+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab816bcb0 con 0x7fbab813ec50 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab816b9a0 con 0x7fbab813ec50 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbab0022070 con 0x7fbab813ec50 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fba9effd700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 0x7fba94057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbab006e470 con 0x7fbab813ec50 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fbab6ffd700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 0x7fba94057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.357+0000 7fbab6ffd700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 0x7fba94057550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fbaa80072f0 tx=0x7fbaa8007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.361+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbab00446d0 con 0x7fbab813ec50 2020-11-16T22:30:44.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.561+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbab8044f60 con 0x7fbab813ec50 2020-11-16T22:30:44.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fba9effd700 1 -- 172.21.15.181:0/4069582454 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4666 (secure 0 0 0) 0x7fbab0047cf0 con 0x7fbab813ec50 2020-11-16T22:30:44.546 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:44.546 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":56,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5294,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2966940494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2966940494},{"type":"v1","addr":"172.21.15.175:6835","nonce":2966940494}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/2764691515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2764691515},{"type":"v1","addr":"172.21.15.181:6839","nonce":2764691515}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/1854821642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":1854821642},{"type":"v1","addr":"172.21.15.181:6841","nonce":1854821642}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":5553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3944223690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3944223690},{"type":"v1","addr":"172.21.15.175:6833","nonce":3944223690}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5574,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/2134370940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":2134370940},{"type":"v1","addr":"172.21.15.175:6839","nonce":2134370940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5577,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/867635159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":867635159},{"type":"v1","addr":"172.21.15.175:6841","nonce":867635159}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:29:30.354786+0000","modified":"2020-11-16T22:30:26.700406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5614,"mds_1":5605,"mds_2":5599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5599":{"gid":5599,"name":"a","rank":2,"incarnation":50,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6835/3960101400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960101400},{"type":"v1","addr":"172.21.15.181:6835","nonce":3960101400}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5605":{"gid":5605,"name":"b","rank":1,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2557321074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2557321074},{"type":"v1","addr":"172.21.15.181:6837","nonce":2557321074}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5614":{"gid":5614,"name":"i","rank":0,"incarnation":43,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/2753478724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2753478724},{"type":"v1","addr":"172.21.15.175:6837","nonce":2753478724}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:30:44.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 msgr2=0x7fba94057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 0x7fba94057550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fbaa80072f0 tx=0x7fbaa8007b80).stop 2020-11-16T22:30:44.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 msgr2=0x7fbab81395c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fbab000ef20 tx=0x7fbab00091d0).stop 2020-11-16T22:30:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 shutdown_connections 2020-11-16T22:30:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fba94055090 0x7fba94057550 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbab813ec50 0x7fbab81395c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbab80615f0 0x7fbab80c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 --2- 172.21.15.181:0/4069582454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab813e870 0x7fbab81390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 >> 172.21.15.181:0/4069582454 conn(0x7fbab80cf630 msgr2=0x7fbab813dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 shutdown_connections 2020-11-16T22:30:44.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.565+0000 7fbab7fff700 1 -- 172.21.15.181:0/4069582454 wait complete. 2020-11-16T22:30:44.549 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 56 2020-11-16T22:30:44.560 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:44.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 Processor -- start 2020-11-16T22:30:44.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- start start 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f480633d0 0x7f0f48161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f480615f0 0x7f0f48062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f48065930 con 0x7f0f480615f0 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f481621b0 con 0x7f0f480633d0 2020-11-16T22:30:44.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48064970 con 0x7f0f480637b0 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f480633d0 0x7f0f48161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40086/0 (socket says 172.21.15.181:40086) 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f480633d0 0x7f0f48161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60474/0 (socket says 172.21.15.181:60474) 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f46ffd700 1 -- 172.21.15.181:0/2506484981 learned_addr learned my addr 172.21.15.181:0/2506484981 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f47fff700 1 --2- 172.21.15.181:0/2506484981 >> v2:172.21.15.175:3300/0 conn(0x7f0f480615f0 0x7f0f48062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 -- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 msgr2=0x7f0f48061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 -- 172.21.15.181:0/2506484981 >> v2:172.21.15.175:3300/0 conn(0x7f0f480615f0 msgr2=0x7f0f48062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 --2- 172.21.15.181:0/2506484981 >> v2:172.21.15.175:3300/0 conn(0x7f0f480615f0 0x7f0f48062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 -- 172.21.15.181:0/2506484981 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48064c10 con 0x7f0f480633d0 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f46ffd700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:44.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f477fe700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480633d0 0x7f0f48161cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f0f480675a0 tx=0x7f0f38006df0).ready entity=mon.1 client_cookie=7e498be523daaaa0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f467fc700 1 -- 172.21.15.181:0/2506484981 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009640 con 0x7f0f480633d0 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f467fc700 1 -- 172.21.15.181:0/2506484981 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f380097a0 con 0x7f0f480633d0 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f467fc700 1 -- 172.21.15.181:0/2506484981 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009b20 con 0x7f0f480633d0 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480633d0 msgr2=0x7f0f48161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480633d0 0x7f0f48161cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f0f480675a0 tx=0x7f0f38006df0).stop 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- 172.21.15.181:0/2506484981 shutdown_connections 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/2506484981 >> v2:172.21.15.175:3300/0 conn(0x7f0f480615f0 0x7f0f48062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480633d0 0x7f0f48161cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/2506484981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480637b0 0x7f0f48061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- 172.21.15.181:0/2506484981 >> 172.21.15.181:0/2506484981 conn(0x7f0f480cf890 msgr2=0x7f0f480cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- 172.21.15.181:0/2506484981 shutdown_connections 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.701+0000 7f0f4dba9700 1 -- 172.21.15.181:0/2506484981 wait complete. 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 Processor -- start 2020-11-16T22:30:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- start start 2020-11-16T22:30:44.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 0x7f0f4814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f480637b0 0x7f0f48160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48173ab0 con 0x7f0f480637b0 2020-11-16T22:30:44.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48172d90 con 0x7f0f480615f0 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48173030 con 0x7f0f480633d0 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60480/0 (socket says 172.21.15.181:60480) 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 0x7f0f4814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f477fe700 1 -- 172.21.15.181:0/3137533252 learned_addr learned my addr 172.21.15.181:0/3137533252 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f47fff700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f480637b0 0x7f0f48160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 -- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 msgr2=0x7f0f4814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 -- 172.21.15.181:0/3137533252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f480637b0 msgr2=0x7f0f48160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f480637b0 0x7f0f48160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 -- 172.21.15.181:0/3137533252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f481732d0 con 0x7f0f480633d0 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f477fe700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f46ffd700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 0x7f0f4814ddb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f0f3c00ecf0 tx=0x7f0f3c006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00b580 con 0x7f0f480633d0 2020-11-16T22:30:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c00ca10 con 0x7f0f480633d0 2020-11-16T22:30:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f48173570 con 0x7f0f480633d0 2020-11-16T22:30:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00cd90 con 0x7f0f480633d0 2020-11-16T22:30:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.705+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f48160800 con 0x7f0f480633d0 2020-11-16T22:30:44.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f48069eb0 con 0x7f0f480633d0 2020-11-16T22:30:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0f3c022070 con 0x7f0f480633d0 2020-11-16T22:30:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f277fe700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 0x7f0f20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0f3c06e380 con 0x7f0f480633d0 2020-11-16T22:30:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f477fe700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 0x7f0f20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.709+0000 7f0f477fe700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 0x7f0f20057520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f0f38009340 tx=0x7f0f38015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:44.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.713+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0f3c0445e0 con 0x7f0f480633d0 2020-11-16T22:30:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.869+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f4805d4e0 con 0x7f0f480633d0 2020-11-16T22:30:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.869+0000 7f0f277fe700 1 -- 172.21.15.181:0/3137533252 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f0f3c047c00 con 0x7f0f480633d0 2020-11-16T22:30:44.852 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:44.853 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:29:31.359585+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:29:28.703069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:29:29.511094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 msgr2=0x7f0f20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 0x7f0f20057520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f0f38009340 tx=0x7f0f38015000).stop 2020-11-16T22:30:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 msgr2=0x7f0f4814ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:44.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 0x7f0f4814ddb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f0f3c00ecf0 tx=0x7f0f3c006eb0).stop 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 shutdown_connections 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f20055060 0x7f0f20057520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f480637b0 0x7f0f48160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f480615f0 0x7f0f4814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 --2- 172.21.15.181:0/3137533252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f480633d0 0x7f0f4814ddb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 >> 172.21.15.181:0/3137533252 conn(0x7f0f480cf890 msgr2=0x7f0f480cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:44.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 shutdown_connections 2020-11-16T22:30:44.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:44.873+0000 7f0f4dba9700 1 -- 172.21.15.181:0/3137533252 wait complete. 2020-11-16T22:30:44.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:30:44.872 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:30:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 Processor -- start 2020-11-16T22:30:44.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- start start 2020-11-16T22:30:44.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2888062180 0x7f2888065900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:44.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2888164180 con 0x7f2888167640 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2888167c90 con 0x7f2888063f30 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2888062c70 con 0x7f2888062180 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288cda7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2888063f30 0x7f2888062560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46564/0 (socket says 172.21.15.181:46564) 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288cda7700 1 -- 172.21.15.181:0/2994178405 learned_addr learned my addr 172.21.15.181:0/2994178405 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288d5a8700 1 --2- 172.21.15.181:0/2994178405 >> v2:172.21.15.175:3300/0 conn(0x7f2888062180 0x7f2888065900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929642642 0 0) 0x7f2888167c90 con 0x7f2888063f30 2020-11-16T22:30:45.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f286c003240 con 0x7f2888063f30 2020-11-16T22:30:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283684197 0 0) 0x7f2888164180 con 0x7f2888167640 2020-11-16T22:30:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2888167c90 con 0x7f2888167640 2020-11-16T22:30:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288d5a8700 1 -- 172.21.15.181:0/2994178405 >> v1:172.21.15.181:6789/0 conn(0x7f2888063f30 legacy=0x7f2888062560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288d5a8700 1 -- 172.21.15.181:0/2994178405 >> v1:172.21.15.175:6789/0 conn(0x7f2888167640 legacy=0x7f2888164850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288d5a8700 1 -- 172.21.15.181:0/2994178405 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888167f30 con 0x7f2888062180 2020-11-16T22:30:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288d5a8700 1 --2- 172.21.15.181:0/2994178405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888062180 0x7f2888065900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f288400b9a0 tx=0x7f2884006df0).ready entity=mon.0 client_cookie=790332331d9ea1c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884009710 con 0x7f2888062180 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884009870 con 0x7f2888062180 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f287bfff700 1 -- 172.21.15.181:0/2994178405 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884009bf0 con 0x7f2888062180 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- 172.21.15.181:0/2994178405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888062180 msgr2=0x7f2888065900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 --2- 172.21.15.181:0/2994178405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888062180 0x7f2888065900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f288400b9a0 tx=0x7f2884006df0).stop 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- 172.21.15.181:0/2994178405 shutdown_connections 2020-11-16T22:30:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 --2- 172.21.15.181:0/2994178405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888062180 0x7f2888065900 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.017+0000 7f288e5aa700 1 -- 172.21.15.181:0/2994178405 >> 172.21.15.181:0/2994178405 conn(0x7f28880d2310 msgr2=0x7f28880d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- 172.21.15.181:0/2994178405 shutdown_connections 2020-11-16T22:30:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- 172.21.15.181:0/2994178405 wait complete. 2020-11-16T22:30:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 Processor -- start 2020-11-16T22:30:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- start start 2020-11-16T22:30:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2888063f30 0x7f2888148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888167640 0x7f288814f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b78b0 con 0x7f2888167640 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b7b50 con 0x7f2888063f30 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288e5aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b7df0 con 0x7f2888062180 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40104/0 (socket says 172.21.15.181:40104) 2020-11-16T22:30:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 -- 172.21.15.181:0/4097882090 learned_addr learned my addr 172.21.15.181:0/4097882090 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288cda7700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2888063f30 0x7f2888148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288dda9700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888167640 0x7f288814f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 -- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2888063f30 msgr2=0x7f2888148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2888063f30 0x7f2888148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 -- 172.21.15.181:0/4097882090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888167640 msgr2=0x7f288814f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888167640 0x7f288814f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 -- 172.21.15.181:0/4097882090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880b8090 con 0x7f2888062180 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.021+0000 7f288d5a8700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f28880cc940 tx=0x7f2884015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011d80 con 0x7f2888062180 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288401e460 con 0x7f2888062180 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28880b8330 con 0x7f2888062180 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288401e7e0 con 0x7f2888062180 2020-11-16T22:30:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28881617b0 con 0x7f2888062180 2020-11-16T22:30:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28880b76e0 con 0x7f2888062180 2020-11-16T22:30:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2884025030 con 0x7f2888062180 2020-11-16T22:30:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 0x7f285c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f288406f380 con 0x7f2888062180 2020-11-16T22:30:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f288cda7700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 0x7f285c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.025+0000 7f288cda7700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 0x7f285c057520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f287c0055a0 tx=0x7f287c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.033+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2884045690 con 0x7f2888062180 2020-11-16T22:30:45.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.233+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2888044f60 con 0x7f2888062180 2020-11-16T22:30:45.840 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:30:45.860+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:30:45.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.877+0000 7f2879ffb700 1 -- 172.21.15.181:0/4097882090 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2884023070 con 0x7f2888062180 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 msgr2=0x7f285c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 0x7f285c057520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f287c0055a0 tx=0x7f287c009040).stop 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 msgr2=0x7f28880b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f28880cc940 tx=0x7f2884015000).stop 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 shutdown_connections 2020-11-16T22:30:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f285c055060 0x7f285c057520 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2888167640 0x7f288814f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2888062180 0x7f28880b71f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 --2- 172.21.15.181:0/4097882090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2888063f30 0x7f2888148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 >> 172.21.15.181:0/4097882090 conn(0x7f28880d2310 msgr2=0x7f28881665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 shutdown_connections 2020-11-16T22:30:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:45.881+0000 7f288e5aa700 1 -- 172.21.15.181:0/4097882090 wait complete. 2020-11-16T22:30:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:30:45.865 INFO:tasks.ceph.mds.b.smithi181.stderr: -1129> 2020-11-16T22:30:18.277+0000 7f1794df4700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1) n(v2 rc2020-11-16T22:29:48.195730+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@4},l=5782 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563a50ced180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:29:48.195730+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.866 INFO:tasks.ceph.mds.b.smithi181.stderr: -1010> 2020-11-16T22:30:18.281+0000 7f179ae00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1) n(v1 rc2020-11-16T22:29:48.195730+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5782=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563a50ced700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.107730+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:29:48.195730+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.867 INFO:tasks.ceph.mds.a.smithi181.stderr: -942> 2020-11-16T22:30:18.281+0000 7f07ae857700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1) n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@4},l=5782 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560bd09f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.111730+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:29:45.111730+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.867 INFO:tasks.ceph.mds.a.smithi181.stderr: -929> 2020-11-16T22:30:18.281+0000 7f07ae857700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/AsLsXsFsx@2},l=5782 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x560bd0ac6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.870 INFO:tasks.ceph.mds.i.smithi175.stderr: -1865> 2020-11-16T22:30:18.280+0000 7f642ec30700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1) n(v4 rc2020-11-16T22:29:48.195730+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={5782=pAsLsXsFsx/pAsLsXsFsx@5},l=5782 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5569a01c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.099730+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:29:48.195730+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.870 INFO:tasks.ceph.mds.i.smithi175.stderr: -1779> 2020-11-16T22:30:18.280+0000 7f6434c3c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1) n(v3 rc2020-11-16T22:29:48.195730+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5782=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5569a01c6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:29:48.195730+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:29:45.103730+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:29:48.195730+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:30:45.882 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:30:45.887 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:30:45.887 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:30:45.891 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:30:46.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.021+0000 7f4ae24d5700 1 Processor -- start 2020-11-16T22:30:46.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.021+0000 7f4ae24d5700 1 -- start start 2020-11-16T22:30:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae24d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4adc0b7c10 0x7f4adc0b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:46.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae24d5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0ceb10 con 0x7f4adc15d510 2020-11-16T22:30:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae24d5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0cedb0 con 0x7f4adc15d8f0 2020-11-16T22:30:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae24d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc0b8050 con 0x7f4adc0b7c10 2020-11-16T22:30:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae0a72700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4adc0b7c10 0x7f4adc0b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:46.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4adc15d8f0 0x7f4adc0b7500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46600/0 (socket says 172.21.15.181:46600) 2020-11-16T22:30:46.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adb7fe700 1 -- 172.21.15.181:0/1075026838 learned_addr learned my addr 172.21.15.181:0/1075026838 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418150618 0 0) 0x7f4adc0cedb0 con 0x7f4adc15d8f0 2020-11-16T22:30:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac8003240 con 0x7f4adc15d8f0 2020-11-16T22:30:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae0a72700 1 -- 172.21.15.181:0/1075026838 >> v1:172.21.15.181:6789/0 conn(0x7f4adc15d8f0 legacy=0x7f4adc0b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70153897 0 0) 0x7f4adc0ceb10 con 0x7f4adc15d510 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae0a72700 1 -- 172.21.15.181:0/1075026838 >> v1:172.21.15.175:6789/0 conn(0x7f4adc15d510 legacy=0x7f4adc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae0a72700 1 -- 172.21.15.181:0/1075026838 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0cf050 con 0x7f4adc0b7c10 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4ae0a72700 1 --2- 172.21.15.181:0/1075026838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc0b89c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4ad4009c90 tx=0x7f4ad40091d0).ready entity=mon.1 client_cookie=61894c83f3d914cd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400ba70 con 0x7f4adc0b7c10 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400bbd0 con 0x7f4adc0b7c10 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.025+0000 7f4adaffd700 1 -- 172.21.15.181:0/1075026838 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011400 con 0x7f4adc0b7c10 2020-11-16T22:30:46.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 -- 172.21.15.181:0/1075026838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 msgr2=0x7f4adc0b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/1075026838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc0b89c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4ad4009c90 tx=0x7f4ad40091d0).stop 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 -- 172.21.15.181:0/1075026838 shutdown_connections 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/1075026838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc0b89c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 -- 172.21.15.181:0/1075026838 >> 172.21.15.181:0/1075026838 conn(0x7f4adc0d3680 msgr2=0x7f4adc0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 -- 172.21.15.181:0/1075026838 shutdown_connections 2020-11-16T22:30:46.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.029+0000 7f4ae24d5700 1 -- 172.21.15.181:0/1075026838 wait complete. 2020-11-16T22:30:46.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 Processor -- start 2020-11-16T22:30:46.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 -- start start 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 0x7f4adc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 0x7f4adc15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15e1a0 con 0x7f4adc15d8f0 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15e440 con 0x7f4adc0b7c10 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae24d5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15e6e0 con 0x7f4adc15d510 2020-11-16T22:30:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 0x7f4adc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60532/0 (socket says 172.21.15.181:60532) 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adbfff700 1 -- 172.21.15.181:0/13014903 learned_addr learned my addr 172.21.15.181:0/13014903 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae0a72700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 0x7f4adc15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 -- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 msgr2=0x7f4adc158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 -- 172.21.15.181:0/13014903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 msgr2=0x7f4adc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 0x7f4adc15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 -- 172.21.15.181:0/13014903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc15e980 con 0x7f4adc15d510 2020-11-16T22:30:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4ae0a72700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 0x7f4adc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4adb7fe700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 0x7f4adc15f360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f4ad000c6e0 tx=0x7f4ad0007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.033+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00072e0 con 0x7f4adc15d510 2020-11-16T22:30:46.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.037+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad0007440 con 0x7f4adc15d510 2020-11-16T22:30:46.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.037+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0005800 con 0x7f4adc15d510 2020-11-16T22:30:46.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.037+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc15ec20 con 0x7f4adc15d510 2020-11-16T22:30:46.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.037+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc159430 con 0x7f4adc15d510 2020-11-16T22:30:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.037+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc069eb0 con 0x7f4adc15d510 2020-11-16T22:30:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.041+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4ad0010070 con 0x7f4adc15d510 2020-11-16T22:30:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.041+0000 7f4abbfff700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 0x7f4ab4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.041+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f4ad006d970 con 0x7f4adc15d510 2020-11-16T22:30:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.041+0000 7f4adbfff700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 0x7f4ab4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:46.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.041+0000 7f4adbfff700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 0x7f4ab4057520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f4acc003400 tx=0x7f4acc0079e0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:46.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.045+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4ad0043b50 con 0x7f4adc15d510 2020-11-16T22:30:46.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.249+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4adc15c9c0 con 0x7f4adc15d510 2020-11-16T22:30:46.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.877+0000 7f4abbfff700 1 -- 172.21.15.181:0/13014903 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ad0047170 con 0x7f4adc15d510 2020-11-16T22:30:46.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 msgr2=0x7f4ab4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 0x7f4ab4057520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f4acc003400 tx=0x7f4acc0079e0).stop 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 msgr2=0x7f4adc15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 0x7f4adc15f360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f4ad000c6e0 tx=0x7f4ad0007a20).stop 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 shutdown_connections 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4ab4055060 0x7f4ab4057520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4adc15d8f0 0x7f4adc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4adc0b7c10 0x7f4adc158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 --2- 172.21.15.181:0/13014903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4adc15d510 0x7f4adc15f360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:46.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 >> 172.21.15.181:0/13014903 conn(0x7f4adc0d3680 msgr2=0x7f4adc0b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:46.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 shutdown_connections 2020-11-16T22:30:46.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:46.881+0000 7f4ae24d5700 1 -- 172.21.15.181:0/13014903 wait complete. 2020-11-16T22:30:46.879 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:30:47.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.073+0000 7fd0996d0700 1 Processor -- start 2020-11-16T22:30:47.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.073+0000 7fd0996d0700 1 -- start start 2020-11-16T22:30:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd094149170 0x7fd094066270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd094149620 con 0x7fd094148d90 2020-11-16T22:30:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940c9a80 con 0x7fd094149170 2020-11-16T22:30:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940c9d20 con 0x7fd094163af0 2020-11-16T22:30:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40154/0 (socket says 172.21.15.181:40154) 2020-11-16T22:30:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 -- 172.21.15.181:0/1301407633 learned_addr learned my addr 172.21.15.181:0/1301407633 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd098ecf700 1 --2- 172.21.15.181:0/1301407633 >> v2:172.21.15.175:3300/0 conn(0x7fd094149170 0x7fd094066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 -- 172.21.15.181:0/1301407633 >> v2:172.21.15.175:3300/0 conn(0x7fd094149170 msgr2=0x7fd094066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 --2- 172.21.15.181:0/1301407633 >> v2:172.21.15.175:3300/0 conn(0x7fd094149170 0x7fd094066270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 -- 172.21.15.181:0/1301407633 >> v1:172.21.15.175:6789/0 conn(0x7fd094148d90 legacy=0x7fd0941633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 -- 172.21.15.181:0/1301407633 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940c9fc0 con 0x7fd094163af0 2020-11-16T22:30:47.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0937fe700 1 --2- 172.21.15.181:0/1301407633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd07c00ba20 tx=0x7fd07c006eb0).ready entity=mon.2 client_cookie=ff53a1b899f24284 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd092ffd700 1 -- 172.21.15.181:0/1301407633 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0097d0 con 0x7fd094163af0 2020-11-16T22:30:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd092ffd700 1 -- 172.21.15.181:0/1301407633 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c009930 con 0x7fd094163af0 2020-11-16T22:30:47.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd092ffd700 1 -- 172.21.15.181:0/1301407633 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c009cb0 con 0x7fd094163af0 2020-11-16T22:30:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- 172.21.15.181:0/1301407633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 msgr2=0x7fd0940c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1301407633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd07c00ba20 tx=0x7fd07c006eb0).stop 2020-11-16T22:30:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- 172.21.15.181:0/1301407633 shutdown_connections 2020-11-16T22:30:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1301407633 >> v2:172.21.15.175:3300/0 conn(0x7fd094149170 0x7fd094066270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1301407633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094163af0 0x7fd0940c93c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- 172.21.15.181:0/1301407633 >> 172.21.15.181:0/1301407633 conn(0x7fd0940d4460 msgr2=0x7fd0940d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- 172.21.15.181:0/1301407633 shutdown_connections 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.077+0000 7fd0996d0700 1 -- 172.21.15.181:0/1301407633 wait complete. 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 Processor -- start 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 -- start start 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094149170 0x7fd09415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd094163af0 0x7fd09415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415ea20 con 0x7fd094163af0 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941616b0 con 0x7fd094148d90 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0996d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094064550 con 0x7fd094149170 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094149170 0x7fd09415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60548/0 (socket says 172.21.15.181:60548) 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 -- 172.21.15.181:0/1539067180 learned_addr learned my addr 172.21.15.181:0/1539067180 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd098ecf700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd094163af0 0x7fd09415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 -- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094149170 msgr2=0x7fd09415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094149170 0x7fd09415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 -- 172.21.15.181:0/1539067180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd094163af0 msgr2=0x7fd09415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.081+0000 7fd093fff700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd094163af0 0x7fd09415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd093fff700 1 -- 172.21.15.181:0/1539067180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09415ec50 con 0x7fd094148d90 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd093fff700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fd08400c710 tx=0x7fd084007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084007020 con 0x7fd094148d90 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084007180 con 0x7fd094148d90 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09415eef0 con 0x7fd094148d90 2020-11-16T22:30:47.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840064f0 con 0x7fd094148d90 2020-11-16T22:30:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09415fba0 con 0x7fd094148d90 2020-11-16T22:30:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094044f60 con 0x7fd094148d90 2020-11-16T22:30:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.085+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd084010070 con 0x7fd094148d90 2020-11-16T22:30:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.089+0000 7fd08bfff700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 0x7fd070057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.089+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fd08406d170 con 0x7fd094148d90 2020-11-16T22:30:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.089+0000 7fd0937fe700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 0x7fd070057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.089+0000 7fd0937fe700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 0x7fd070057520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd07c006ef0 tx=0x7fd07c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.093+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd0840432d0 con 0x7fd094148d90 2020-11-16T22:30:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.241+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0941465c0 con 0x7fd094148d90 2020-11-16T22:30:47.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.933+0000 7fd08bfff700 1 -- 172.21.15.181:0/1539067180 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd0840468f0 con 0x7fd094148d90 2020-11-16T22:30:47.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 msgr2=0x7fd070057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 0x7fd070057520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd07c006ef0 tx=0x7fd07c015000).stop 2020-11-16T22:30:47.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 msgr2=0x7fd0941611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:47.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fd08400c710 tx=0x7fd084007c50).stop 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 shutdown_connections 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd070055060 0x7fd070057520 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd094163af0 0x7fd09415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094148d90 0x7fd0941611c0 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0x7fd08400c710 tx=0x7fd084007c50).stop 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 --2- 172.21.15.181:0/1539067180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd094149170 0x7fd09415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 >> 172.21.15.181:0/1539067180 conn(0x7fd0940d4460 msgr2=0x7fd0940d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 shutdown_connections 2020-11-16T22:30:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:47.941+0000 7fd0996d0700 1 -- 172.21.15.181:0/1539067180 wait complete. 2020-11-16T22:30:47.925 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:30:47.938 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:30:48.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.085+0000 7f1faa61f700 1 Processor -- start 2020-11-16T22:30:48.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.085+0000 7f1faa61f700 1 -- start start 2020-11-16T22:30:48.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa4149170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:48.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa4149660 con 0x7f1fa4163af0 2020-11-16T22:30:48.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa40c9a80 con 0x7f1fa4148d90 2020-11-16T22:30:48.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40c9870 con 0x7f1fa40c93c0 2020-11-16T22:30:48.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa4149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:48.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa961d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1fa4148d90 0x7f1fa41633e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46628/0 (socket says 172.21.15.181:46628) 2020-11-16T22:30:48.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa961d700 1 -- 172.21.15.181:0/1251423927 learned_addr learned my addr 172.21.15.181:0/1251423927 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:48.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa9e1e700 1 -- 172.21.15.181:0/1251423927 >> v1:172.21.15.181:6789/0 conn(0x7f1fa4148d90 legacy=0x7f1fa41633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1f8ffff700 1 -- 172.21.15.181:0/1251423927 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878007675 0 0) 0x7f1fa40c9a80 con 0x7f1fa4148d90 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa9e1e700 1 -- 172.21.15.181:0/1251423927 >> v1:172.21.15.175:6789/0 conn(0x7f1fa4163af0 legacy=0x7f1fa4066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa9e1e700 1 -- 172.21.15.181:0/1251423927 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa40c9d20 con 0x7f1fa40c93c0 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1fa9e1e700 1 --2- 172.21.15.181:0/1251423927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa4149170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1f9c009cc0 tx=0x7f1f9c0091d0).ready entity=mon.2 client_cookie=15d4a5b52043dd5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1f8ffff700 1 -- 172.21.15.181:0/1251423927 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00bac0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1f8ffff700 1 -- 172.21.15.181:0/1251423927 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c00bc20 con 0x7f1fa40c93c0 2020-11-16T22:30:48.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1f8ffff700 1 -- 172.21.15.181:0/1251423927 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011400 con 0x7f1fa40c93c0 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- 172.21.15.181:0/1251423927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 msgr2=0x7f1fa4149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 --2- 172.21.15.181:0/1251423927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa4149170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1f9c009cc0 tx=0x7f1f9c0091d0).stop 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- 172.21.15.181:0/1251423927 shutdown_connections 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 --2- 172.21.15.181:0/1251423927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa4149170 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- 172.21.15.181:0/1251423927 >> 172.21.15.181:0/1251423927 conn(0x7f1fa40d4460 msgr2=0x7f1fa40d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- 172.21.15.181:0/1251423927 shutdown_connections 2020-11-16T22:30:48.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.089+0000 7f1faa61f700 1 -- 172.21.15.181:0/1251423927 wait complete. 2020-11-16T22:30:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 Processor -- start 2020-11-16T22:30:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 -- start start 2020-11-16T22:30:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa4148d90 0x7f1fa4061b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fa4163af0 0x7f1fa4062070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40ce1e0 con 0x7f1fa4148d90 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40ce480 con 0x7f1fa4163af0 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1faa61f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41703e0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40172/0 (socket says 172.21.15.181:40172) 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 -- 172.21.15.181:0/15795858 learned_addr learned my addr 172.21.15.181:0/15795858 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa9e1e700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fa4163af0 0x7f1fa4062070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa8e1c700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa4148d90 0x7f1fa4061b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 -- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fa4163af0 msgr2=0x7f1fa4062070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fa4163af0 0x7f1fa4062070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 -- 172.21.15.181:0/15795858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa4148d90 msgr2=0x7f1fa4061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa4148d90 0x7f1fa4061b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 -- 172.21.15.181:0/15795858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4170680 con 0x7f1fa40c93c0 2020-11-16T22:30:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.093+0000 7f1fa961d700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f1fa000c760 tx=0x7f1fa0007a60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00072e0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0007440 con 0x7f1fa40c93c0 2020-11-16T22:30:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa4170920 con 0x7f1fa40c93c0 2020-11-16T22:30:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005840 con 0x7f1fa40c93c0 2020-11-16T22:30:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa40cbcd0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa4044f60 con 0x7f1fa40c93c0 2020-11-16T22:30:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1fa0010070 con 0x7f1fa40c93c0 2020-11-16T22:30:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 0x7f1f78057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f1fa006d3f0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1fa8e1c700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 0x7f1f78057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.097+0000 7f1fa8e1c700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 0x7f1f78057520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1fa4148840 tx=0x7f1f98009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.105+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1fa0043700 con 0x7f1fa40c93c0 2020-11-16T22:30:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.257+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1fa4068dc0 con 0x7f1fa40c93c0 2020-11-16T22:30:48.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.945+0000 7f1f95ffb700 1 -- 172.21.15.181:0/15795858 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1fa0046d20 con 0x7f1fa40c93c0 2020-11-16T22:30:48.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 msgr2=0x7f1f78057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 0x7f1f78057520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1fa4148840 tx=0x7f1f98009040).stop 2020-11-16T22:30:48.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 msgr2=0x7f1fa40cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f1fa000c760 tx=0x7f1fa0007a60).stop 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 shutdown_connections 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1f78055060 0x7f1f78057520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fa4163af0 0x7f1fa4062070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fa40c93c0 0x7f1fa40cb760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 --2- 172.21.15.181:0/15795858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fa4148d90 0x7f1fa4061b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:48.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 >> 172.21.15.181:0/15795858 conn(0x7f1fa40d4460 msgr2=0x7f1fa40d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:48.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 shutdown_connections 2020-11-16T22:30:48.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:48.949+0000 7f1faa61f700 1 -- 172.21.15.181:0/15795858 wait complete. 2020-11-16T22:30:48.932 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:30:48.947 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:30:48.947 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:30:48.947 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:30:48.947 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.948 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:30:48.948 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:30:48.948 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:30:48.948 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.949 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:30:48.949 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:30:48.949 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:30:48.949 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.949 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:30:48.950 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:30:48.950 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:30:48.950 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.950 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:30:48.950 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:30:48.951 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:30:48.951 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.951 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:30:48.951 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:30:48.952 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:30:48.952 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.952 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:30:48.952 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:30:48.952 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:30:48.953 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.953 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:30:48.953 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:30:48.953 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:30:48.953 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.954 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:30:48.954 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:30:48.954 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:30:48.954 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:30:48.964 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:30:48.964 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:30:48.965 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:30:48.965 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:30:48.965 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:30:48.966 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:30:48.966 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:30:48.966 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:30:48.967 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:30:48.967 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:30:48.970 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:30:48.971 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:30:48.971 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:30:48.971 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:30:48.973 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:30:48.973 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:30:48.973 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:30:48.974 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:30:48.974 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:30:48.975 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.006 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:30:49.006 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.008 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:30:49.008 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:30:49.009 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:30:49.009 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.009 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.010 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.016 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:30:49.016 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:30:49.017 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.017 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.018 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:30:49.018 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:30:49.019 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.019 DEBUG:teuthology.parallel:result is None 2020-11-16T22:30:49.019 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:30:49.039 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:30:49.074 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:30:49.084 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:30:49.090 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:30:49.091 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:30:49.099 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:30:49.101 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:30:49.102 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:30:49.103 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:30:49.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.169+0000 7fcd2a975700 1 Processor -- start 2020-11-16T22:30:49.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.169+0000 7fcd2a975700 1 -- start start 2020-11-16T22:30:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd240cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2415b850 con 0x7fcd2415b3d0 2020-11-16T22:30:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240b1f80 con 0x7fcd2415aff0 2020-11-16T22:30:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240c8750 con 0x7fcd24060800 2020-11-16T22:30:49.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd29973700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcd2415aff0 0x7fcd240c7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46688/0 (socket says 172.21.15.181:46688) 2020-11-16T22:30:49.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd29973700 1 -- 172.21.15.181:0/2695548685 learned_addr learned my addr 172.21.15.181:0/2695548685 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:49.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a174700 1 --2- 172.21.15.181:0/2695548685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd240cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:49.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd28971700 1 -- 172.21.15.181:0/2695548685 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635134327 0 0) 0x7fcd240b1f80 con 0x7fcd2415aff0 2020-11-16T22:30:49.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd28971700 1 -- 172.21.15.181:0/2695548685 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0c003240 con 0x7fcd2415aff0 2020-11-16T22:30:49.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a174700 1 -- 172.21.15.181:0/2695548685 >> v1:172.21.15.181:6789/0 conn(0x7fcd2415aff0 legacy=0x7fcd240c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a174700 1 -- 172.21.15.181:0/2695548685 >> v1:172.21.15.175:6789/0 conn(0x7fcd2415b3d0 legacy=0x7fcd240600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a174700 1 -- 172.21.15.181:0/2695548685 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240c7a30 con 0x7fcd24060800 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a174700 1 --2- 172.21.15.181:0/2695548685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd240cd010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fcd1c009cc0 tx=0x7fcd1c0091d0).ready entity=mon.2 client_cookie=b6bfa9dd67563b2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd28971700 1 -- 172.21.15.181:0/2695548685 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00bad0 con 0x7fcd24060800 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd28971700 1 -- 172.21.15.181:0/2695548685 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c00bc30 con 0x7fcd24060800 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd28971700 1 -- 172.21.15.181:0/2695548685 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c011400 con 0x7fcd24060800 2020-11-16T22:30:49.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- 172.21.15.181:0/2695548685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 msgr2=0x7fcd240cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 --2- 172.21.15.181:0/2695548685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd240cd010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fcd1c009cc0 tx=0x7fcd1c0091d0).stop 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- 172.21.15.181:0/2695548685 shutdown_connections 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 --2- 172.21.15.181:0/2695548685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd240cd010 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- 172.21.15.181:0/2695548685 >> 172.21.15.181:0/2695548685 conn(0x7fcd240d22f0 msgr2=0x7fcd240d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- 172.21.15.181:0/2695548685 shutdown_connections 2020-11-16T22:30:49.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- 172.21.15.181:0/2695548685 wait complete. 2020-11-16T22:30:49.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 Processor -- start 2020-11-16T22:30:49.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.173+0000 7fcd2a975700 1 -- start start 2020-11-16T22:30:49.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:49.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2415aff0 0x7fcd2415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:49.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd2415b3d0 0x7fcd2415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2416dcd0 con 0x7fcd2415b3d0 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415e950 con 0x7fcd2415aff0 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2416da30 con 0x7fcd24060800 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40232/0 (socket says 172.21.15.181:40232) 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 -- 172.21.15.181:0/237538517 learned_addr learned my addr 172.21.15.181:0/237538517 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:49.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29172700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2415aff0 0x7fcd2415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a174700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd2415b3d0 0x7fcd2415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 -- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2415aff0 msgr2=0x7fcd2415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2415aff0 0x7fcd2415df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 -- 172.21.15.181:0/237538517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd2415b3d0 msgr2=0x7fcd2415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd2415b3d0 0x7fcd2415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 -- 172.21.15.181:0/237538517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2415d2f0 con 0x7fcd24060800 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd29973700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fcd2000c710 tx=0x7fcd20007aa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:49.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200072e0 con 0x7fcd24060800 2020-11-16T22:30:49.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2415d590 con 0x7fcd24060800 2020-11-16T22:30:49.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd240b23a0 con 0x7fcd24060800 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20007440 con 0x7fcd24060800 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.177+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200057a0 con 0x7fcd24060800 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.181+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcd2001a060 con 0x7fcd24060800 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.181+0000 7fcd11ffb700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 0x7fccfc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.181+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5511+0+0 (secure 0 0 0) 0x7fcd2006d280 con 0x7fcd24060800 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.181+0000 7fcd29172700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 0x7fccfc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:49.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.181+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd24044f60 con 0x7fcd24060800 2020-11-16T22:30:49.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.185+0000 7fcd29172700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 0x7fccfc057520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fcd241555d0 tx=0x7fcd14000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:49.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.185+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcd20018020 con 0x7fcd24060800 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:49.353+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcd2405d550 con 0x7fcd24060800 2020-11-16T22:30:50.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.165+0000 7fcd11ffb700 1 -- 172.21.15.181:0/237538517 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd20043710 con 0x7fcd24060800 2020-11-16T22:30:50.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 msgr2=0x7fccfc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 0x7fccfc057520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fcd241555d0 tx=0x7fcd14000b80).stop 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 msgr2=0x7fcd24145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fcd2000c710 tx=0x7fcd20007aa0).stop 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 shutdown_connections 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccfc055060 0x7fccfc057520 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd2415b3d0 0x7fcd2415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd24060800 0x7fcd24145420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 --2- 172.21.15.181:0/237538517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2415aff0 0x7fcd2415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 >> 172.21.15.181:0/237538517 conn(0x7fcd240d22f0 msgr2=0x7fcd240600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 shutdown_connections 2020-11-16T22:30:50.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.169+0000 7fcd2a975700 1 -- 172.21.15.181:0/237538517 wait complete. 2020-11-16T22:30:50.151 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:30:50.163 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:30:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.301+0000 7f10c3fff700 1 Processor -- start 2020-11-16T22:30:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.301+0000 7f10c3fff700 1 -- start start 2020-11-16T22:30:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 0x7f10c4062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c40c8db0 con 0x7f10c4062740 2020-11-16T22:30:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c4165030 con 0x7f10c4161480 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40c8330 con 0x7f10c4161860 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f10c4161480 0x7f10c4164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46708/0 (socket says 172.21.15.181:46708) 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c2ffd700 1 -- 172.21.15.181:0/1233921238 learned_addr learned my addr 172.21.15.181:0/1233921238 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c27fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 0x7f10c4062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1233921238 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359018103 0 0) 0x7f10c4165030 con 0x7f10c4161480 2020-11-16T22:30:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1233921238 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a8003240 con 0x7f10c4161480 2020-11-16T22:30:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c27fc700 1 -- 172.21.15.181:0/1233921238 >> v1:172.21.15.181:6789/0 conn(0x7f10c4161480 legacy=0x7f10c4164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c27fc700 1 -- 172.21.15.181:0/1233921238 >> v1:172.21.15.175:6789/0 conn(0x7f10c4062740 legacy=0x7f10c40644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c27fc700 1 -- 172.21.15.181:0/1233921238 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40c85d0 con 0x7f10c4161860 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c27fc700 1 --2- 172.21.15.181:0/1233921238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 0x7f10c4062250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f10b4009cc0 tx=0x7f10b40091d0).ready entity=mon.2 client_cookie=4c0880b9545576b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1233921238 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b400bad0 con 0x7f10c4161860 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1233921238 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b400bc30 con 0x7f10c4161860 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1233921238 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4011400 con 0x7f10c4161860 2020-11-16T22:30:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- 172.21.15.181:0/1233921238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 msgr2=0x7f10c4062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 --2- 172.21.15.181:0/1233921238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 0x7f10c4062250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f10b4009cc0 tx=0x7f10b40091d0).stop 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- 172.21.15.181:0/1233921238 shutdown_connections 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 --2- 172.21.15.181:0/1233921238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161860 0x7f10c4062250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- 172.21.15.181:0/1233921238 >> 172.21.15.181:0/1233921238 conn(0x7f10c40d2dc0 msgr2=0x7f10c40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- 172.21.15.181:0/1233921238 shutdown_connections 2020-11-16T22:30:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.305+0000 7f10c3fff700 1 -- 172.21.15.181:0/1233921238 wait complete. 2020-11-16T22:30:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 Processor -- start 2020-11-16T22:30:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- start start 2020-11-16T22:30:50.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c4062740 0x7f10c40630a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 0x7f10c4063590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41608b0 con 0x7f10c4062740 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4174560 con 0x7f10c4161860 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4174800 con 0x7f10c4161480 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 0x7f10c4063590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60644/0 (socket says 172.21.15.181:60644) 2020-11-16T22:30:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c37fe700 1 -- 172.21.15.181:0/2246764089 learned_addr learned my addr 172.21.15.181:0/2246764089 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:50.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c2ffd700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c4062740 0x7f10c40630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 -- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 msgr2=0x7f10c41738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 -- 172.21.15.181:0/2246764089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c4062740 msgr2=0x7f10c40630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c4062740 0x7f10c40630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 -- 172.21.15.181:0/2246764089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40b62f0 con 0x7f10c4161480 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c37fe700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c27fc700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 0x7f10c4063590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f10b4009cc0 tx=0x7f10b4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4011e20 con 0x7f10c4161480 2020-11-16T22:30:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b401c5f0 con 0x7f10c4161480 2020-11-16T22:30:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c40b6590 con 0x7f10c4161480 2020-11-16T22:30:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b401c990 con 0x7f10c4161480 2020-11-16T22:30:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.309+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c4173de0 con 0x7f10c4161480 2020-11-16T22:30:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c4044f60 con 0x7f10c4161480 2020-11-16T22:30:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f10b4023030 con 0x7f10c4161480 2020-11-16T22:30:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10baffd700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 0x7f1094057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10c2ffd700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 0x7f1094057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f10b406eff0 con 0x7f10c4161480 2020-11-16T22:30:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.313+0000 7f10c2ffd700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 0x7f1094057520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f10bc0055a0 tx=0x7f10bc009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.321+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f10b4045580 con 0x7f10c4161480 2020-11-16T22:30:50.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.521+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f10c40671e0 con 0x7f10c4161480 2020-11-16T22:30:50.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10baffd700 1 -- 172.21.15.181:0/2246764089 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5978 (secure 0 0 0) 0x7f10b4015070 con 0x7f10c4161480 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:30:50.506 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.508 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:30:50.509 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.511 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.512 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.513 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.514 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.515 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:30:50.516 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.517 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.519 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:30:50.520 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:30:50.521 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.523 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.524 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:30:50.525 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.526 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.529 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:30:50.530 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.531 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.532 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:30:50.533 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:30:50.534 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:30:50.535 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:30:50.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 msgr2=0x7f1094057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 0x7f1094057520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f10bc0055a0 tx=0x7f10bc009040).stop 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 msgr2=0x7f10c4063590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 0x7f10c4063590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f10b4009cc0 tx=0x7f10b4016000).stop 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 shutdown_connections 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1094055060 0x7f1094057520 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c4161860 0x7f10c41738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c4062740 0x7f10c40630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 --2- 172.21.15.181:0/2246764089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c4161480 0x7f10c4063590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 >> 172.21.15.181:0/2246764089 conn(0x7f10c40d2dc0 msgr2=0x7f10c40d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 shutdown_connections 2020-11-16T22:30:50.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.525+0000 7f10c3fff700 1 -- 172.21.15.181:0/2246764089 wait complete. 2020-11-16T22:30:50.539 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.681+0000 7f858eddb700 1 Processor -- start 2020-11-16T22:30:50.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- start start 2020-11-16T22:30:50.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8588161480 0x7f8588164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85880c8db0 con 0x7f8588164e10 2020-11-16T22:30:50.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880c8330 con 0x7f8588161480 2020-11-16T22:30:50.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85880c85d0 con 0x7f8588161860 2020-11-16T22:30:50.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858ddd9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8588161480 0x7f8588164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40262/0 (socket says 172.21.15.181:40262) 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 -- 172.21.15.181:0/2770276583 learned_addr learned my addr 172.21.15.181:0/2770276583 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 -- 172.21.15.181:0/2770276583 >> v2:172.21.15.175:3300/0 conn(0x7f8588161480 msgr2=0x7f8588164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 --2- 172.21.15.181:0/2770276583 >> v2:172.21.15.175:3300/0 conn(0x7f8588161480 0x7f8588164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 -- 172.21.15.181:0/2770276583 >> v1:172.21.15.175:6789/0 conn(0x7f8588164e10 legacy=0x7f8588062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 -- 172.21.15.181:0/2770276583 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85880c8870 con 0x7f8588161860 2020-11-16T22:30:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858e5da700 1 --2- 172.21.15.181:0/2770276583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f8578006fd0 tx=0x7f8578006eb0).ready entity=mon.2 client_cookie=51df188e540f5f45 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858cdd7700 1 -- 172.21.15.181:0/2770276583 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578009810 con 0x7f8588161860 2020-11-16T22:30:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858cdd7700 1 -- 172.21.15.181:0/2770276583 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578009970 con 0x7f8588161860 2020-11-16T22:30:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858cdd7700 1 -- 172.21.15.181:0/2770276583 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578009cf0 con 0x7f8588161860 2020-11-16T22:30:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- 172.21.15.181:0/2770276583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 msgr2=0x7f8588062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 --2- 172.21.15.181:0/2770276583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f8578006fd0 tx=0x7f8578006eb0).stop 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- 172.21.15.181:0/2770276583 shutdown_connections 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 --2- 172.21.15.181:0/2770276583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588161860 0x7f8588062960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 --2- 172.21.15.181:0/2770276583 >> v2:172.21.15.175:3300/0 conn(0x7f8588161480 0x7f8588164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- 172.21.15.181:0/2770276583 >> 172.21.15.181:0/2770276583 conn(0x7f85880d2dc0 msgr2=0x7f85880ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- 172.21.15.181:0/2770276583 shutdown_connections 2020-11-16T22:30:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.685+0000 7f858eddb700 1 -- 172.21.15.181:0/2770276583 wait complete. 2020-11-16T22:30:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.689+0000 7f858eddb700 1 Processor -- start 2020-11-16T22:30:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.689+0000 7f858eddb700 1 -- start start 2020-11-16T22:30:50.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8588161480 0x7f85880b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588160b90 0x7f85880621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881457e0 con 0x7f8588161480 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588173810 con 0x7f8588161860 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588173ab0 con 0x7f8588160b90 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858e5da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588160b90 0x7f85880621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60658/0 (socket says 172.21.15.181:60658) 2020-11-16T22:30:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 -- 172.21.15.181:0/3769718923 learned_addr learned my addr 172.21.15.181:0/3769718923 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858ddd9700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8588161480 0x7f85880b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 -- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588160b90 msgr2=0x7f85880621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588160b90 0x7f85880621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 -- 172.21.15.181:0/3769718923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8588161480 msgr2=0x7f85880b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8588161480 0x7f85880b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 -- 172.21.15.181:0/3769718923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588173d50 con 0x7f8588161860 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858d5d8700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f857000c710 tx=0x7f8570007bc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570007020 con 0x7f8588161860 2020-11-16T22:30:50.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8570007180 con 0x7f8588161860 2020-11-16T22:30:50.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8588173ff0 con 0x7f8588161860 2020-11-16T22:30:50.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700064f0 con 0x7f8588161860 2020-11-16T22:30:50.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.693+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85881734d0 con 0x7f8588161860 2020-11-16T22:30:50.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588044f60 con 0x7f8588161860 2020-11-16T22:30:50.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8570010070 con 0x7f8588161860 2020-11-16T22:30:50.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f8585ffb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 0x7f8564057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f857006ccf0 con 0x7f8588161860 2020-11-16T22:30:50.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f858ddd9700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 0x7f8564057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.697+0000 7f858ddd9700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 0x7f8564057520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f85800099f0 tx=0x7f85800091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.705+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8570043280 con 0x7f8588161860 2020-11-16T22:30:50.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.857+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8588173660 con 0x7f8588161860 2020-11-16T22:30:50.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.857+0000 7f8585ffb700 1 -- 172.21.15.181:0/3769718923 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f85700235b0 con 0x7f8588161860 2020-11-16T22:30:50.840 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:50.840 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":59,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:50.157138+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 msgr2=0x7f8564057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 0x7f8564057520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f85800099f0 tx=0x7f85800091d0).stop 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 msgr2=0x7f85881606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f857000c710 tx=0x7f8570007bc0).stop 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 shutdown_connections 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8564055060 0x7f8564057520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8588160b90 0x7f85880621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8588161480 0x7f85880b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 --2- 172.21.15.181:0/3769718923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8588161860 0x7f85881606a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 >> 172.21.15.181:0/3769718923 conn(0x7f85880d2dc0 msgr2=0x7f85880600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 shutdown_connections 2020-11-16T22:30:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.861+0000 7f858eddb700 1 -- 172.21.15.181:0/3769718923 wait complete. 2020-11-16T22:30:50.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:30:50.854 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:30:50.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.997+0000 7f646b60f700 1 Processor -- start 2020-11-16T22:30:50.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:50.997+0000 7f646b60f700 1 -- start start 2020-11-16T22:30:50.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64641428e0 0x7f6464162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f64640cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6464161870 con 0x7f6464162f20 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64641633a0 con 0x7f64641428e0 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6464161b10 con 0x7f6464163c60 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f64693ab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64641428e0 0x7f6464162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f64640cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6468baa700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6464162f20 0x7f6464163550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46738/0 (socket says 172.21.15.181:46738) 2020-11-16T22:30:50.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6468baa700 1 -- 172.21.15.181:0/2024298263 learned_addr learned my addr 172.21.15.181:0/2024298263 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:50.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f645bfff700 1 -- 172.21.15.181:0/2024298263 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844135625 0 0) 0x7f6464161870 con 0x7f6464162f20 2020-11-16T22:30:50.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f645bfff700 1 -- 172.21.15.181:0/2024298263 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6448003240 con 0x7f6464162f20 2020-11-16T22:30:50.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 -- 172.21.15.181:0/2024298263 >> v2:172.21.15.181:3300/0 conn(0x7f64641428e0 msgr2=0x7f6464162a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 --2- 172.21.15.181:0/2024298263 >> v2:172.21.15.181:3300/0 conn(0x7f64641428e0 0x7f6464162a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 -- 172.21.15.181:0/2024298263 >> v1:172.21.15.181:6789/0 conn(0x7f6464162f20 legacy=0x7f6464163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 -- 172.21.15.181:0/2024298263 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464161db0 con 0x7f6464163c60 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f6469bac700 1 --2- 172.21.15.181:0/2024298263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f64640cd8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f645c009c90 tx=0x7f645c0091d0).ready entity=mon.2 client_cookie=f08251e5b597e5ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f645bfff700 1 -- 172.21.15.181:0/2024298263 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00ba50 con 0x7f6464163c60 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f645bfff700 1 -- 172.21.15.181:0/2024298263 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c00bbb0 con 0x7f6464163c60 2020-11-16T22:30:50.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f645bfff700 1 -- 172.21.15.181:0/2024298263 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c011400 con 0x7f6464163c60 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- 172.21.15.181:0/2024298263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 msgr2=0x7f64640cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 --2- 172.21.15.181:0/2024298263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f64640cd8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f645c009c90 tx=0x7f645c0091d0).stop 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- 172.21.15.181:0/2024298263 shutdown_connections 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 --2- 172.21.15.181:0/2024298263 >> v2:172.21.15.181:3300/0 conn(0x7f64641428e0 0x7f6464162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 --2- 172.21.15.181:0/2024298263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f64640cd8f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- 172.21.15.181:0/2024298263 >> 172.21.15.181:0/2024298263 conn(0x7f6464060bc0 msgr2=0x7f64640d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:50.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- 172.21.15.181:0/2024298263 shutdown_connections 2020-11-16T22:30:50.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- 172.21.15.181:0/2024298263 wait complete. 2020-11-16T22:30:50.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 Processor -- start 2020-11-16T22:30:50.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.001+0000 7f646b60f700 1 -- start start 2020-11-16T22:30:50.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6464162f20 0x7f646415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646416f5a0 con 0x7f6464162f20 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646416f790 con 0x7f64641428e0 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646416fa30 con 0x7f6464163c60 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f6469bac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60672/0 (socket says 172.21.15.181:60672) 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f6469bac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40284/0 (socket says 172.21.15.181:40284) 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 -- 172.21.15.181:0/4077816503 learned_addr learned my addr 172.21.15.181:0/4077816503 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f6468baa700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6464162f20 0x7f646415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 -- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 msgr2=0x7f646415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 -- 172.21.15.181:0/4077816503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6464162f20 msgr2=0x7f646415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:50.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6464162f20 0x7f646415c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 -- 172.21.15.181:0/4077816503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646416fcd0 con 0x7f64641428e0 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f6469bac700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64693ab700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f646000ebd0 tx=0x7f6460006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000c9b0 con 0x7f64641428e0 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646000cb10 con 0x7f64641428e0 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646416ff70 con 0x7f64641428e0 2020-11-16T22:30:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000ce90 con 0x7f64641428e0 2020-11-16T22:30:50.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64640b7e00 con 0x7f64641428e0 2020-11-16T22:30:50.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.005+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640b8050 con 0x7f64641428e0 2020-11-16T22:30:50.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.009+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6460012600 con 0x7f64641428e0 2020-11-16T22:30:50.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.009+0000 7f64597fa700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 0x7f643c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:50.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.009+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f646006df30 con 0x7f64641428e0 2020-11-16T22:30:50.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.009+0000 7f6468baa700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 0x7f643c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:50.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.009+0000 7f6468baa700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 0x7f643c057520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f64540055a0 tx=0x7f6454009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:50.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.013+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6460044350 con 0x7f64641428e0 2020-11-16T22:30:51.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.173+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6464069eb0 con 0x7f64641428e0 2020-11-16T22:30:51.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.173+0000 7f64597fa700 1 -- 172.21.15.181:0/4077816503 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v59) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6460047970 con 0x7f64641428e0 2020-11-16T22:30:51.156 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:30:51.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 msgr2=0x7f643c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 0x7f643c057520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f64540055a0 tx=0x7f6454009040).stop 2020-11-16T22:30:51.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 msgr2=0x7f64640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f646000ebd0 tx=0x7f6460006df0).stop 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 shutdown_connections 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f643c055060 0x7f643c057520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6464163c60 0x7f646415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64641428e0 0x7f64640b7910 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 --2- 172.21.15.181:0/4077816503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6464162f20 0x7f646415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 >> 172.21.15.181:0/4077816503 conn(0x7f6464060bc0 msgr2=0x7f646415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 shutdown_connections 2020-11-16T22:30:51.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.177+0000 7f646b60f700 1 -- 172.21.15.181:0/4077816503 wait complete. 2020-11-16T22:30:51.171 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:30:51.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 Processor -- start 2020-11-16T22:30:51.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 -- start start 2020-11-16T22:30:51.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f90880816e0 0x7f9088145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9088061b50 con 0x7f9088145a30 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90880618b0 con 0x7f90880816e0 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908ef31700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880cb930 con 0x7f90881492c0 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40294/0 (socket says 172.21.15.181:40294) 2020-11-16T22:30:51.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 -- 172.21.15.181:0/563273377 learned_addr learned my addr 172.21.15.181:0/563273377 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908df2f700 1 --1- 172.21.15.181:0/563273377 >> v1:172.21.15.181:6789/0 conn(0x7f9088145a30 0x7f9088080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46752/0 (socket says 172.21.15.181:46752) 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908d72e700 1 --2- 172.21.15.181:0/563273377 >> v2:172.21.15.175:3300/0 conn(0x7f90880816e0 0x7f9088145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 -- 172.21.15.181:0/563273377 >> v2:172.21.15.175:3300/0 conn(0x7f90880816e0 msgr2=0x7f9088145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 --2- 172.21.15.181:0/563273377 >> v2:172.21.15.175:3300/0 conn(0x7f90880816e0 0x7f9088145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 -- 172.21.15.181:0/563273377 >> v1:172.21.15.181:6789/0 conn(0x7f9088145a30 legacy=0x7f9088080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 -- 172.21.15.181:0/563273377 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880cbbd0 con 0x7f90881492c0 2020-11-16T22:30:51.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908e730700 1 --2- 172.21.15.181:0/563273377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f9080009d50 tx=0x7f9080009290).ready entity=mon.2 client_cookie=abbb479079c45e9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.317+0000 7f908cf2d700 1 -- 172.21.15.181:0/563273377 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000bbc0 con 0x7f90881492c0 2020-11-16T22:30:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908cf2d700 1 -- 172.21.15.181:0/563273377 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908000bd20 con 0x7f90881492c0 2020-11-16T22:30:51.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908cf2d700 1 -- 172.21.15.181:0/563273377 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080011610 con 0x7f90881492c0 2020-11-16T22:30:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- 172.21.15.181:0/563273377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 msgr2=0x7f90880cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 --2- 172.21.15.181:0/563273377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f9080009d50 tx=0x7f9080009290).stop 2020-11-16T22:30:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- 172.21.15.181:0/563273377 shutdown_connections 2020-11-16T22:30:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 --2- 172.21.15.181:0/563273377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f90880cc5b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 --2- 172.21.15.181:0/563273377 >> v2:172.21.15.175:3300/0 conn(0x7f90880816e0 0x7f9088145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- 172.21.15.181:0/563273377 >> 172.21.15.181:0/563273377 conn(0x7f90880d68c0 msgr2=0x7f90880d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- 172.21.15.181:0/563273377 shutdown_connections 2020-11-16T22:30:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- 172.21.15.181:0/563273377 wait complete. 2020-11-16T22:30:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 Processor -- start 2020-11-16T22:30:51.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.321+0000 7f908ef31700 1 -- start start 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90880816e0 0x7f90880bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9088145a30 0x7f9088172690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880d1b60 con 0x7f90880816e0 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088173070 con 0x7f9088145a30 2020-11-16T22:30:51.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9088173300 con 0x7f90881492c0 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908d72e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9088145a30 0x7f9088172690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40300/0 (socket says 172.21.15.181:40300) 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908df2f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90880816e0 0x7f90880bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 -- 172.21.15.181:0/3723613840 learned_addr learned my addr 172.21.15.181:0/3723613840 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 -- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9088145a30 msgr2=0x7f9088172690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9088145a30 0x7f9088172690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 -- 172.21.15.181:0/3723613840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90880816e0 msgr2=0x7f90880bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90880816e0 0x7f90880bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 -- 172.21.15.181:0/3723613840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908816f2c0 con 0x7f90881492c0 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908e730700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9080007fd0 tx=0x7f9080016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080011e70 con 0x7f90881492c0 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908000be90 con 0x7f90881492c0 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908816f560 con 0x7f90881492c0 2020-11-16T22:30:51.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080010c90 con 0x7f90881492c0 2020-11-16T22:30:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908816faa0 con 0x7f90881492c0 2020-11-16T22:30:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908816f110 con 0x7f90881492c0 2020-11-16T22:30:51.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9080017070 con 0x7f90881492c0 2020-11-16T22:30:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 0x7f9060057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f908006f730 con 0x7f90881492c0 2020-11-16T22:30:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.325+0000 7f908df2f700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 0x7f9060057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.329+0000 7f908df2f700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 0x7f9060057520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f907c000bc0 tx=0x7f907c007b40).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.337+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9080045cc0 con 0x7f90881492c0 2020-11-16T22:30:51.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.485+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f908816fc30 con 0x7f90881492c0 2020-11-16T22:30:51.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f9079ffb700 1 -- 172.21.15.181:0/3723613840 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9080015070 con 0x7f90881492c0 2020-11-16T22:30:51.469 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:30:51.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 msgr2=0x7f9060057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 0x7f9060057520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f907c000bc0 tx=0x7f907c007b40).stop 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 msgr2=0x7f9088172b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9080007fd0 tx=0x7f9080016000).stop 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 shutdown_connections 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9060055060 0x7f9060057520 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90881492c0 0x7f9088172b80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90880816e0 0x7f90880bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 --2- 172.21.15.181:0/3723613840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9088145a30 0x7f9088172690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 >> 172.21.15.181:0/3723613840 conn(0x7f90880d68c0 msgr2=0x7f9088060ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 shutdown_connections 2020-11-16T22:30:51.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.489+0000 7f908ef31700 1 -- 172.21.15.181:0/3723613840 wait complete. 2020-11-16T22:30:51.484 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:30:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 Processor -- start 2020-11-16T22:30:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 -- start start 2020-11-16T22:30:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d9013e870 0x7f3d9013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9015d180 con 0x7f3d9015fdc0 2020-11-16T22:30:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d90160730 con 0x7f3d9013e870 2020-11-16T22:30:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d95128700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9013f140 con 0x7f3d90061810 2020-11-16T22:30:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d94927700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d94927700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40308/0 (socket says 172.21.15.181:40308) 2020-11-16T22:30:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d94927700 1 -- 172.21.15.181:0/3838961398 learned_addr learned my addr 172.21.15.181:0/3838961398 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.621+0000 7f3d8ffff700 1 --2- 172.21.15.181:0/3838961398 >> v2:172.21.15.175:3300/0 conn(0x7f3d9013e870 0x7f3d9013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d94927700 1 -- 172.21.15.181:0/3838961398 >> v2:172.21.15.175:3300/0 conn(0x7f3d9013e870 msgr2=0x7f3d9013ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d94927700 1 --2- 172.21.15.181:0/3838961398 >> v2:172.21.15.175:3300/0 conn(0x7f3d9013e870 0x7f3d9013ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d94927700 1 -- 172.21.15.181:0/3838961398 >> v1:172.21.15.175:6789/0 conn(0x7f3d9015fdc0 legacy=0x7f3d90061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d94927700 1 -- 172.21.15.181:0/3838961398 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9015d810 con 0x7f3d90061810 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d94927700 1 --2- 172.21.15.181:0/3838961398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3d80006fd0 tx=0x7f3d80006eb0).ready entity=mon.2 client_cookie=b58c4b8ec53acbc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d8effd700 1 -- 172.21.15.181:0/3838961398 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80009810 con 0x7f3d90061810 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d8effd700 1 -- 172.21.15.181:0/3838961398 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d80009970 con 0x7f3d90061810 2020-11-16T22:30:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d8effd700 1 -- 172.21.15.181:0/3838961398 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80009cf0 con 0x7f3d90061810 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 -- 172.21.15.181:0/3838961398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 msgr2=0x7f3d90160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 --2- 172.21.15.181:0/3838961398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3d80006fd0 tx=0x7f3d80006eb0).stop 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 -- 172.21.15.181:0/3838961398 shutdown_connections 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 --2- 172.21.15.181:0/3838961398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d90061810 0x7f3d90160240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 --2- 172.21.15.181:0/3838961398 >> v2:172.21.15.175:3300/0 conn(0x7f3d9013e870 0x7f3d9013ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 -- 172.21.15.181:0/3838961398 >> 172.21.15.181:0/3838961398 conn(0x7f3d900cf630 msgr2=0x7f3d900cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 -- 172.21.15.181:0/3838961398 shutdown_connections 2020-11-16T22:30:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.625+0000 7f3d95128700 1 -- 172.21.15.181:0/3838961398 wait complete. 2020-11-16T22:30:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 Processor -- start 2020-11-16T22:30:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 -- start start 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d90061810 0x7f3d90060730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 0x7f3d90060c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900c8840 con 0x7f3d90061810 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900c8ae0 con 0x7f3d9015fdc0 2020-11-16T22:30:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d95128700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900c8d80 con 0x7f3d9013e870 2020-11-16T22:30:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 0x7f3d90060c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60704/0 (socket says 172.21.15.181:60704) 2020-11-16T22:30:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d8ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d90061810 0x7f3d90060730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 -- 172.21.15.181:0/1639108174 learned_addr learned my addr 172.21.15.181:0/1639108174 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 -- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 msgr2=0x7f3d90060c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 0x7f3d90060c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 -- 172.21.15.181:0/1639108174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d90061810 msgr2=0x7f3d90060730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d90061810 0x7f3d90060730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d94927700 1 -- 172.21.15.181:0/1639108174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900c9020 con 0x7f3d9015fdc0 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.629+0000 7f3d8f7fe700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 0x7f3d90060c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d94927700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3d80002fc0 tx=0x7f3d80017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80020070 con 0x7f3d9015fdc0 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800195f0 con 0x7f3d9015fdc0 2020-11-16T22:30:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80009810 con 0x7f3d9015fdc0 2020-11-16T22:30:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900c92c0 con 0x7f3d9015fdc0 2020-11-16T22:30:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9016baf0 con 0x7f3d9015fdc0 2020-11-16T22:30:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d900ce620 con 0x7f3d9015fdc0 2020-11-16T22:30:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3d8002c070 con 0x7f3d9015fdc0 2020-11-16T22:30:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 0x7f3d60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3d800715a0 con 0x7f3d9015fdc0 2020-11-16T22:30:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d8ffff700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 0x7f3d60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.633+0000 7f3d8ffff700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 0x7f3d60057520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3d880099f0 tx=0x7f3d880091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.645+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3d900ce620 con 0x7f3d9015fdc0 2020-11-16T22:30:51.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3d900ce620 con 0x7f3d9015fdc0 2020-11-16T22:30:51.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d87fff700 1 -- 172.21.15.181:0/1639108174 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3d900ce620 con 0x7f3d9015fdc0 2020-11-16T22:30:51.781 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:30:51.782 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:30:51.783 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:30:51.783 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:30:51.783 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 msgr2=0x7f3d60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 0x7f3d60057520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3d880099f0 tx=0x7f3d880091d0).stop 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 msgr2=0x7f3d9016b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3d80002fc0 tx=0x7f3d80017000).stop 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 shutdown_connections 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d60055060 0x7f3d60057520 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9015fdc0 0x7f3d9016b600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d90061810 0x7f3d90060730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 --2- 172.21.15.181:0/1639108174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9013e870 0x7f3d90060c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 >> 172.21.15.181:0/1639108174 conn(0x7f3d900cf630 msgr2=0x7f3d900b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 shutdown_connections 2020-11-16T22:30:51.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.801+0000 7f3d95128700 1 -- 172.21.15.181:0/1639108174 wait complete. 2020-11-16T22:30:51.796 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:30:51.796 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:30:51.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.945+0000 7ff5950d0700 1 Processor -- start 2020-11-16T22:30:51.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.945+0000 7ff5950d0700 1 -- start start 2020-11-16T22:30:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 0x7ff590145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5900814c0 0x7ff5900cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff590061b50 con 0x7ff5900814c0 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5900618b0 con 0x7ff590145a30 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900cb930 con 0x7ff5901492c0 2020-11-16T22:30:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 0x7ff590145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40320/0 (socket says 172.21.15.181:40320) 2020-11-16T22:30:51.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 0x7ff590145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60708/0 (socket says 172.21.15.181:60708) 2020-11-16T22:30:51.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 -- 172.21.15.181:0/3518420988 learned_addr learned my addr 172.21.15.181:0/3518420988 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5948cf700 1 --2- 172.21.15.181:0/3518420988 >> v2:172.21.15.175:3300/0 conn(0x7ff5900814c0 0x7ff5900cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 -- 172.21.15.181:0/3518420988 >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 msgr2=0x7ff590145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:51.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/3518420988 >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 0x7ff590145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 -- 172.21.15.181:0/3518420988 >> v2:172.21.15.175:3300/0 conn(0x7ff5900814c0 msgr2=0x7ff5900cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/3518420988 >> v2:172.21.15.175:3300/0 conn(0x7ff5900814c0 0x7ff5900cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 -- 172.21.15.181:0/3518420988 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900cbbd0 con 0x7ff5901492c0 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/3518420988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7ff5900d43d0 tx=0x7ff580006eb0).ready entity=mon.2 client_cookie=8f4df16e4f18c7d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58effd700 1 -- 172.21.15.181:0/3518420988 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580009720 con 0x7ff5901492c0 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58effd700 1 -- 172.21.15.181:0/3518420988 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580009880 con 0x7ff5901492c0 2020-11-16T22:30:51.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff58effd700 1 -- 172.21.15.181:0/3518420988 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580009c00 con 0x7ff5901492c0 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- 172.21.15.181:0/3518420988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 msgr2=0x7ff590080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- 172.21.15.181:0/3518420988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7ff5900d43d0 tx=0x7ff580006eb0).stop 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- 172.21.15.181:0/3518420988 shutdown_connections 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- 172.21.15.181:0/3518420988 >> v2:172.21.15.175:3300/0 conn(0x7ff5900814c0 0x7ff5900cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- 172.21.15.181:0/3518420988 >> v2:172.21.15.181:3300/0 conn(0x7ff590145a30 0x7ff590145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 --2- 172.21.15.181:0/3518420988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901492c0 0x7ff590080fd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- 172.21.15.181:0/3518420988 >> 172.21.15.181:0/3518420988 conn(0x7ff5900d68c0 msgr2=0x7ff5900d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:51.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- 172.21.15.181:0/3518420988 shutdown_connections 2020-11-16T22:30:51.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.949+0000 7ff5950d0700 1 -- 172.21.15.181:0/3518420988 wait complete. 2020-11-16T22:30:51.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 Processor -- start 2020-11-16T22:30:51.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 -- start start 2020-11-16T22:30:51.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5900814c0 0x7ff5900d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 0x7ff590060620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 0x7ff590060b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901614b0 con 0x7ff5900814c0 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900ce950 con 0x7ff5901704c0 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5950d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900cebf0 con 0x7ff590145a30 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5948cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 0x7ff590060b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 0x7ff590060620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5948cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 0x7ff590060b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60714/0 (socket says 172.21.15.181:60714) 2020-11-16T22:30:51.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff5948cf700 1 -- 172.21.15.181:0/4270681781 learned_addr learned my addr 172.21.15.181:0/4270681781 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58ffff700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5900814c0 0x7ff5900d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 -- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 msgr2=0x7ff590060b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 0x7ff590060b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 -- 172.21.15.181:0/4270681781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5900814c0 msgr2=0x7ff5900d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5900814c0 0x7ff5900d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 -- 172.21.15.181:0/4270681781 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5901616e0 con 0x7ff590145a30 2020-11-16T22:30:51.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff58f7fe700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 0x7ff590060620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7ff580012000 tx=0x7ff580015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.953+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580011af0 con 0x7ff590145a30 2020-11-16T22:30:51.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580011c50 con 0x7ff590145a30 2020-11-16T22:30:51.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff590161980 con 0x7ff590145a30 2020-11-16T22:30:51.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58001e460 con 0x7ff590145a30 2020-11-16T22:30:51.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5900cee90 con 0x7ff590145a30 2020-11-16T22:30:51.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff580025030 con 0x7ff590145a30 2020-11-16T22:30:51.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff590161b10 con 0x7ff590145a30 2020-11-16T22:30:51.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff56ffff700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 0x7ff568057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:51.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff58006f0c0 con 0x7ff590145a30 2020-11-16T22:30:51.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.957+0000 7ff58ffff700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 0x7ff568057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:51.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.961+0000 7ff58ffff700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 0x7ff568057520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff584006fd0 tx=0x7ff584006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:51.965+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff580045540 con 0x7ff590145a30 2020-11-16T22:30:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.117+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff590044f60 con 0x7ff590145a30 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.237+0000 7ff56ffff700 1 -- 172.21.15.181:0/4270681781 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff5800230a0 con 0x7ff590145a30 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 msgr2=0x7ff568057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 0x7ff568057520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff584006fd0 tx=0x7ff584006df0).stop 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 msgr2=0x7ff590060620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 0x7ff590060620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7ff580012000 tx=0x7ff580015000).stop 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 shutdown_connections 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff568055060 0x7ff568057520 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5901704c0 0x7ff590060b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5900814c0 0x7ff5900d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 --2- 172.21.15.181:0/4270681781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff590145a30 0x7ff590060620 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 >> 172.21.15.181:0/4270681781 conn(0x7ff5900d68c0 msgr2=0x7ff5900ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 shutdown_connections 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.241+0000 7ff5950d0700 1 -- 172.21.15.181:0/4270681781 wait complete. 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:30:52.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.381+0000 7fd7ec388700 1 Processor -- start 2020-11-16T22:30:52.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.381+0000 7fd7ec388700 1 -- start start 2020-11-16T22:30:52.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd7e40612a0 0x7fd7e4061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e4061b70 con 0x7fd7e40cf0a0 2020-11-16T22:30:52.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e415cda0 con 0x7fd7e40612a0 2020-11-16T22:30:52.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415d040 con 0x7fd7e40ce750 2020-11-16T22:30:52.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ea925700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd7e40cf0a0 0x7fd7e415fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46796/0 (socket says 172.21.15.181:46796) 2020-11-16T22:30:52.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ea925700 1 -- 172.21.15.181:0/1371658394 learned_addr learned my addr 172.21.15.181:0/1371658394 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:52.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ea124700 1 --2- 172.21.15.181:0/1371658394 >> v2:172.21.15.175:3300/0 conn(0x7fd7e40612a0 0x7fd7e4061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9122700 1 -- 172.21.15.181:0/1371658394 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143592061 0 0) 0x7fd7e4061b70 con 0x7fd7e40cf0a0 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9122700 1 -- 172.21.15.181:0/1371658394 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc003240 con 0x7fd7e40cf0a0 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 -- 172.21.15.181:0/1371658394 >> v2:172.21.15.175:3300/0 conn(0x7fd7e40612a0 msgr2=0x7fd7e4061680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 --2- 172.21.15.181:0/1371658394 >> v2:172.21.15.175:3300/0 conn(0x7fd7e40612a0 0x7fd7e4061680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 -- 172.21.15.181:0/1371658394 >> v1:172.21.15.181:6789/0 conn(0x7fd7e40cf0a0 legacy=0x7fd7e415fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 -- 172.21.15.181:0/1371658394 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e415d2e0 con 0x7fd7e40ce750 2020-11-16T22:30:52.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9923700 1 --2- 172.21.15.181:0/1371658394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e40cebb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd7d400b9a0 tx=0x7fd7d4006df0).ready entity=mon.2 client_cookie=5f229bca35d57b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9122700 1 -- 172.21.15.181:0/1371658394 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4009680 con 0x7fd7e40ce750 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9122700 1 -- 172.21.15.181:0/1371658394 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d40097e0 con 0x7fd7e40ce750 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7e9122700 1 -- 172.21.15.181:0/1371658394 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4009b60 con 0x7fd7e40ce750 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- 172.21.15.181:0/1371658394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 msgr2=0x7fd7e40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 --2- 172.21.15.181:0/1371658394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e40cebb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd7d400b9a0 tx=0x7fd7d4006df0).stop 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- 172.21.15.181:0/1371658394 shutdown_connections 2020-11-16T22:30:52.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 --2- 172.21.15.181:0/1371658394 >> v2:172.21.15.175:3300/0 conn(0x7fd7e40612a0 0x7fd7e4061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 --2- 172.21.15.181:0/1371658394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e40cebb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- 172.21.15.181:0/1371658394 >> 172.21.15.181:0/1371658394 conn(0x7fd7e40cf890 msgr2=0x7fd7e40cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:52.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- 172.21.15.181:0/1371658394 shutdown_connections 2020-11-16T22:30:52.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.385+0000 7fd7ec388700 1 -- 172.21.15.181:0/1371658394 wait complete. 2020-11-16T22:30:52.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 Processor -- start 2020-11-16T22:30:52.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 -- start start 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7e40cf0a0 0x7fd7e41727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415b040 con 0x7fd7e40cf0a0 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e4173060 con 0x7fd7e40612a0 2020-11-16T22:30:52.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ec388700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e4173300 con 0x7fd7e40ce750 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7e9923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60726/0 (socket says 172.21.15.181:60726) 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7e9923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40338/0 (socket says 172.21.15.181:40338) 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 -- 172.21.15.181:0/2590423420 learned_addr learned my addr 172.21.15.181:0/2590423420 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea925700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7e40cf0a0 0x7fd7e41727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 -- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 msgr2=0x7fd7e415e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 -- 172.21.15.181:0/2590423420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7e40cf0a0 msgr2=0x7fd7e41727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7e40cf0a0 0x7fd7e41727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7ea124700 1 -- 172.21.15.181:0/2590423420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e41735a0 con 0x7fd7e40612a0 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.389+0000 7fd7e9923700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:52.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7ea124700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd7e000edd0 tx=0x7fd7e0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:52.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000aae0 con 0x7fd7e40612a0 2020-11-16T22:30:52.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000ac40 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000adb0 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e416f4e0 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4170190 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e415ed30 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd7e0020070 con 0x7fd7e40612a0 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 0x7fd7bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fd7e006d940 con 0x7fd7e40612a0 2020-11-16T22:30:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7e9923700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 0x7fd7bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.393+0000 7fd7e9923700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 0x7fd7bc057520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fd7d400c010 tx=0x7fd7d4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:52.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.401+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd7e0043d70 con 0x7fd7e40612a0 2020-11-16T22:30:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:52.553+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd7e416f980 con 0x7fd7e40612a0 2020-11-16T22:30:53.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.145+0000 7fd7d27fc700 1 -- 172.21.15.181:0/2590423420 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v61) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd7e416f980 con 0x7fd7e40612a0 2020-11-16T22:30:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 msgr2=0x7fd7bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 0x7fd7bc057520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fd7d400c010 tx=0x7fd7d4015000).stop 2020-11-16T22:30:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 msgr2=0x7fd7e416fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd7e000edd0 tx=0x7fd7e0008040).stop 2020-11-16T22:30:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 shutdown_connections 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd7bc055060 0x7fd7bc057520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7e40cf0a0 0x7fd7e41727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7e40612a0 0x7fd7e416fc20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 --2- 172.21.15.181:0/2590423420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7e40ce750 0x7fd7e415e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 >> 172.21.15.181:0/2590423420 conn(0x7fd7e40cf890 msgr2=0x7fd7e4063390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 shutdown_connections 2020-11-16T22:30:53.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.153+0000 7fd7ec388700 1 -- 172.21.15.181:0/2590423420 wait complete. 2020-11-16T22:30:53.136 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:30:53.151 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:30:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 Processor -- start 2020-11-16T22:30:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 -- start start 2020-11-16T22:30:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b84162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 0x7f9b84163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b840b14a0 con 0x7f9b84163ac0 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840b1740 con 0x7f9b840b2ad0 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8c56d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840b19e0 con 0x7f9b84162aa0 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8a309700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b84162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b8ab0a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 0x7f9b84163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b89b08700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9b84163ac0 0x7f9b840b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46810/0 (socket says 172.21.15.181:46810) 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.293+0000 7f9b89b08700 1 -- 172.21.15.181:0/1971518964 learned_addr learned my addr 172.21.15.181:0/1971518964 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8a309700 1 -- 172.21.15.181:0/1971518964 >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 msgr2=0x7f9b84163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8a309700 1 --2- 172.21.15.181:0/1971518964 >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 0x7f9b84163f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8a309700 1 -- 172.21.15.181:0/1971518964 >> v1:172.21.15.181:6789/0 conn(0x7f9b84163ac0 legacy=0x7f9b840b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b89307700 1 -- 172.21.15.181:0/1971518964 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983075413 0 0) 0x7f9b840b14a0 con 0x7f9b84163ac0 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8a309700 1 -- 172.21.15.181:0/1971518964 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840b1c80 con 0x7f9b84162aa0 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8ab0a700 1 --2- 172.21.15.181:0/1971518964 >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 0x7f9b84163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8a309700 1 --2- 172.21.15.181:0/1971518964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b84162e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f9b7c009c90 tx=0x7f9b7c0091d0).ready entity=mon.2 client_cookie=e0d638da9d07f76c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b89307700 1 -- 172.21.15.181:0/1971518964 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00ba70 con 0x7f9b84162aa0 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b89307700 1 -- 172.21.15.181:0/1971518964 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c00bbd0 con 0x7f9b84162aa0 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b89307700 1 -- 172.21.15.181:0/1971518964 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c011400 con 0x7f9b84162aa0 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- 172.21.15.181:0/1971518964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 msgr2=0x7f9b84162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/1971518964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b84162e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f9b7c009c90 tx=0x7f9b7c0091d0).stop 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- 172.21.15.181:0/1971518964 shutdown_connections 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/1971518964 >> v2:172.21.15.181:3300/0 conn(0x7f9b840b2ad0 0x7f9b84163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/1971518964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b84162e80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- 172.21.15.181:0/1971518964 >> 172.21.15.181:0/1971518964 conn(0x7f9b840d35d0 msgr2=0x7f9b840d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- 172.21.15.181:0/1971518964 shutdown_connections 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- 172.21.15.181:0/1971518964 wait complete. 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 Processor -- start 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.297+0000 7f9b8c56d700 1 -- start start 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b841487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 0x7f9b84148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8416f320 con 0x7f9b84163ac0 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8416f5c0 con 0x7f9b840b2ad0 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8416f860 con 0x7f9b84162aa0 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60742/0 (socket says 172.21.15.181:60742) 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 -- 172.21.15.181:0/3163686189 learned_addr learned my addr 172.21.15.181:0/3163686189 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b89b08700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b841487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8ab0a700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 0x7f9b84148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 -- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 msgr2=0x7f9b841487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b841487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 -- 172.21.15.181:0/3163686189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 msgr2=0x7f9b84148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 0x7f9b84148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 -- 172.21.15.181:0/3163686189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8416fb00 con 0x7f9b840b2ad0 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8ab0a700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 0x7f9b84148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b89b08700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b841487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8a309700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f9b7c00b770 tx=0x7f9b7c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c01c5f0 con 0x7f9b840b2ad0 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c01c750 con 0x7f9b840b2ad0 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8416fda0 con 0x7f9b840b2ad0 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c01cad0 con 0x7f9b840b2ad0 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84165680 con 0x7f9b840b2ad0 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84069eb0 con 0x7f9b840b2ad0 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.301+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9b7c017070 con 0x7f9b840b2ad0 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.305+0000 7f9b827fc700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 0x7f9b5c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.305+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f9b7c06fd60 con 0x7f9b840b2ad0 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.305+0000 7f9b89b08700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 0x7f9b5c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.305+0000 7f9b89b08700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 0x7f9b5c057520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9b841628d0 tx=0x7f9b78007a00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.313+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9b7c046030 con 0x7f9b840b2ad0 2020-11-16T22:30:53.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.509+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f9b84141150 con 0x7f9b840b2ad0 2020-11-16T22:30:53.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.525+0000 7f9b827fc700 1 -- 172.21.15.181:0/3163686189 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v61) v1 ==== 188+0+0 (secure 0 0 0) 0x7f9b7c015070 con 0x7f9b840b2ad0 2020-11-16T22:30:53.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 msgr2=0x7f9b5c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 0x7f9b5c057520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9b841628d0 tx=0x7f9b78007a00).stop 2020-11-16T22:30:53.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 msgr2=0x7f9b841607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f9b7c00b770 tx=0x7f9b7c016000).stop 2020-11-16T22:30:53.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 shutdown_connections 2020-11-16T22:30:53.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b5c055060 0x7f9b5c057520 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b84163ac0 0x7f9b84148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b840b2ad0 0x7f9b841607a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 --2- 172.21.15.181:0/3163686189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b84162aa0 0x7f9b841487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 >> 172.21.15.181:0/3163686189 conn(0x7f9b840d35d0 msgr2=0x7f9b84064c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 shutdown_connections 2020-11-16T22:30:53.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.533+0000 7f9b8c56d700 1 -- 172.21.15.181:0/3163686189 wait complete. 2020-11-16T22:30:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 9 and data pool 10 2020-11-16T22:30:53.532 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.661+0000 7fd5a3906700 1 Processor -- start 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.661+0000 7fd5a3906700 1 -- start start 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 0x7fd59c061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd59c065fa0 0x7fd59c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c062b70 con 0x7fd59c066420 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c061b70 con 0x7fd59c0612a0 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c15db10 con 0x7fd59c065fa0 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd59c065fa0 0x7fd59c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a1ea3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd59c066420 0x7fd59c15fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46836/0 (socket says 172.21.15.181:46836) 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a1ea3700 1 -- 172.21.15.181:0/1749874658 learned_addr learned my addr 172.21.15.181:0/1749874658 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a16a2700 1 --2- 172.21.15.181:0/1749874658 >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 0x7fd59c061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 -- 172.21.15.181:0/1749874658 >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 msgr2=0x7fd59c061680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 --2- 172.21.15.181:0/1749874658 >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 0x7fd59c061680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 -- 172.21.15.181:0/1749874658 >> v1:172.21.15.181:6789/0 conn(0x7fd59c066420 legacy=0x7fd59c15fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 -- 172.21.15.181:0/1749874658 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0cd790 con 0x7fd59c065fa0 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd58bfff700 1 -- 172.21.15.181:0/1749874658 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846708902 0 0) 0x7fd59c062b70 con 0x7fd59c066420 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a16a2700 1 --2- 172.21.15.181:0/1749874658 >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 0x7fd59c061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a0ea1700 1 --2- 172.21.15.181:0/1749874658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c065fa0 0x7fd59c15f570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fd59c0b5a80 tx=0x7fd58c006df0).ready entity=mon.1 client_cookie=d8a06d49b64fe416 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd58bfff700 1 -- 172.21.15.181:0/1749874658 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c009840 con 0x7fd59c065fa0 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd58bfff700 1 -- 172.21.15.181:0/1749874658 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c0099a0 con 0x7fd59c065fa0 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd58bfff700 1 -- 172.21.15.181:0/1749874658 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c009d20 con 0x7fd59c065fa0 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- 172.21.15.181:0/1749874658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c065fa0 msgr2=0x7fd59c15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 --2- 172.21.15.181:0/1749874658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c065fa0 0x7fd59c15f570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fd59c0b5a80 tx=0x7fd58c006df0).stop 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- 172.21.15.181:0/1749874658 shutdown_connections 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 --2- 172.21.15.181:0/1749874658 >> v2:172.21.15.175:3300/0 conn(0x7fd59c0612a0 0x7fd59c061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 --2- 172.21.15.181:0/1749874658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c065fa0 0x7fd59c15f570 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- 172.21.15.181:0/1749874658 >> 172.21.15.181:0/1749874658 conn(0x7fd59c0cf890 msgr2=0x7fd59c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- 172.21.15.181:0/1749874658 shutdown_connections 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.665+0000 7fd5a3906700 1 -- 172.21.15.181:0/1749874658 wait complete. 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 Processor -- start 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 -- start start 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 0x7fd59c16f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c065fa0 0x7fd59c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c16fb80 con 0x7fd59c065fa0 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c170ae0 con 0x7fd59c066420 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c16fdc0 con 0x7fd59c0612a0 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a16a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 0x7fd59c16f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60770/0 (socket says 172.21.15.181:60770) 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a16a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 0x7fd59c16f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40376/0 (socket says 172.21.15.181:40376) 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 -- 172.21.15.181:0/2679636406 learned_addr learned my addr 172.21.15.181:0/2679636406 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a0ea1700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c065fa0 0x7fd59c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 -- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 msgr2=0x7fd59c16f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 0x7fd59c16f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 -- 172.21.15.181:0/2679636406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c065fa0 msgr2=0x7fd59c1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c065fa0 0x7fd59c1577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 -- 172.21.15.181:0/2679636406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c170060 con 0x7fd59c066420 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a1ea3700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd59400c730 tx=0x7fd594007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940072e0 con 0x7fd59c066420 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594007440 con 0x7fd59c066420 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.669+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c170300 con 0x7fd59c066420 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594005820 con 0x7fd59c066420 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c0b2ef0 con 0x7fd59c066420 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c069eb0 con 0x7fd59c066420 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd594010070 con 0x7fd59c066420 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5897fa700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 0x7fd574057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd59406d4c0 con 0x7fd59c066420 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5a16a2700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 0x7fd574057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.673+0000 7fd5a16a2700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 0x7fd574057520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fd598009920 tx=0x7fd598008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.677+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd5940436c0 con 0x7fd59c066420 2020-11-16T22:30:53.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.825+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd59c05d4e0 con 0x7fd59c066420 2020-11-16T22:30:53.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.825+0000 7fd5897fa700 1 -- 172.21.15.181:0/2679636406 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16889 (secure 0 0 0) 0x7fd594018090 con 0x7fd59c066420 2020-11-16T22:30:53.809 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:53.809 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":63,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:53.516055+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:53.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 msgr2=0x7fd574057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 0x7fd574057520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fd598009920 tx=0x7fd598008040).stop 2020-11-16T22:30:53.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 msgr2=0x7fd59c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd59400c730 tx=0x7fd594007960).stop 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 shutdown_connections 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd574055060 0x7fd574057520 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd59c066420 0x7fd59c157ce0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd59c0612a0 0x7fd59c16f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 --2- 172.21.15.181:0/2679636406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd59c065fa0 0x7fd59c1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 >> 172.21.15.181:0/2679636406 conn(0x7fd59c0cf890 msgr2=0x7fd59c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 shutdown_connections 2020-11-16T22:30:53.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.829+0000 7fd5a3906700 1 -- 172.21.15.181:0/2679636406 wait complete. 2020-11-16T22:30:53.823 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.965+0000 7fe125dcd700 1 Processor -- start 2020-11-16T22:30:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.965+0000 7fe125dcd700 1 -- start start 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.965+0000 7fe125dcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe1200cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.965+0000 7fe125dcd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe12015d460 0x7fe120066310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe120065150 con 0x7fe1200600f0 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1200653f0 con 0x7fe12015d460 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe120065690 con 0x7fe12015d080 2020-11-16T22:30:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe12015d460 0x7fe120066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe12015d460 0x7fe120066310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60774/0 (socket says 172.21.15.181:60774) 2020-11-16T22:30:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 -- 172.21.15.181:0/3768268038 learned_addr learned my addr 172.21.15.181:0/3768268038 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe124dcb700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe1200cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 -- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 msgr2=0x7fe1200cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe1200cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 -- 172.21.15.181:0/3768268038 >> v1:172.21.15.175:6789/0 conn(0x7fe1200600f0 legacy=0x7fe1200605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 -- 172.21.15.181:0/3768268038 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120065930 con 0x7fe12015d460 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11ffff700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe120066310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fe114006fd0 tx=0x7fe114006eb0).ready entity=mon.1 client_cookie=5e508bd8fad9df11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11f7fe700 1 -- 172.21.15.181:0/3768268038 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009810 con 0x7fe12015d460 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11f7fe700 1 -- 172.21.15.181:0/3768268038 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114009970 con 0x7fe12015d460 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe11f7fe700 1 -- 172.21.15.181:0/3768268038 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009cf0 con 0x7fe12015d460 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 msgr2=0x7fe120066310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe120066310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fe114006fd0 tx=0x7fe114006eb0).stop 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- 172.21.15.181:0/3768268038 shutdown_connections 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe1200cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- 172.21.15.181:0/3768268038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe120066310 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- 172.21.15.181:0/3768268038 >> 172.21.15.181:0/3768268038 conn(0x7fe1200cfe90 msgr2=0x7fe1200d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- 172.21.15.181:0/3768268038 shutdown_connections 2020-11-16T22:30:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- 172.21.15.181:0/3768268038 wait complete. 2020-11-16T22:30:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 Processor -- start 2020-11-16T22:30:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- start start 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200600f0 0x7fe1200b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe12014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200b4fc0 con 0x7fe1200600f0 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.969+0000 7fe125dcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200cb7c0 con 0x7fe12015d460 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe125dcd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200cba60 con 0x7fe12015d080 2020-11-16T22:30:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe1255cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe12014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40392/0 (socket says 172.21.15.181:40392) 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 -- 172.21.15.181:0/1987266626 learned_addr learned my addr 172.21.15.181:0/1987266626 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe124dcb700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200600f0 0x7fe1200b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 -- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 msgr2=0x7fe12014f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe12014f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 -- 172.21.15.181:0/1987266626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200600f0 msgr2=0x7fe1200b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200600f0 0x7fe1200b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 -- 172.21.15.181:0/1987266626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200cbd00 con 0x7fe12015d080 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11ffff700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe120067b50 tx=0x7fe114015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114011c80 con 0x7fe12015d080 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114011de0 con 0x7fe12015d080 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1200cbfa0 con 0x7fe12015d080 2020-11-16T22:30:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11401e6b0 con 0x7fe12015d080 2020-11-16T22:30:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1200cc4e0 con 0x7fe12015d080 2020-11-16T22:30:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe114025030 con 0x7fe12015d080 2020-11-16T22:30:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.973+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120044f60 con 0x7fe12015d080 2020-11-16T22:30:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.977+0000 7fe11cff9700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 0x7fe0f8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.977+0000 7fe124dcb700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 0x7fe0f8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.977+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe11406f4e0 con 0x7fe12015d080 2020-11-16T22:30:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.977+0000 7fe124dcb700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 0x7fe0f8057520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fe110006fd0 tx=0x7fe110006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:53.981+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe114045740 con 0x7fe12015d080 2020-11-16T22:30:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.137+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe12005c4a0 con 0x7fe12015d080 2020-11-16T22:30:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe11cff9700 1 -- 172.21.15.181:0/1987266626 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16889 (secure 0 0 0) 0x7fe114048d60 con 0x7fe12015d080 2020-11-16T22:30:54.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:54.122 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":63,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:53.516055+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 msgr2=0x7fe0f8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 0x7fe0f8057520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fe110006fd0 tx=0x7fe110006df0).stop 2020-11-16T22:30:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 msgr2=0x7fe12014ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe120067b50 tx=0x7fe114015000).stop 2020-11-16T22:30:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 shutdown_connections 2020-11-16T22:30:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0f8055060 0x7fe0f8057520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe12015d460 0x7fe12014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1200600f0 0x7fe1200b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 --2- 172.21.15.181:0/1987266626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12015d080 0x7fe12014ede0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 >> 172.21.15.181:0/1987266626 conn(0x7fe1200cfe90 msgr2=0x7fe1200d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:54.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 shutdown_connections 2020-11-16T22:30:54.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.141+0000 7fe125dcd700 1 -- 172.21.15.181:0/1987266626 wait complete. 2020-11-16T22:30:54.136 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:30:54.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.281+0000 7f474ffff700 1 Processor -- start 2020-11-16T22:30:54.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.281+0000 7f474ffff700 1 -- start start 2020-11-16T22:30:54.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4750145c60 0x7f4750080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47500814c0 0x7f47500606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500d1c70 con 0x7f4750145880 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500d1f10 con 0x7f4750145c60 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500d21b0 con 0x7f47500814c0 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47500814c0 0x7f47500606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47500814c0 0x7f47500606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60792/0 (socket says 172.21.15.181:60792) 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 -- 172.21.15.181:0/1941208552 learned_addr learned my addr 172.21.15.181:0/1941208552 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474effd700 1 --1- 172.21.15.181:0/1941208552 >> v1:172.21.15.181:6789/0 conn(0x7f4750145880 0x7f47500d1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46860/0 (socket says 172.21.15.181:46860) 2020-11-16T22:30:54.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474e7fc700 1 --2- 172.21.15.181:0/1941208552 >> v2:172.21.15.175:3300/0 conn(0x7f4750145c60 0x7f4750080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 -- 172.21.15.181:0/1941208552 >> v2:172.21.15.175:3300/0 conn(0x7f4750145c60 msgr2=0x7f4750080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 --2- 172.21.15.181:0/1941208552 >> v2:172.21.15.175:3300/0 conn(0x7f4750145c60 0x7f4750080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 -- 172.21.15.181:0/1941208552 >> v1:172.21.15.181:6789/0 conn(0x7f4750145880 legacy=0x7f47500d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474dffb700 1 -- 172.21.15.181:0/1941208552 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462819509 0 0) 0x7f47500d1c70 con 0x7f4750145880 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 -- 172.21.15.181:0/1941208552 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500d2450 con 0x7f47500814c0 2020-11-16T22:30:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474f7fe700 1 --2- 172.21.15.181:0/1941208552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47500814c0 0x7f47500606b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f4750164290 tx=0x7f4748009290).ready entity=mon.1 client_cookie=43e38dff1171d4f2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474dffb700 1 -- 172.21.15.181:0/1941208552 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800bbc0 con 0x7f47500814c0 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474dffb700 1 -- 172.21.15.181:0/1941208552 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474800bd20 con 0x7f47500814c0 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474dffb700 1 -- 172.21.15.181:0/1941208552 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011610 con 0x7f47500814c0 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- 172.21.15.181:0/1941208552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47500814c0 msgr2=0x7f47500606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 --2- 172.21.15.181:0/1941208552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47500814c0 0x7f47500606b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f4750164290 tx=0x7f4748009290).stop 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- 172.21.15.181:0/1941208552 shutdown_connections 2020-11-16T22:30:54.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 --2- 172.21.15.181:0/1941208552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47500814c0 0x7f47500606b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 --2- 172.21.15.181:0/1941208552 >> v2:172.21.15.175:3300/0 conn(0x7f4750145c60 0x7f4750080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- 172.21.15.181:0/1941208552 >> 172.21.15.181:0/1941208552 conn(0x7f47500d6770 msgr2=0x7f47500d6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:54.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- 172.21.15.181:0/1941208552 shutdown_connections 2020-11-16T22:30:54.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.285+0000 7f474ffff700 1 -- 172.21.15.181:0/1941208552 wait complete. 2020-11-16T22:30:54.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 Processor -- start 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- start start 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 0x7f475015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750145c60 0x7f475016f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750144870 con 0x7f4750145c60 2020-11-16T22:30:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475016fd70 con 0x7f4750145880 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4750170010 con 0x7f47500814c0 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 0x7f475015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40404/0 (socket says 172.21.15.181:40404) 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 0x7f475015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60796/0 (socket says 172.21.15.181:60796) 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 -- 172.21.15.181:0/2346075759 learned_addr learned my addr 172.21.15.181:0/2346075759 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474f7fe700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750145c60 0x7f475016f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 -- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 msgr2=0x7f475015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 0x7f475015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 -- 172.21.15.181:0/2346075759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750145c60 msgr2=0x7f475016f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750145c60 0x7f475016f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 -- 172.21.15.181:0/2346075759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750170890 con 0x7f47500814c0 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474effd700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f4738001e70 tx=0x7f4738007ae0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47380072e0 con 0x7f47500814c0 2020-11-16T22:30:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738005530 con 0x7f47500814c0 2020-11-16T22:30:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47380058b0 con 0x7f47500814c0 2020-11-16T22:30:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750170b30 con 0x7f47500814c0 2020-11-16T22:30:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.289+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475016f780 con 0x7f47500814c0 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500b9300 con 0x7f47500814c0 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4738010070 con 0x7f47500814c0 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f4746ffd700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 0x7f472c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f473806d430 con 0x7f47500814c0 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f474e7fc700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 0x7f472c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.293+0000 7f474e7fc700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 0x7f472c057550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f4750164210 tx=0x7f4740006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.297+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f47500b9300 con 0x7f47500814c0 2020-11-16T22:30:54.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.497+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f47500b9300 con 0x7f47500814c0 2020-11-16T22:30:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f4746ffd700 1 -- 172.21.15.181:0/2346075759 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f47500b9300 con 0x7f47500814c0 2020-11-16T22:30:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 msgr2=0x7f472c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 0x7f472c057550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f4750164210 tx=0x7f4740006df0).stop 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 msgr2=0x7f475015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f4738001e70 tx=0x7f4738007ae0).stop 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 shutdown_connections 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f472c055090 0x7f472c057550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4750145c60 0x7f475016f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500814c0 0x7f475015ee40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 --2- 172.21.15.181:0/2346075759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750145880 0x7f475015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 >> 172.21.15.181:0/2346075759 conn(0x7f47500d6770 msgr2=0x7f47500ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 shutdown_connections 2020-11-16T22:30:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.525+0000 7f474ffff700 1 -- 172.21.15.181:0/2346075759 wait complete. 2020-11-16T22:30:54.524 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:54.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 Processor -- start 2020-11-16T22:30:54.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- start start 2020-11-16T22:30:54.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 0x7f33e4145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33e40d2a30 0x7f33e41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e4081f20 con 0x7f33e4160450 2020-11-16T22:30:54.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e4060470 con 0x7f33e40d2a30 2020-11-16T22:30:54.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4060710 con 0x7f33e4145770 2020-11-16T22:30:54.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33e40d2a30 0x7f33e41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 0x7f33e4145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33dbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f33e4160450 0x7f33e40d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46876/0 (socket says 172.21.15.181:46876) 2020-11-16T22:30:54.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33e40d2a30 0x7f33e41608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60802/0 (socket says 172.21.15.181:60802) 2020-11-16T22:30:54.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33dbfff700 1 -- 172.21.15.181:0/3447997977 learned_addr learned my addr 172.21.15.181:0/3447997977 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:54.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 -- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 msgr2=0x7f33e4145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:54.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 0x7f33e4145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 -- 172.21.15.181:0/3447997977 >> v1:172.21.15.181:6789/0 conn(0x7f33e4160450 legacy=0x7f33e40d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 -- 172.21.15.181:0/3447997977 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40609b0 con 0x7f33e40d2a30 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e2d9d700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 0x7f33e4145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e359e700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40d2a30 0x7f33e41608b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f33dc009c90 tx=0x7f33dc0091d0).ready entity=mon.1 client_cookie=2cbd5a09d3d266a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e259c700 1 -- 172.21.15.181:0/3447997977 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00ba70 con 0x7f33e40d2a30 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e259c700 1 -- 172.21.15.181:0/3447997977 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc00bbd0 con 0x7f33e40d2a30 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e259c700 1 -- 172.21.15.181:0/3447997977 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc011400 con 0x7f33e40d2a30 2020-11-16T22:30:54.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40d2a30 msgr2=0x7f33e41608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40d2a30 0x7f33e41608b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f33dc009c90 tx=0x7f33dc0091d0).stop 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- 172.21.15.181:0/3447997977 shutdown_connections 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40d2a30 0x7f33e41608b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 --2- 172.21.15.181:0/3447997977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e4145770 0x7f33e4145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- 172.21.15.181:0/3447997977 >> 172.21.15.181:0/3447997977 conn(0x7f33e40d6760 msgr2=0x7f33e40d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- 172.21.15.181:0/3447997977 shutdown_connections 2020-11-16T22:30:54.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.661+0000 7f33e887e700 1 -- 172.21.15.181:0/3447997977 wait complete. 2020-11-16T22:30:54.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 Processor -- start 2020-11-16T22:30:54.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- start start 2020-11-16T22:30:54.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4145770 0x7f33e413e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 0x7f33e416f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41727e0 con 0x7f33e4160450 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4172a80 con 0x7f33e4145770 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4172d20 con 0x7f33e40d2a30 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40416/0 (socket says 172.21.15.181:40416) 2020-11-16T22:30:54.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 -- 172.21.15.181:0/154815963 learned_addr learned my addr 172.21.15.181:0/154815963 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33dbfff700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4145770 0x7f33e413e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e359e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 0x7f33e416f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 -- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4145770 msgr2=0x7f33e413e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4145770 0x7f33e413e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 -- 172.21.15.181:0/154815963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 msgr2=0x7f33e416f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 0x7f33e416f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:54.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 -- 172.21.15.181:0/154815963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4172fc0 con 0x7f33e40d2a30 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e359e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 0x7f33e416f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e2d9d700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f33d400ef50 tx=0x7f33d4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400b5c0 con 0x7f33e40d2a30 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d400cb30 con 0x7f33e40d2a30 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4173260 con 0x7f33e40d2a30 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40123f0 con 0x7f33e40d2a30 2020-11-16T22:30:54.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.665+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e416ef50 con 0x7f33e40d2a30 2020-11-16T22:30:54.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e416f1a0 con 0x7f33e40d2a30 2020-11-16T22:30:54.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f33d4012760 con 0x7f33e40d2a30 2020-11-16T22:30:54.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33daffd700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 0x7f33c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:54.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33dbfff700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 0x7f33c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:54.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f33d406e420 con 0x7f33e40d2a30 2020-11-16T22:30:54.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.669+0000 7f33dbfff700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 0x7f33c0057520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f33e40d2060 tx=0x7f33cc007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:54.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.677+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f33e416f1a0 con 0x7f33e40d2a30 2020-11-16T22:30:54.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.873+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e4044f60 con 0x7f33e40d2a30 2020-11-16T22:30:54.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.873+0000 7f33daffd700 1 -- 172.21.15.181:0/154815963 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f33d4044680 con 0x7f33e40d2a30 2020-11-16T22:30:54.856 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:54.856 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6313,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:30:54.519644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:30:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 msgr2=0x7f33c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 0x7f33c0057520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f33e40d2060 tx=0x7f33cc007b80).stop 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 msgr2=0x7f33e40d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f33d400ef50 tx=0x7f33d4006df0).stop 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 shutdown_connections 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f33c0055060 0x7f33c0057520 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33e4160450 0x7f33e416f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40d2a30 0x7f33e40d3b70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 --2- 172.21.15.181:0/154815963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4145770 0x7f33e413e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 >> 172.21.15.181:0/154815963 conn(0x7f33e40d6760 msgr2=0x7f33e40d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 shutdown_connections 2020-11-16T22:30:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:54.877+0000 7f33e887e700 1 -- 172.21.15.181:0/154815963 wait complete. 2020-11-16T22:30:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 62 2020-11-16T22:30:55.263 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:55.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.413+0000 7f85c143e700 1 Processor -- start 2020-11-16T22:30:55.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.413+0000 7f85c143e700 1 -- start start 2020-11-16T22:30:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.413+0000 7f85c143e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.413+0000 7f85c143e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85bc160530 0x7f85bc061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.413+0000 7f85c143e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc062700 con 0x7f85bc063620 2020-11-16T22:30:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0629a0 con 0x7f85bc160530 2020-11-16T22:30:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc062c40 con 0x7f85bc142660 2020-11-16T22:30:55.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:55.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40424/0 (socket says 172.21.15.181:40424) 2020-11-16T22:30:55.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 -- 172.21.15.181:0/1901438317 learned_addr learned my addr 172.21.15.181:0/1901438317 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:55.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bb7fe700 1 --1- 172.21.15.181:0/1901438317 >> v1:172.21.15.181:6789/0 conn(0x7f85bc063620 0x7f85bc063ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46890/0 (socket says 172.21.15.181:46890) 2020-11-16T22:30:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c0c3d700 1 --2- 172.21.15.181:0/1901438317 >> v2:172.21.15.175:3300/0 conn(0x7f85bc160530 0x7f85bc061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85baffd700 1 -- 172.21.15.181:0/1901438317 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486144490 0 0) 0x7f85bc062700 con 0x7f85bc063620 2020-11-16T22:30:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 -- 172.21.15.181:0/1901438317 >> v2:172.21.15.175:3300/0 conn(0x7f85bc160530 msgr2=0x7f85bc061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 --2- 172.21.15.181:0/1901438317 >> v2:172.21.15.175:3300/0 conn(0x7f85bc160530 0x7f85bc061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 -- 172.21.15.181:0/1901438317 >> v1:172.21.15.181:6789/0 conn(0x7f85bc063620 legacy=0x7f85bc063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 -- 172.21.15.181:0/1901438317 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc062ee0 con 0x7f85bc142660 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85bbfff700 1 --2- 172.21.15.181:0/1901438317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f85ac006fd0 tx=0x7f85ac006eb0).ready entity=mon.2 client_cookie=ec480274a61617ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85baffd700 1 -- 172.21.15.181:0/1901438317 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0098a0 con 0x7f85bc142660 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85baffd700 1 -- 172.21.15.181:0/1901438317 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac009a00 con 0x7f85bc142660 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85baffd700 1 -- 172.21.15.181:0/1901438317 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac009d80 con 0x7f85bc142660 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- 172.21.15.181:0/1901438317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 msgr2=0x7f85bc142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 --2- 172.21.15.181:0/1901438317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f85ac006fd0 tx=0x7f85ac006eb0).stop 2020-11-16T22:30:55.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- 172.21.15.181:0/1901438317 shutdown_connections 2020-11-16T22:30:55.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 --2- 172.21.15.181:0/1901438317 >> v2:172.21.15.175:3300/0 conn(0x7f85bc160530 0x7f85bc061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 --2- 172.21.15.181:0/1901438317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc142a40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- 172.21.15.181:0/1901438317 >> 172.21.15.181:0/1901438317 conn(0x7f85bc0d2dc0 msgr2=0x7f85bc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- 172.21.15.181:0/1901438317 shutdown_connections 2020-11-16T22:30:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- 172.21.15.181:0/1901438317 wait complete. 2020-11-16T22:30:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 Processor -- start 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.417+0000 7f85c143e700 1 -- start start 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 0x7f85bc0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 0x7f85bc0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b5940 con 0x7f85bc160530 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b5be0 con 0x7f85bc063620 2020-11-16T22:30:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b5e80 con 0x7f85bc142660 2020-11-16T22:30:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40430/0 (socket says 172.21.15.181:40430) 2020-11-16T22:30:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 -- 172.21.15.181:0/4144776817 learned_addr learned my addr 172.21.15.181:0/4144776817 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bbfff700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 0x7f85bc0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c0c3d700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 0x7f85bc0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 -- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 msgr2=0x7f85bc0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 0x7f85bc0b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 -- 172.21.15.181:0/4144776817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 msgr2=0x7f85bc0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 0x7f85bc0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 -- 172.21.15.181:0/4144776817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0b6120 con 0x7f85bc142660 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c0c3d700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 0x7f85bc0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bbfff700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 0x7f85bc0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85bb7fe700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f85b400cae0 tx=0x7f85b4007db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4005ac0 con 0x7f85bc142660 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4005c20 con 0x7f85bc142660 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0b69d0 con 0x7f85bc142660 2020-11-16T22:30:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4007110 con 0x7f85bc142660 2020-11-16T22:30:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc15ccb0 con 0x7f85bc142660 2020-11-16T22:30:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc0b5770 con 0x7f85bc142660 2020-11-16T22:30:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.421+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f85b4010030 con 0x7f85bc142660 2020-11-16T22:30:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.425+0000 7f85b3fff700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.425+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85b406dd30 con 0x7f85bc142660 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.425+0000 7f85bbfff700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.425+0000 7f85bbfff700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f85bc0b4fc0 tx=0x7f85ac015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.429+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f85b4043f90 con 0x7f85bc142660 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.577+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85bc0b62b0 con 0x7f85bc142660 2020-11-16T22:30:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85b3fff700 1 -- 172.21.15.181:0/4144776817 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f85b40475b0 con 0x7f85bc142660 2020-11-16T22:30:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 msgr2=0x7f8590057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f85bc0b4fc0 tx=0x7f85ac015000).stop 2020-11-16T22:30:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 msgr2=0x7f85bc15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f85b400cae0 tx=0x7f85b4007db0).stop 2020-11-16T22:30:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 shutdown_connections 2020-11-16T22:30:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8590055060 0x7f8590057520 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85bc160530 0x7f85bc0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc063620 0x7f85bc0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 --2- 172.21.15.181:0/4144776817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc142660 0x7f85bc15c7c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 >> 172.21.15.181:0/4144776817 conn(0x7f85bc0d2dc0 msgr2=0x7f85bc063ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 shutdown_connections 2020-11-16T22:30:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:55.581+0000 7f85c143e700 1 -- 172.21.15.181:0/4144776817 wait complete. 2020-11-16T22:30:56.423 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:56.423 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:54.520377+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:56.626 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:30:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.793+0000 7f1379c72700 1 Processor -- start 2020-11-16T22:30:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.793+0000 7f1379c72700 1 -- start start 2020-11-16T22:30:56.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13740ce7b0 0x7f13740c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13740c7b30 con 0x7f137415e830 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740c7d60 con 0x7f13740ce7b0 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740c8000 con 0x7f13740ceb90 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40440/0 (socket says 172.21.15.181:40440) 2020-11-16T22:30:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 -- 172.21.15.181:0/74152103 learned_addr learned my addr 172.21.15.181:0/74152103 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:56.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1378c70700 1 --2- 172.21.15.181:0/74152103 >> v2:172.21.15.175:3300/0 conn(0x7f13740ce7b0 0x7f13740c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 -- 172.21.15.181:0/74152103 >> v2:172.21.15.175:3300/0 conn(0x7f13740ce7b0 msgr2=0x7f13740c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:56.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 --2- 172.21.15.181:0/74152103 >> v2:172.21.15.175:3300/0 conn(0x7f13740ce7b0 0x7f13740c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 -- 172.21.15.181:0/74152103 >> v1:172.21.15.175:6789/0 conn(0x7f137415e830 legacy=0x7f13740600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 -- 172.21.15.181:0/74152103 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740c82a0 con 0x7f13740ceb90 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1373fff700 1 --2- 172.21.15.181:0/74152103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f135c006fd0 tx=0x7f135c006eb0).ready entity=mon.2 client_cookie=6fc6032222d908b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f13737fe700 1 -- 172.21.15.181:0/74152103 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c009810 con 0x7f13740ceb90 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f13737fe700 1 -- 172.21.15.181:0/74152103 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c009970 con 0x7f13740ceb90 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f13737fe700 1 -- 172.21.15.181:0/74152103 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c009cf0 con 0x7f13740ceb90 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- 172.21.15.181:0/74152103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 msgr2=0x7f137415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 --2- 172.21.15.181:0/74152103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f135c006fd0 tx=0x7f135c006eb0).stop 2020-11-16T22:30:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- 172.21.15.181:0/74152103 shutdown_connections 2020-11-16T22:30:56.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 --2- 172.21.15.181:0/74152103 >> v2:172.21.15.175:3300/0 conn(0x7f13740ce7b0 0x7f13740c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 --2- 172.21.15.181:0/74152103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ceb90 0x7f137415e340 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- 172.21.15.181:0/74152103 >> 172.21.15.181:0/74152103 conn(0x7f13740d3670 msgr2=0x7f13740cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:56.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- 172.21.15.181:0/74152103 shutdown_connections 2020-11-16T22:30:56.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.797+0000 7f1379c72700 1 -- 172.21.15.181:0/74152103 wait complete. 2020-11-16T22:30:56.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 Processor -- start 2020-11-16T22:30:56.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- start start 2020-11-16T22:30:56.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ce7b0 0x7f13740630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137415e830 0x7f13740622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137415d800 con 0x7f137415e830 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374060510 con 0x7f13740ceb90 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740607b0 con 0x7f13740ce7b0 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1378c70700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ce7b0 0x7f13740630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60836/0 (socket says 172.21.15.181:60836) 2020-11-16T22:30:56.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 -- 172.21.15.181:0/3727903305 learned_addr learned my addr 172.21.15.181:0/3727903305 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:56.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379471700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137415e830 0x7f13740622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 -- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ce7b0 msgr2=0x7f13740630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ce7b0 0x7f13740630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 -- 172.21.15.181:0/3727903305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137415e830 msgr2=0x7f13740622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137415e830 0x7f13740622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 -- 172.21.15.181:0/3727903305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374060a50 con 0x7f13740ceb90 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1373fff700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f135c009e10 tx=0x7f135c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c011d50 con 0x7f13740ceb90 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c01e460 con 0x7f13740ceb90 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c01e7e0 con 0x7f13740ceb90 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1374060cf0 con 0x7f13740ceb90 2020-11-16T22:30:56.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.801+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1374061230 con 0x7f13740ceb90 2020-11-16T22:30:56.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374069eb0 con 0x7f13740ceb90 2020-11-16T22:30:56.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f135c025030 con 0x7f13740ceb90 2020-11-16T22:30:56.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1370ff9700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 0x7f1350057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:56.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f135c06f3f0 con 0x7f13740ceb90 2020-11-16T22:30:56.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1378c70700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 0x7f1350057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:56.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.805+0000 7f1378c70700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 0x7f1350057520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f13640099f0 tx=0x7f13640091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:56.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:56.813+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f135c045700 con 0x7f13740ceb90 2020-11-16T22:30:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.013+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f13740d3a50 con 0x7f13740ceb90 2020-11-16T22:30:57.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1370ff9700 1 -- 172.21.15.181:0/3727903305 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f135c023070 con 0x7f13740ceb90 2020-11-16T22:30:57.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 msgr2=0x7f1350057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 0x7f1350057520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f13640099f0 tx=0x7f13640091d0).stop 2020-11-16T22:30:57.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 msgr2=0x7f1374061df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f135c009e10 tx=0x7f135c015000).stop 2020-11-16T22:30:57.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 shutdown_connections 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1350055060 0x7f1350057520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137415e830 0x7f13740622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13740ce7b0 0x7f13740630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 --2- 172.21.15.181:0/3727903305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740ceb90 0x7f1374061df0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 >> 172.21.15.181:0/3727903305 conn(0x7f13740d3670 msgr2=0x7f1374142340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 shutdown_connections 2020-11-16T22:30:57.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.025+0000 7f1379c72700 1 -- 172.21.15.181:0/3727903305 wait complete. 2020-11-16T22:30:57.010 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:30:57.027 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:30:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 Processor -- start 2020-11-16T22:30:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 -- start start 2020-11-16T22:30:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0161860 0x7f30b0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30b0062740 0x7f30b00644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b00c8db0 con 0x7f30b0161480 2020-11-16T22:30:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b0165030 con 0x7f30b0062740 2020-11-16T22:30:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b5fcd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00c8330 con 0x7f30b0161860 2020-11-16T22:30:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b4fcb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f30b0161480 0x7f30b0164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46916/0 (socket says 172.21.15.181:46916) 2020-11-16T22:30:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b4fcb700 1 -- 172.21.15.181:0/2789505143 learned_addr learned my addr 172.21.15.181:0/2789505143 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b57cc700 1 --2- 172.21.15.181:0/2789505143 >> v2:172.21.15.181:3300/0 conn(0x7f30b0062740 0x7f30b00644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30affff700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0161860 0x7f30b0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535190268 0 0) 0x7f30b00c8db0 con 0x7f30b0161480 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f309c003240 con 0x7f30b0161480 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b57cc700 1 -- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0161860 msgr2=0x7f30b0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b57cc700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0161860 0x7f30b0062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1812088184 0 0) 0x7f309c003240 con 0x7f30b0161480 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b57cc700 1 -- 172.21.15.181:0/2789505143 >> v1:172.21.15.181:6789/0 conn(0x7f30b0161480 legacy=0x7f30b0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.169+0000 7f30b57cc700 1 -- 172.21.15.181:0/2789505143 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00c85d0 con 0x7f30b0062740 2020-11-16T22:30:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b57cc700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0062740 0x7f30b00644e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f30a8009c90 tx=0x7f30a80091d0).ready entity=mon.1 client_cookie=a90d6669b94a5197 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0001500 con 0x7f30b0062740 2020-11-16T22:30:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a800b6b0 con 0x7f30b0062740 2020-11-16T22:30:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30af7fe700 1 -- 172.21.15.181:0/2789505143 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a800ba30 con 0x7f30b0062740 2020-11-16T22:30:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 -- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0062740 msgr2=0x7f30b00644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0062740 0x7f30b00644e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f30a8009c90 tx=0x7f30a80091d0).stop 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 -- 172.21.15.181:0/2789505143 shutdown_connections 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0062740 0x7f30b00644e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/2789505143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0161860 0x7f30b0062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 -- 172.21.15.181:0/2789505143 >> 172.21.15.181:0/2789505143 conn(0x7f30b00d2dc0 msgr2=0x7f30b00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 -- 172.21.15.181:0/2789505143 shutdown_connections 2020-11-16T22:30:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.173+0000 7f30b5fcd700 1 -- 172.21.15.181:0/2789505143 wait complete. 2020-11-16T22:30:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 Processor -- start 2020-11-16T22:30:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- start start 2020-11-16T22:30:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 0x7f30b0167840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0161860 0x7f30b00d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01749f0 con 0x7f30b0161860 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00d0cb0 con 0x7f30b0161480 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00d0f50 con 0x7f30b0062740 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b4fcb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 0x7f30b0167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b4fcb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 0x7f30b0167840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40458/0 (socket says 172.21.15.181:40458) 2020-11-16T22:30:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60850/0 (socket says 172.21.15.181:60850) 2020-11-16T22:30:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b4fcb700 1 -- 172.21.15.181:0/111626169 learned_addr learned my addr 172.21.15.181:0/111626169 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b57cc700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0161860 0x7f30b00d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 -- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 msgr2=0x7f30b0167840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 0x7f30b0167840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 -- 172.21.15.181:0/111626169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0161860 msgr2=0x7f30b00d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0161860 0x7f30b00d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 -- 172.21.15.181:0/111626169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00d11f0 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30affff700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f30a400ebd0 tx=0x7f30a4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400b2c0 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a400c8f0 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b0174c20 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400ccb0 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.177+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b00d0690 con 0x7f30b0161480 2020-11-16T22:30:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0044f60 con 0x7f30b0161480 2020-11-16T22:30:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f30a401c460 con 0x7f30b0161480 2020-11-16T22:30:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30acff9700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 0x7f3088057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f30a406e150 con 0x7f30b0161480 2020-11-16T22:30:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30b4fcb700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 0x7f3088057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:57.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.181+0000 7f30b4fcb700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 0x7f3088057520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f30a0004780 tx=0x7f30a00012d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.185+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f30a4044240 con 0x7f30b0161480 2020-11-16T22:30:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.389+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b00671e0 con 0x7f30b0161480 2020-11-16T22:30:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30acff9700 1 -- 172.21.15.181:0/111626169 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f30a4047860 con 0x7f30b0161480 2020-11-16T22:30:57.373 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:57.373 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6313,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:30:54.519644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:30:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 msgr2=0x7f3088057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 0x7f3088057520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f30a0004780 tx=0x7f30a00012d0).stop 2020-11-16T22:30:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 msgr2=0x7f30b0167d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f30a400ebd0 tx=0x7f30a4006df0).stop 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 shutdown_connections 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3088055060 0x7f3088057520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0161860 0x7f30b00d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30b0062740 0x7f30b0167840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 --2- 172.21.15.181:0/111626169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30b0161480 0x7f30b0167d30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 >> 172.21.15.181:0/111626169 conn(0x7f30b00d2dc0 msgr2=0x7f30b00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 shutdown_connections 2020-11-16T22:30:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:57.393+0000 7f30b5fcd700 1 -- 172.21.15.181:0/111626169 wait complete. 2020-11-16T22:30:57.377 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 62 2020-11-16T22:30:58.295 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:30:53.516445+0000', 'modified': '2020-11-16T22:30:54.519644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6316}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6316': {'gid': 6316, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3625693713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3625693713}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3625693713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:30:58.296 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:30:58.296 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:30:58.434 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6316,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.924517561,"mdsmap_epoch":62,"osdmap_epoch":64,"osdmap_epoch_barrier":63,"uptime":9.3183180809999993} 2020-11-16T22:30:58.435 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6316, "mdsmap_epoch": 62, "osdmap_epoch": 64, "osdmap_epoch_barrier": 63, "rank_uptime": 4.924517561, "state": "up:active", "uptime": 9.318318081, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:30:58.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:30:58.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:30:58.436 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:30:58.436 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:30:58.436 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:30:58.437 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:30:58.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:30:58.437 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:30:58.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.438 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:30:58.456 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:30:58.456 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:30:58.456 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:30:58.456 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.456 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:30:58.457 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.458 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:30:58.459 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:30:58.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.461 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:30:58.531 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:30:58.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.536 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:30:58.551 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:30:58.551 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:30:58.551 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:30:58.551 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:30:58.551 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:30:58.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.552 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:30:58.553 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:30:58.553 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:30:58.553 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:30:58.554 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:30:58.554 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:30:58.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.653 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:30:58.668 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:30:58.668 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:30:58.669 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:30:58.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:30:58.669 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:30:58.700 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:30:58.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 Processor -- start 2020-11-16T22:30:58.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 -- start start 2020-11-16T22:30:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5e90142660 0x7f5e90142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e90062700 con 0x7f5e90160530 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e900629a0 con 0x7f5e90142660 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e96d97700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90062c40 con 0x7f5e90063620 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40470/0 (socket says 172.21.15.181:40470) 2020-11-16T22:30:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 -- 172.21.15.181:0/301693189 learned_addr learned my addr 172.21.15.181:0/301693189 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95d95700 1 --2- 172.21.15.181:0/301693189 >> v2:172.21.15.175:3300/0 conn(0x7f5e90142660 0x7f5e90142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 -- 172.21.15.181:0/301693189 >> v2:172.21.15.175:3300/0 conn(0x7f5e90142660 msgr2=0x7f5e90142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 --2- 172.21.15.181:0/301693189 >> v2:172.21.15.175:3300/0 conn(0x7f5e90142660 0x7f5e90142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 -- 172.21.15.181:0/301693189 >> v1:172.21.15.175:6789/0 conn(0x7f5e90160530 legacy=0x7f5e90061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.849+0000 7f5e95594700 1 -- 172.21.15.181:0/301693189 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90062ee0 con 0x7f5e90063620 2020-11-16T22:30:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e95594700 1 --2- 172.21.15.181:0/301693189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f5e80006fd0 tx=0x7f5e80006eb0).ready entity=mon.2 client_cookie=b712404cc230c63c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e94d93700 1 -- 172.21.15.181:0/301693189 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800098a0 con 0x7f5e90063620 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e94d93700 1 -- 172.21.15.181:0/301693189 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80009a00 con 0x7f5e90063620 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e94d93700 1 -- 172.21.15.181:0/301693189 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80009d80 con 0x7f5e90063620 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- 172.21.15.181:0/301693189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 msgr2=0x7f5e90063ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 --2- 172.21.15.181:0/301693189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f5e80006fd0 tx=0x7f5e80006eb0).stop 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- 172.21.15.181:0/301693189 shutdown_connections 2020-11-16T22:30:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 --2- 172.21.15.181:0/301693189 >> v2:172.21.15.175:3300/0 conn(0x7f5e90142660 0x7f5e90142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 --2- 172.21.15.181:0/301693189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90063620 0x7f5e90063ec0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- 172.21.15.181:0/301693189 >> 172.21.15.181:0/301693189 conn(0x7f5e900d2dc0 msgr2=0x7f5e900ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- 172.21.15.181:0/301693189 shutdown_connections 2020-11-16T22:30:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- 172.21.15.181:0/301693189 wait complete. 2020-11-16T22:30:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 Processor -- start 2020-11-16T22:30:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.853+0000 7f5e96d97700 1 -- start start 2020-11-16T22:30:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90063620 0x7f5e900cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90160530 0x7f5e9016fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90140bd0 con 0x7f5e90160530 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90143070 con 0x7f5e90063620 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90143310 con 0x7f5e90142660 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40474/0 (socket says 172.21.15.181:40474) 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 -- 172.21.15.181:0/1683762397 learned_addr learned my addr 172.21.15.181:0/1683762397 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95d95700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90063620 0x7f5e900cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96596700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90160530 0x7f5e9016fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 -- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90063620 msgr2=0x7f5e900cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90063620 0x7f5e900cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 -- 172.21.15.181:0/1683762397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90160530 msgr2=0x7f5e9016fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90160530 0x7f5e9016fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 -- 172.21.15.181:0/1683762397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e901435b0 con 0x7f5e90142660 2020-11-16T22:30:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95594700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5e80006ef0 tx=0x7f5e80015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80011d50 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8001e460 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8001e820 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e90143850 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e900cd370 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5e80025030 con 0x7f5e90142660 2020-11-16T22:30:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e90044f60 con 0x7f5e90142660 2020-11-16T22:30:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 0x7f5e68057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e8006f550 con 0x7f5e90142660 2020-11-16T22:30:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.857+0000 7f5e95d95700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 0x7f5e68057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.861+0000 7f5e95d95700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 0x7f5e68057520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f5e900c80e0 tx=0x7f5e8c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:58.865+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5e800457f0 con 0x7f5e90142660 2020-11-16T22:30:59.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.021+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e900cd590 con 0x7f5e90142660 2020-11-16T22:30:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.021+0000 7f5e7dffb700 1 -- 172.21.15.181:0/1683762397 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f5e80023020 con 0x7f5e90142660 2020-11-16T22:30:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:30:59.004 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:54.520377+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:30:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 msgr2=0x7f5e68057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 0x7f5e68057520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f5e900c80e0 tx=0x7f5e8c006df0).stop 2020-11-16T22:30:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 msgr2=0x7f5e9016f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5e80006ef0 tx=0x7f5e80015000).stop 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 shutdown_connections 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5e68055060 0x7f5e68057520 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e90160530 0x7f5e9016fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e90063620 0x7f5e900cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 --2- 172.21.15.181:0/1683762397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e90142660 0x7f5e9016f840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 >> 172.21.15.181:0/1683762397 conn(0x7f5e900d2dc0 msgr2=0x7f5e9015eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:59.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 shutdown_connections 2020-11-16T22:30:59.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.025+0000 7f5e96d97700 1 -- 172.21.15.181:0/1683762397 wait complete. 2020-11-16T22:30:59.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:30:52.179372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:30:52.615461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:30:59.222 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:30:59.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 Processor -- start 2020-11-16T22:30:59.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 -- start start 2020-11-16T22:30:59.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc413e870 0x7fbcc413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc415d180 con 0x7fbcc4061810 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc413f140 con 0x7fbcc415fdc0 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc3fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc415d810 con 0x7fbcc413e870 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc413e870 0x7fbcc413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcba7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbcc415fdc0 0x7fbcc4061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46944/0 (socket says 172.21.15.181:46944) 2020-11-16T22:30:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcba7fc700 1 -- 172.21.15.181:0/4003323897 learned_addr learned my addr 172.21.15.181:0/4003323897 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:59.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc2ffd700 1 -- 172.21.15.181:0/4003323897 >> v1:172.21.15.181:6789/0 conn(0x7fbcc415fdc0 legacy=0x7fbcc4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc2ffd700 1 -- 172.21.15.181:0/4003323897 >> v1:172.21.15.175:6789/0 conn(0x7fbcc4061810 legacy=0x7fbcc415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc2ffd700 1 -- 172.21.15.181:0/4003323897 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc415dab0 con 0x7fbcc413e870 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc2ffd700 1 --2- 172.21.15.181:0/4003323897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc413e870 0x7fbcc413ec50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fbcb400b9a0 tx=0x7fbcb4006df0).ready entity=mon.1 client_cookie=9fca066691aefe39 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.381+0000 7fbcc27fc700 1 -- 172.21.15.181:0/4003323897 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4014070 con 0x7fbcc413e870 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc27fc700 1 -- 172.21.15.181:0/4003323897 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4009770 con 0x7fbcc413e870 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc27fc700 1 -- 172.21.15.181:0/4003323897 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4009af0 con 0x7fbcc413e870 2020-11-16T22:30:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- 172.21.15.181:0/4003323897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc413e870 msgr2=0x7fbcc413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/4003323897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc413e870 0x7fbcc413ec50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fbcb400b9a0 tx=0x7fbcb4006df0).stop 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- 172.21.15.181:0/4003323897 shutdown_connections 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/4003323897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc413e870 0x7fbcc413ec50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- 172.21.15.181:0/4003323897 >> 172.21.15.181:0/4003323897 conn(0x7fbcc40cf630 msgr2=0x7fbcc40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- 172.21.15.181:0/4003323897 shutdown_connections 2020-11-16T22:30:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- 172.21.15.181:0/4003323897 wait complete. 2020-11-16T22:30:59.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 Processor -- start 2020-11-16T22:30:59.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.385+0000 7fbcc3fff700 1 -- start start 2020-11-16T22:30:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc4061810 0x7fbcc413cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcc413e870 0x7fbcc40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc413a820 con 0x7fbcc413e870 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc413d5a0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc413d840 con 0x7fbcc4061810 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60880/0 (socket says 172.21.15.181:60880) 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc4061810 0x7fbcc413cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 -- 172.21.15.181:0/2048862313 learned_addr learned my addr 172.21.15.181:0/2048862313 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:30:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcba7fc700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcc413e870 0x7fbcc40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 -- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc4061810 msgr2=0x7fbcc413cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc4061810 0x7fbcc413cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 -- 172.21.15.181:0/2048862313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcc413e870 msgr2=0x7fbcc40cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcc413e870 0x7fbcc40cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 -- 172.21.15.181:0/2048862313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc413dae0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc37fe700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fbcbc00cae0 tx=0x7fbcbc007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00a730 con 0x7fbcc415fdc0 2020-11-16T22:30:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc00a8b0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc413dd80 con 0x7fbcc415fdc0 2020-11-16T22:30:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00ac70 con 0x7fbcc415fdc0 2020-11-16T22:30:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc416f1a0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc416f390 con 0x7fbcc415fdc0 2020-11-16T22:30:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.389+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbcbc019030 con 0x7fbcc415fdc0 2020-11-16T22:30:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.393+0000 7fbcbb7fe700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 0x7fbca0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.393+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcbc06d4e0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.393+0000 7fbcc2ffd700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 0x7fbca0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.393+0000 7fbcc2ffd700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 0x7fbca0057550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fbcb400b150 tx=0x7fbcb4016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:30:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.397+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbcbc0437f0 con 0x7fbcc415fdc0 2020-11-16T22:30:59.448 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:59.455 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:59.458 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:30:59.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.549+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbcc413a960 con 0x7fbcc415fdc0 2020-11-16T22:30:59.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.549+0000 7fbcbb7fe700 1 -- 172.21.15.181:0/2048862313 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v64) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbcbc046e10 con 0x7fbcc415fdc0 2020-11-16T22:30:59.532 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:30:59.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 msgr2=0x7fbca0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 0x7fbca0057550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fbcb400b150 tx=0x7fbcb4016000).stop 2020-11-16T22:30:59.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 msgr2=0x7fbcc40cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fbcbc00cae0 tx=0x7fbcbc007f20).stop 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 shutdown_connections 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbca0055090 0x7fbca0057550 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415fdc0 0x7fbcc40cdaf0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc4061810 0x7fbcc413cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 --2- 172.21.15.181:0/2048862313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcc413e870 0x7fbcc40cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 >> 172.21.15.181:0/2048862313 conn(0x7fbcc40cf630 msgr2=0x7fbcc40d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:30:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 shutdown_connections 2020-11-16T22:30:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.553+0000 7fbcc3fff700 1 -- 172.21.15.181:0/2048862313 wait complete. 2020-11-16T22:30:59.545 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:30:59.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.737+0000 7f99ef59e700 1 Processor -- start 2020-11-16T22:30:59.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.737+0000 7f99ef59e700 1 -- start start 2020-11-16T22:30:59.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99f0161480 0x7f99f0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:30:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00c8db0 con 0x7f99f0164e10 2020-11-16T22:30:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00c8330 con 0x7f99f0161480 2020-11-16T22:30:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c85d0 con 0x7f99f0161860 2020-11-16T22:30:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99edd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f99f0164e10 0x7f99f0062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46960/0 (socket says 172.21.15.181:46960) 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40494/0 (socket says 172.21.15.181:40494) 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99edd9b700 1 -- 172.21.15.181:0/2104137914 learned_addr learned my addr 172.21.15.181:0/2104137914 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ee59c700 1 --2- 172.21.15.181:0/2104137914 >> v2:172.21.15.175:3300/0 conn(0x7f99f0161480 0x7f99f0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 -- 172.21.15.181:0/2104137914 >> v2:172.21.15.175:3300/0 conn(0x7f99f0161480 msgr2=0x7f99f0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 --2- 172.21.15.181:0/2104137914 >> v2:172.21.15.175:3300/0 conn(0x7f99f0161480 0x7f99f0164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:30:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ed59a700 1 -- 172.21.15.181:0/2104137914 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535875173 0 0) 0x7f99f00c8db0 con 0x7f99f0164e10 2020-11-16T22:30:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 -- 172.21.15.181:0/2104137914 >> v1:172.21.15.181:6789/0 conn(0x7f99f0164e10 legacy=0x7f99f0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:30:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 -- 172.21.15.181:0/2104137914 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00c8870 con 0x7f99f0161860 2020-11-16T22:30:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99eed9d700 1 --2- 172.21.15.181:0/2104137914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f99e800b9a0 tx=0x7f99e8006df0).ready entity=mon.2 client_cookie=647966caaab30557 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ed59a700 1 -- 172.21.15.181:0/2104137914 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80097b0 con 0x7f99f0161860 2020-11-16T22:31:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ed59a700 1 -- 172.21.15.181:0/2104137914 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8009910 con 0x7f99f0161860 2020-11-16T22:31:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ed59a700 1 -- 172.21.15.181:0/2104137914 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8009c90 con 0x7f99f0161860 2020-11-16T22:31:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- 172.21.15.181:0/2104137914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 msgr2=0x7f99f0062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 --2- 172.21.15.181:0/2104137914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f99e800b9a0 tx=0x7f99e8006df0).stop 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- 172.21.15.181:0/2104137914 shutdown_connections 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 --2- 172.21.15.181:0/2104137914 >> v2:172.21.15.175:3300/0 conn(0x7f99f0161480 0x7f99f0164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 --2- 172.21.15.181:0/2104137914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0062960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- 172.21.15.181:0/2104137914 >> 172.21.15.181:0/2104137914 conn(0x7f99f00d2dc0 msgr2=0x7f99f00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- 172.21.15.181:0/2104137914 shutdown_connections 2020-11-16T22:31:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.741+0000 7f99ef59e700 1 -- 172.21.15.181:0/2104137914 wait complete. 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 Processor -- start 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 -- start start 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 0x7f99f0162e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0163460 con 0x7f99f0161480 2020-11-16T22:31:00.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0163700 con 0x7f99f00d0120 2020-11-16T22:31:00.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01639a0 con 0x7f99f0161860 2020-11-16T22:31:00.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60894/0 (socket says 172.21.15.181:60894) 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99eed9d700 1 -- 172.21.15.181:0/1873208345 learned_addr learned my addr 172.21.15.181:0/1873208345 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40502/0 (socket says 172.21.15.181:40502) 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ee59c700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 0x7f99f0162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 -- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 msgr2=0x7f99f01491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:00.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 -- 172.21.15.181:0/1873208345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 msgr2=0x7f99f0162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:00.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 0x7f99f0162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 -- 172.21.15.181:0/1873208345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0163c40 con 0x7f99f0161860 2020-11-16T22:31:00.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99eed9d700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:00.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ee59c700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 0x7f99f0162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:00.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99edd9b700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f99d800c730 tx=0x7f99d8007960).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:00.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80072e0 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d8007440 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8005800 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.745+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0144650 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0144b90 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0044f60 con 0x7f99f0161860 2020-11-16T22:31:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f99d8010070 con 0x7f99f0161860 2020-11-16T22:31:00.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99e67fc700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 0x7f99cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:00.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99d806d450 con 0x7f99f0161860 2020-11-16T22:31:00.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99ee59c700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 0x7f99cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:00.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.749+0000 7f99ee59c700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 0x7f99cc057520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f99e000cd30 tx=0x7f99e0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:00.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.757+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f99d80436b0 con 0x7f99f0161860 2020-11-16T22:31:00.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.905+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99f00671e0 con 0x7f99f0161860 2020-11-16T22:31:00.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99e67fc700 1 -- 172.21.15.181:0/1873208345 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99d8018020 con 0x7f99f0161860 2020-11-16T22:31:00.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 msgr2=0x7f99cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 0x7f99cc057520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f99e000cd30 tx=0x7f99e0008040).stop 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 msgr2=0x7f99f0145030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f99d800c730 tx=0x7f99d8007960).stop 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 shutdown_connections 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99cc055060 0x7f99cc057520 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99f00d0120 0x7f99f01491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99f0161480 0x7f99f0162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 --2- 172.21.15.181:0/1873208345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99f0161860 0x7f99f0145030 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 >> 172.21.15.181:0/1873208345 conn(0x7f99f00d2dc0 msgr2=0x7f99f0063220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 shutdown_connections 2020-11-16T22:31:00.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:30:59.909+0000 7f99ef59e700 1 -- 172.21.15.181:0/1873208345 wait complete. 2020-11-16T22:31:00.954 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:31:00.955 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:31:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.113+0000 7f82a0c63700 1 Processor -- start 2020-11-16T22:31:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.113+0000 7f82a0c63700 1 -- start start 2020-11-16T22:31:01.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c15d140 0x7f829c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f829c0cf920 0x7f829c0656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829c0c85e0 con 0x7f829c0cf4a0 2020-11-16T22:31:01.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829c15ff70 con 0x7f829c0cf920 2020-11-16T22:31:01.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c0c8840 con 0x7f829c15d140 2020-11-16T22:31:01.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f829c0cf920 0x7f829c0656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c15d140 0x7f829c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f829c0cf920 0x7f829c0656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60898/0 (socket says 172.21.15.181:60898) 2020-11-16T22:31:01.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 -- 172.21.15.181:0/3513369400 learned_addr learned my addr 172.21.15.181:0/3513369400 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 -- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c15d140 msgr2=0x7f829c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 --2- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c15d140 0x7f829c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 -- 172.21.15.181:0/3513369400 >> v1:172.21.15.175:6789/0 conn(0x7f829c0cf4a0 legacy=0x7f829c064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 -- 172.21.15.181:0/3513369400 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c0c8ae0 con 0x7f829c0cf920 2020-11-16T22:31:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829bfff700 1 --2- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c0cf920 0x7f829c0656e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f8294006fd0 tx=0x7f8294006eb0).ready entity=mon.1 client_cookie=988c6168ee6f4b5f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829a7fc700 1 -- 172.21.15.181:0/3513369400 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82940097d0 con 0x7f829c0cf920 2020-11-16T22:31:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829a7fc700 1 -- 172.21.15.181:0/3513369400 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294009930 con 0x7f829c0cf920 2020-11-16T22:31:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f829a7fc700 1 -- 172.21.15.181:0/3513369400 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294009cb0 con 0x7f829c0cf920 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c0cf920 msgr2=0x7f829c0656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 --2- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c0cf920 0x7f829c0656e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f8294006fd0 tx=0x7f8294006eb0).stop 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- 172.21.15.181:0/3513369400 shutdown_connections 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 --2- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c0cf920 0x7f829c0656e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 --2- 172.21.15.181:0/3513369400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c15d140 0x7f829c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- 172.21.15.181:0/3513369400 >> 172.21.15.181:0/3513369400 conn(0x7f829c0d2dc0 msgr2=0x7f829c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:01.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- 172.21.15.181:0/3513369400 shutdown_connections 2020-11-16T22:31:01.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.117+0000 7f82a0c63700 1 -- 172.21.15.181:0/3513369400 wait complete. 2020-11-16T22:31:01.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 Processor -- start 2020-11-16T22:31:01.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- start start 2020-11-16T22:31:01.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f829c0cf4a0 0x7f829c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c0cf920 0x7f829c1704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c0b2080 con 0x7f829c0cf4a0 2020-11-16T22:31:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c0b2320 con 0x7f829c15d140 2020-11-16T22:31:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c0b25c0 con 0x7f829c0cf920 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60906/0 (socket says 172.21.15.181:60906) 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 -- 172.21.15.181:0/4077418755 learned_addr learned my addr 172.21.15.181:0/4077418755 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829affd700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c0cf920 0x7f829c1704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829b7fe700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f829c0cf4a0 0x7f829c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 -- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c0cf920 msgr2=0x7f829c1704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c0cf920 0x7f829c1704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 -- 172.21.15.181:0/4077418755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f829c0cf4a0 msgr2=0x7f829c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f829c0cf4a0 0x7f829c15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 -- 172.21.15.181:0/4077418755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c0b2860 con 0x7f829c15d140 2020-11-16T22:31:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f829bfff700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f8294006ef0 tx=0x7f8294015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82940097d0 con 0x7f829c15d140 2020-11-16T22:31:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294006b20 con 0x7f829c15d140 2020-11-16T22:31:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c0b2b00 con 0x7f829c15d140 2020-11-16T22:31:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829401d800 con 0x7f829c15d140 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.121+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c0b2fa0 con 0x7f829c15d140 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c044f60 con 0x7f829c15d140 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8294025030 con 0x7f829c15d140 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f82937fe700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 0x7f826c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f829406f570 con 0x7f829c15d140 2020-11-16T22:31:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f829b7fe700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 0x7f826c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:01.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.125+0000 7f829b7fe700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 0x7f826c057520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f828c009aa0 tx=0x7f828c0091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:01.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.133+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f82940457d0 con 0x7f829c15d140 2020-11-16T22:31:01.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.289+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f829c062dc0 con 0x7f829c15d140 2020-11-16T22:31:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82937fe700 1 -- 172.21.15.181:0/4077418755 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8294023070 con 0x7f829c15d140 2020-11-16T22:31:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 msgr2=0x7f826c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 0x7f826c057520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f828c009aa0 tx=0x7f828c0091d0).stop 2020-11-16T22:31:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 msgr2=0x7f829c1709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f8294006ef0 tx=0x7f8294015000).stop 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 shutdown_connections 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f826c055060 0x7f826c057520 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c15d140 0x7f829c1709c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f829c0cf4a0 0x7f829c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 --2- 172.21.15.181:0/4077418755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f829c0cf920 0x7f829c1704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 >> 172.21.15.181:0/4077418755 conn(0x7f829c0d2dc0 msgr2=0x7f829c15dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:01.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 shutdown_connections 2020-11-16T22:31:01.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:01.293+0000 7f82a0c63700 1 -- 172.21.15.181:0/4077418755 wait complete. 2020-11-16T22:31:01.920 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:31:01.920 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:31:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.281+0000 7fe698a45700 1 Processor -- start 2020-11-16T22:31:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.281+0000 7fe698a45700 1 -- start start 2020-11-16T22:31:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe6940cf390 0x7fe6940cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe694163970 0x7fe694163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe694065190 con 0x7fe6940600f0 2020-11-16T22:31:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6940cfc60 con 0x7fe6940cf390 2020-11-16T22:31:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe694065430 con 0x7fe694163970 2020-11-16T22:31:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe694163970 0x7fe694163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69359e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe6940600f0 0x7fe694060580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46986/0 (socket says 172.21.15.181:46986) 2020-11-16T22:31:02.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69359e700 1 -- 172.21.15.181:0/1509194842 learned_addr learned my addr 172.21.15.181:0/1509194842 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:02.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe692d9d700 1 --2- 172.21.15.181:0/1509194842 >> v2:172.21.15.175:3300/0 conn(0x7fe6940cf390 0x7fe6940cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe691d9b700 1 -- 172.21.15.181:0/1509194842 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543123234 0 0) 0x7fe694065190 con 0x7fe6940600f0 2020-11-16T22:31:02.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 -- 172.21.15.181:0/1509194842 >> v2:172.21.15.175:3300/0 conn(0x7fe6940cf390 msgr2=0x7fe6940cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:02.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 --2- 172.21.15.181:0/1509194842 >> v2:172.21.15.175:3300/0 conn(0x7fe6940cf390 0x7fe6940cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 -- 172.21.15.181:0/1509194842 >> v1:172.21.15.181:6789/0 conn(0x7fe6940600f0 legacy=0x7fe694060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 -- 172.21.15.181:0/1509194842 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6940656d0 con 0x7fe694163970 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe69259c700 1 --2- 172.21.15.181:0/1509194842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe694163df0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fe6940ca440 tx=0x7fe6880091d0).ready entity=mon.1 client_cookie=435ca938e3621f86 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe691d9b700 1 -- 172.21.15.181:0/1509194842 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68800ba70 con 0x7fe694163970 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe691d9b700 1 -- 172.21.15.181:0/1509194842 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68800bbd0 con 0x7fe694163970 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe691d9b700 1 -- 172.21.15.181:0/1509194842 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688011400 con 0x7fe694163970 2020-11-16T22:31:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- 172.21.15.181:0/1509194842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 msgr2=0x7fe694163df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 --2- 172.21.15.181:0/1509194842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe694163df0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fe6940ca440 tx=0x7fe6880091d0).stop 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- 172.21.15.181:0/1509194842 shutdown_connections 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 --2- 172.21.15.181:0/1509194842 >> v2:172.21.15.175:3300/0 conn(0x7fe6940cf390 0x7fe6940cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 --2- 172.21.15.181:0/1509194842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe694163df0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- 172.21.15.181:0/1509194842 >> 172.21.15.181:0/1509194842 conn(0x7fe6940cfe90 msgr2=0x7fe6940d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- 172.21.15.181:0/1509194842 shutdown_connections 2020-11-16T22:31:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.285+0000 7fe698a45700 1 -- 172.21.15.181:0/1509194842 wait complete. 2020-11-16T22:31:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 Processor -- start 2020-11-16T22:31:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 -- start start 2020-11-16T22:31:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 0x7fe69416f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69416fd20 con 0x7fe6940cf390 2020-11-16T22:31:02.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69416ffc0 con 0x7fe694163970 2020-11-16T22:31:02.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe698a45700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe694170260 con 0x7fe6940600f0 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe69359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe692d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe692d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40528/0 (socket says 172.21.15.181:40528) 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe69359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60922/0 (socket says 172.21.15.181:60922) 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe692d9d700 1 -- 172.21.15.181:0/667567493 learned_addr learned my addr 172.21.15.181:0/667567493 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:02.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.289+0000 7fe69259c700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 0x7fe69416f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 -- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 msgr2=0x7fe694164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 -- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 msgr2=0x7fe69413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 -- 172.21.15.181:0/667567493 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69413e410 con 0x7fe6940cf390 2020-11-16T22:31:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69359e700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe692d9d700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe69259c700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 0x7fe69416f3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fe6940ca440 tx=0x7fe688016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68801c5f0 con 0x7fe6940cf390 2020-11-16T22:31:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68801c750 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68801caf0 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69413e6b0 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.293+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69416f8e0 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6940658b0 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe688017070 con 0x7fe6940cf390 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe67affd700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe68806fe10 con 0x7fe6940cf390 2020-11-16T22:31:02.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe692d9d700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:02.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.297+0000 7fe692d9d700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 0x7fe670057520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fe684006a90 tx=0x7fe684006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:02.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.309+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe688046070 con 0x7fe6940cf390 2020-11-16T22:31:02.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.461+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe694044f60 con 0x7fe6940cf390 2020-11-16T22:31:02.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe67affd700 1 -- 172.21.15.181:0/667567493 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe688015070 con 0x7fe6940cf390 2020-11-16T22:31:02.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 msgr2=0x7fe670057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 0x7fe670057520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fe684006a90 tx=0x7fe684006df0).stop 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 msgr2=0x7fe69416f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 0x7fe69416f3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fe6940ca440 tx=0x7fe688016000).stop 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 shutdown_connections 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe694163970 0x7fe69413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6940600f0 0x7fe694164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 --2- 172.21.15.181:0/667567493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6940cf390 0x7fe69416f3f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:02.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 >> 172.21.15.181:0/667567493 conn(0x7fe6940cfe90 msgr2=0x7fe694060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:03.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 shutdown_connections 2020-11-16T22:31:03.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:02.465+0000 7fe698a45700 1 -- 172.21.15.181:0/667567493 wait complete. 2020-11-16T22:31:03.090 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:31:03.091 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:31:03.092 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:03.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.241+0000 7f7b64f7b700 1 Processor -- start 2020-11-16T22:31:03.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.241+0000 7f7b64f7b700 1 -- start start 2020-11-16T22:31:03.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b6013ec50 0x7f7b60061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 0x7f7b6015e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6015d180 con 0x7f7b6013e870 2020-11-16T22:31:03.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6015d810 con 0x7f7b6013ec50 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015dab0 con 0x7f7b600615f0 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7b6013e870 0x7f7b6015fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47000/0 (socket says 172.21.15.181:47000) 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5f7fe700 1 -- 172.21.15.181:0/718213545 learned_addr learned my addr 172.21.15.181:0/718213545 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/718213545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 0x7f7b6015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5effd700 1 --2- 172.21.15.181:0/718213545 >> v2:172.21.15.175:3300/0 conn(0x7f7b6013ec50 0x7f7b60061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5e7fc700 1 -- 172.21.15.181:0/718213545 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773741955 0 0) 0x7f7b6015d180 con 0x7f7b6013e870 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5e7fc700 1 -- 172.21.15.181:0/718213545 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b44003240 con 0x7f7b6013e870 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 -- 172.21.15.181:0/718213545 >> v2:172.21.15.175:3300/0 conn(0x7f7b6013ec50 msgr2=0x7f7b60061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/718213545 >> v2:172.21.15.175:3300/0 conn(0x7f7b6013ec50 0x7f7b60061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 -- 172.21.15.181:0/718213545 >> v1:172.21.15.181:6789/0 conn(0x7f7b6013e870 legacy=0x7f7b6015fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 -- 172.21.15.181:0/718213545 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6015dd50 con 0x7f7b600615f0 2020-11-16T22:31:03.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/718213545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 0x7f7b6015e730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7b5800b9a0 tx=0x7f7b58006df0).ready entity=mon.2 client_cookie=a963d8508e9073b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:03.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5e7fc700 1 -- 172.21.15.181:0/718213545 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58009680 con 0x7f7b600615f0 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5e7fc700 1 -- 172.21.15.181:0/718213545 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b580097e0 con 0x7f7b600615f0 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b5e7fc700 1 -- 172.21.15.181:0/718213545 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58009b60 con 0x7f7b600615f0 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- 172.21.15.181:0/718213545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 msgr2=0x7f7b6015e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/718213545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 0x7f7b6015e730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7b5800b9a0 tx=0x7f7b58006df0).stop 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- 172.21.15.181:0/718213545 shutdown_connections 2020-11-16T22:31:03.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/718213545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600615f0 0x7f7b6015e730 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/718213545 >> v2:172.21.15.175:3300/0 conn(0x7f7b6013ec50 0x7f7b60061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- 172.21.15.181:0/718213545 >> 172.21.15.181:0/718213545 conn(0x7f7b600cf630 msgr2=0x7f7b600cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:03.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- 172.21.15.181:0/718213545 shutdown_connections 2020-11-16T22:31:03.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.245+0000 7f7b64f7b700 1 -- 172.21.15.181:0/718213545 wait complete. 2020-11-16T22:31:03.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 Processor -- start 2020-11-16T22:31:03.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 -- start start 2020-11-16T22:31:03.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 0x7f7b600c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b6013e870 0x7f7b60158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015a1e0 con 0x7f7b600615f0 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015a440 con 0x7f7b6013ec50 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b64f7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015a6e0 con 0x7f7b6013e870 2020-11-16T22:31:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b6013e870 0x7f7b60158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60936/0 (socket says 172.21.15.181:60936) 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 -- 172.21.15.181:0/1065339763 learned_addr learned my addr 172.21.15.181:0/1065339763 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5f7fe700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 0x7f7b600c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 -- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b6013e870 msgr2=0x7f7b60158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b6013e870 0x7f7b60158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 -- 172.21.15.181:0/1065339763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 msgr2=0x7f7b600c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 0x7f7b600c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 -- 172.21.15.181:0/1065339763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6015a980 con 0x7f7b6013ec50 2020-11-16T22:31:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5f7fe700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 0x7f7b600c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b5ffff700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f7b58006fd0 tx=0x7f7b58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.249+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58006b20 con 0x7f7b6013ec50 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58011e00 con 0x7f7b6013ec50 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6015ac20 con 0x7f7b6013ec50 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5801d7e0 con 0x7f7b6013ec50 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b600c9840 con 0x7f7b6013ec50 2020-11-16T22:31:03.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b600c9a90 con 0x7f7b6013ec50 2020-11-16T22:31:03.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7b58025030 con 0x7f7b6013ec50 2020-11-16T22:31:03.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b577fe700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 0x7f7b30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:03.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b5806f680 con 0x7f7b6013ec50 2020-11-16T22:31:03.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b5f7fe700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 0x7f7b30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:03.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.253+0000 7f7b5f7fe700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 0x7f7b30057520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f7b500055a0 tx=0x7f7b50009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:03.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.261+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7b58045880 con 0x7f7b6013ec50 2020-11-16T22:31:03.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b60044f60 con 0x7f7b6013ec50 2020-11-16T22:31:03.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b577fe700 1 -- 172.21.15.181:0/1065339763 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4627 (secure 0 0 0) 0x7f7b58023070 con 0x7f7b6013ec50 2020-11-16T22:31:03.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 msgr2=0x7f7b30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 0x7f7b30057520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f7b500055a0 tx=0x7f7b50009040).stop 2020-11-16T22:31:03.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 msgr2=0x7f7b60158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f7b58006fd0 tx=0x7f7b58015000).stop 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 shutdown_connections 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7b30055060 0x7f7b30057520 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b6013ec50 0x7f7b60158a10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b600615f0 0x7f7b600c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:03.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 --2- 172.21.15.181:0/1065339763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b6013e870 0x7f7b60158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:04.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 >> 172.21.15.181:0/1065339763 conn(0x7f7b600cf630 msgr2=0x7f7b600cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:04.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 shutdown_connections 2020-11-16T22:31:04.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:03.461+0000 7f7b64f7b700 1 -- 172.21.15.181:0/1065339763 wait complete. 2020-11-16T22:31:04.050 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 62 2020-11-16T22:31:04.051 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:04.051 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6313,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:30:54.519644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:05.160 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:30:53.516445+0000', 'modified': '2020-11-16T22:30:54.519644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6316}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6316': {'gid': 6316, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3625693713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3625693713}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3625693713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:31:05.161 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:31:05.298 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6316,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.789011093999999,"mdsmap_epoch":62,"osdmap_epoch":64,"osdmap_epoch_barrier":63,"uptime":16.182794736000002} 2020-11-16T22:31:05.299 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6316, "mdsmap_epoch": 62, "osdmap_epoch": 64, "osdmap_epoch_barrier": 63, "rank_uptime": 11.789011094, "state": "up:active", "uptime": 16.182794736, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:31:05.299 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:31:05.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:31:05.300 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:31:05.319 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:31:05.319 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:31:05.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:31:05.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.613+0000 7fbb47fff700 1 Processor -- start 2020-11-16T22:31:05.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.613+0000 7fbb47fff700 1 -- start start 2020-11-16T22:31:05.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb48161480 0x7fbb48164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb480c8db0 con 0x7fbb48164e10 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb480c8330 con 0x7fbb48161480 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480c85d0 con 0x7fbb48161860 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40554/0 (socket says 172.21.15.181:40554) 2020-11-16T22:31:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 -- 172.21.15.181:0/105327087 learned_addr learned my addr 172.21.15.181:0/105327087 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb46ffd700 1 --2- 172.21.15.181:0/105327087 >> v2:172.21.15.175:3300/0 conn(0x7fbb48161480 0x7fbb48164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 -- 172.21.15.181:0/105327087 >> v2:172.21.15.175:3300/0 conn(0x7fbb48161480 msgr2=0x7fbb48164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 --2- 172.21.15.181:0/105327087 >> v2:172.21.15.175:3300/0 conn(0x7fbb48161480 0x7fbb48164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 -- 172.21.15.181:0/105327087 >> v1:172.21.15.175:6789/0 conn(0x7fbb48164e10 legacy=0x7fbb48062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 -- 172.21.15.181:0/105327087 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480c8870 con 0x7fbb48161860 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb477fe700 1 --2- 172.21.15.181:0/105327087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fbb40009d50 tx=0x7fbb40009290).ready entity=mon.2 client_cookie=72d6a9398aa71529 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb45ffb700 1 -- 172.21.15.181:0/105327087 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000bc80 con 0x7fbb48161860 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb45ffb700 1 -- 172.21.15.181:0/105327087 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4000bde0 con 0x7fbb48161860 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb45ffb700 1 -- 172.21.15.181:0/105327087 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011610 con 0x7fbb48161860 2020-11-16T22:31:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- 172.21.15.181:0/105327087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 msgr2=0x7fbb48062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 --2- 172.21.15.181:0/105327087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fbb40009d50 tx=0x7fbb40009290).stop 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- 172.21.15.181:0/105327087 shutdown_connections 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 --2- 172.21.15.181:0/105327087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48062960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 --2- 172.21.15.181:0/105327087 >> v2:172.21.15.175:3300/0 conn(0x7fbb48161480 0x7fbb48164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- 172.21.15.181:0/105327087 >> 172.21.15.181:0/105327087 conn(0x7fbb480d2dc0 msgr2=0x7fbb480ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- 172.21.15.181:0/105327087 shutdown_connections 2020-11-16T22:31:05.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.617+0000 7fbb47fff700 1 -- 172.21.15.181:0/105327087 wait complete. 2020-11-16T22:31:05.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 Processor -- start 2020-11-16T22:31:05.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- start start 2020-11-16T22:31:05.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48174c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb481607d0 0x7fbb481736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481681f0 con 0x7fbb481607d0 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48146e30 con 0x7fbb48161480 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4814a150 con 0x7fbb48161860 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60946/0 (socket says 172.21.15.181:60946) 2020-11-16T22:31:05.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb467fc700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48174c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 -- 172.21.15.181:0/2468922673 learned_addr learned my addr 172.21.15.181:0/2468922673 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb477fe700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb481607d0 0x7fbb481736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 -- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 msgr2=0x7fbb48174c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48174c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 -- 172.21.15.181:0/2468922673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb481607d0 msgr2=0x7fbb481736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:05.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb481607d0 0x7fbb481736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 -- 172.21.15.181:0/2468922673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48174110 con 0x7fbb48161480 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb46ffd700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fbb3800db90 tx=0x7fbb38008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38009820 con 0x7fbb48161480 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38009980 con 0x7fbb48161480 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38009d00 con 0x7fbb48161480 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb481743b0 con 0x7fbb48161480 2020-11-16T22:31:05.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.621+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb48149280 con 0x7fbb48161480 2020-11-16T22:31:05.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb38013690 con 0x7fbb48161480 2020-11-16T22:31:05.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48044f60 con 0x7fbb48161480 2020-11-16T22:31:05.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb3effd700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 0x7fbb18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:05.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb3806e080 con 0x7fbb48161480 2020-11-16T22:31:05.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb467fc700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 0x7fbb18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:05.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.625+0000 7fbb467fc700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 0x7fbb18057520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fbb480cd7f0 tx=0x7fbb30007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.633+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbb380442e0 con 0x7fbb48161480 2020-11-16T22:31:05.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:05.837+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fbb480671e0 con 0x7fbb48161480 2020-11-16T22:31:06.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb3effd700 1 -- 172.21.15.181:0/2468922673 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v63) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbb38009e60 con 0x7fbb48161480 2020-11-16T22:31:06.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 msgr2=0x7fbb18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 0x7fbb18057520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fbb480cd7f0 tx=0x7fbb30007b00).stop 2020-11-16T22:31:06.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 msgr2=0x7fbb48149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fbb3800db90 tx=0x7fbb38008040).stop 2020-11-16T22:31:06.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 shutdown_connections 2020-11-16T22:31:06.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb18055060 0x7fbb18057520 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb481607d0 0x7fbb481736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb48161480 0x7fbb48149c60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 --2- 172.21.15.181:0/2468922673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb48161860 0x7fbb48174c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 >> 172.21.15.181:0/2468922673 conn(0x7fbb480d2dc0 msgr2=0x7fbb48164410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 shutdown_connections 2020-11-16T22:31:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.725+0000 7fbb47fff700 1 -- 172.21.15.181:0/2468922673 wait complete. 2020-11-16T22:31:06.726 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:06.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.853+0000 7f0659ef6700 1 Processor -- start 2020-11-16T22:31:06.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.853+0000 7f0659ef6700 1 -- start start 2020-11-16T22:31:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065415f5b0 0x7f06540d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06540615f0 0x7f06540d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0654160e40 con 0x7f06540622a0 2020-11-16T22:31:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06541609a0 con 0x7f06540615f0 2020-11-16T22:31:06.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0654160ba0 con 0x7f065415f5b0 2020-11-16T22:31:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0653fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f06540622a0 0x7f0654062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47030/0 (socket says 172.21.15.181:47030) 2020-11-16T22:31:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0653fff700 1 -- 172.21.15.181:0/1584832524 learned_addr learned my addr 172.21.15.181:0/1584832524 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 --2- 172.21.15.181:0/1584832524 >> v2:172.21.15.181:3300/0 conn(0x7f06540615f0 0x7f06540d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f06537fe700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065415f5b0 0x7f06540d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 -- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065415f5b0 msgr2=0x7f06540d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065415f5b0 0x7f06540d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 -- 172.21.15.181:0/1584832524 >> v1:172.21.15.181:6789/0 conn(0x7f06540622a0 legacy=0x7f0654062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f06527fc700 1 -- 172.21.15.181:0/1584832524 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103757927 0 0) 0x7f0654160e40 con 0x7f06540622a0 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 -- 172.21.15.181:0/1584832524 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654060970 con 0x7f06540615f0 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0652ffd700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540615f0 0x7f06540d1830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f064400b9a0 tx=0x7f0644006df0).ready entity=mon.1 client_cookie=fd7531ac3c528b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:06.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f06527fc700 1 -- 172.21.15.181:0/1584832524 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440097b0 con 0x7f06540615f0 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f06527fc700 1 -- 172.21.15.181:0/1584832524 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644009910 con 0x7f06540615f0 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f06527fc700 1 -- 172.21.15.181:0/1584832524 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644009c90 con 0x7f06540615f0 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540615f0 msgr2=0x7f06540d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540615f0 0x7f06540d1830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f064400b9a0 tx=0x7f0644006df0).stop 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- 172.21.15.181:0/1584832524 shutdown_connections 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065415f5b0 0x7f06540d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 --2- 172.21.15.181:0/1584832524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540615f0 0x7f06540d1830 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- 172.21.15.181:0/1584832524 >> 172.21.15.181:0/1584832524 conn(0x7f06540d69a0 msgr2=0x7f06540d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- 172.21.15.181:0/1584832524 shutdown_connections 2020-11-16T22:31:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.857+0000 7f0659ef6700 1 -- 172.21.15.181:0/1584832524 wait complete. 2020-11-16T22:31:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 Processor -- start 2020-11-16T22:31:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- start start 2020-11-16T22:31:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 0x7f06540d3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065415f5b0 0x7f06541612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065416ff70 con 0x7f06540622a0 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0654170210 con 0x7f065415f5b0 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06541595d0 con 0x7f06540615f0 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40570/0 (socket says 172.21.15.181:40570) 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 -- 172.21.15.181:0/2391639763 learned_addr learned my addr 172.21.15.181:0/2391639763 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0653fff700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065415f5b0 0x7f06541612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0652ffd700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 0x7f06540d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 -- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065415f5b0 msgr2=0x7f06541612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065415f5b0 0x7f06541612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 -- 172.21.15.181:0/2391639763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 msgr2=0x7f06540d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 0x7f06540d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 -- 172.21.15.181:0/2391639763 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654159870 con 0x7f06540615f0 2020-11-16T22:31:06.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0652ffd700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 0x7f06540d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f06537fe700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f063c00eba0 tx=0x7f063c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00c8e0 con 0x7f06540615f0 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c00ca40 con 0x7f06540615f0 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00cde0 con 0x7f06540615f0 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0654159b10 con 0x7f06540615f0 2020-11-16T22:31:06.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065415a250 con 0x7f06540615f0 2020-11-16T22:31:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.861+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0654069eb0 con 0x7f06540615f0 2020-11-16T22:31:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.865+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f063c012600 con 0x7f06540615f0 2020-11-16T22:31:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.865+0000 7f064b7fe700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 0x7f0630057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.865+0000 7f0652ffd700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 0x7f0630057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.865+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f063c06e0a0 con 0x7f06540615f0 2020-11-16T22:31:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.865+0000 7f0652ffd700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 0x7f0630057550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f064400b9a0 tx=0x7f0644015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:06.873+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f063c044300 con 0x7f06540615f0 2020-11-16T22:31:07.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.069+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f065415a440 con 0x7f06540615f0 2020-11-16T22:31:07.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.069+0000 7f064b7fe700 1 -- 172.21.15.181:0/2391639763 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4643 (secure 0 0 0) 0x7f063c016020 con 0x7f06540615f0 2020-11-16T22:31:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:07.053 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:06.725918+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6316,"mds_1":6313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:creating","state_seq":2,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 msgr2=0x7f0630057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 0x7f0630057550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f064400b9a0 tx=0x7f0644015000).stop 2020-11-16T22:31:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 msgr2=0x7f06540d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f063c00eba0 tx=0x7f063c008040).stop 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 shutdown_connections 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0630055090 0x7f0630057550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065415f5b0 0x7f06541612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06540615f0 0x7f06540d3390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 --2- 172.21.15.181:0/2391639763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06540622a0 0x7f06540d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 >> 172.21.15.181:0/2391639763 conn(0x7f06540d69a0 msgr2=0x7f0654061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 shutdown_connections 2020-11-16T22:31:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:07.073+0000 7f0659ef6700 1 -- 172.21.15.181:0/2391639763 wait complete. 2020-11-16T22:31:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 64 2020-11-16T22:31:07.978 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:30:53.516445+0000', 'modified': '2020-11-16T22:31:06.725918+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6316, 'mds_1': 6313}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6313': {'gid': 6313, 'name': 'i', 'rank': 1, 'incarnation': 64, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.15.175:6841/2108826019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 2108826019}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2108826019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6316': {'gid': 6316, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3625693713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3625693713}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3625693713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:31:07.978 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6313:up:creating 2020-11-16T22:31:08.232 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:31:08.249+0000 7f7198fca700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:31:08.980 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.129+0000 7f1b29830700 1 Processor -- start 2020-11-16T22:31:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.129+0000 7f1b29830700 1 -- start start 2020-11-16T22:31:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b24167720 con 0x7f1b24166bb0 2020-11-16T22:31:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b24168390 con 0x7f1b24167ac0 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b5f00 con 0x7f1b240b3930 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40594/0 (socket says 172.21.15.181:40594) 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 -- 172.21.15.181:0/1971553615 learned_addr learned my addr 172.21.15.181:0/1971553615 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b22ffd700 1 --1- 172.21.15.181:0/1971553615 >> v1:172.21.15.181:6789/0 conn(0x7f1b24167ac0 0x7f1b240b3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47056/0 (socket says 172.21.15.181:47056) 2020-11-16T22:31:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 -- 172.21.15.181:0/1971553615 >> v1:172.21.15.181:6789/0 conn(0x7f1b24167ac0 legacy=0x7f1b240b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 -- 172.21.15.181:0/1971553615 >> v1:172.21.15.175:6789/0 conn(0x7f1b24166bb0 legacy=0x7f1b24167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 -- 172.21.15.181:0/1971553615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240b2060 con 0x7f1b240b3930 2020-11-16T22:31:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b21ffb700 1 -- 172.21.15.181:0/1971553615 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783800719 0 0) 0x7f1b24168390 con 0x7f1b24167ac0 2020-11-16T22:31:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b227fc700 1 --2- 172.21.15.181:0/1971553615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1b1800ba20 tx=0x7f1b18006eb0).ready entity=mon.2 client_cookie=3dcda6bc19c56aaf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b21ffb700 1 -- 172.21.15.181:0/1971553615 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180098d0 con 0x7f1b240b3930 2020-11-16T22:31:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b21ffb700 1 -- 172.21.15.181:0/1971553615 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18009a30 con 0x7f1b240b3930 2020-11-16T22:31:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b21ffb700 1 -- 172.21.15.181:0/1971553615 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18009db0 con 0x7f1b240b3930 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- 172.21.15.181:0/1971553615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 msgr2=0x7f1b24167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 --2- 172.21.15.181:0/1971553615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1b1800ba20 tx=0x7f1b18006eb0).stop 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- 172.21.15.181:0/1971553615 shutdown_connections 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 --2- 172.21.15.181:0/1971553615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b24167ea0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- 172.21.15.181:0/1971553615 >> 172.21.15.181:0/1971553615 conn(0x7f1b24060bc0 msgr2=0x7f1b240d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- 172.21.15.181:0/1971553615 shutdown_connections 2020-11-16T22:31:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 -- 172.21.15.181:0/1971553615 wait complete. 2020-11-16T22:31:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.133+0000 7f1b29830700 1 Processor -- start 2020-11-16T22:31:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- start start 2020-11-16T22:31:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 0x7f1b240d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 0x7f1b240b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240a3350 con 0x7f1b24167ac0 2020-11-16T22:31:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b6150 con 0x7f1b24166bb0 2020-11-16T22:31:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24164fa0 con 0x7f1b240b3930 2020-11-16T22:31:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40600/0 (socket says 172.21.15.181:40600) 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 -- 172.21.15.181:0/4194519963 learned_addr learned my addr 172.21.15.181:0/4194519963 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b227fc700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 0x7f1b240d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b237fe700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 0x7f1b240b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 -- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 msgr2=0x7f1b240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 0x7f1b240d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 -- 172.21.15.181:0/4194519963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 msgr2=0x7f1b240b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 0x7f1b240b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 -- 172.21.15.181:0/4194519963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24164520 con 0x7f1b240b3930 2020-11-16T22:31:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b237fe700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 0x7f1b240b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b227fc700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 0x7f1b240d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b22ffd700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f1b14001e70 tx=0x7f1b14007ae0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:09.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14007320 con 0x7f1b240b3930 2020-11-16T22:31:09.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14005530 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b24164820 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b140058b0 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b240b7fe0 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.137+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b240b6310 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.141+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1b14010030 con 0x7f1b240b3930 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.141+0000 7f1b0affd700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 0x7f1b00057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:09.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.141+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b1406d940 con 0x7f1b240b3930 2020-11-16T22:31:09.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.141+0000 7f1b227fc700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 0x7f1b00057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:09.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.141+0000 7f1b227fc700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 0x7f1b00057520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f1b1800b1b0 tx=0x7f1b18015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:09.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.145+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1b240b6310 con 0x7f1b240b3930 2020-11-16T22:31:09.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.345+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b24069eb0 con 0x7f1b240b3930 2020-11-16T22:31:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b0affd700 1 -- 172.21.15.181:0/4194519963 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f1b14043ba0 con 0x7f1b240b3930 2020-11-16T22:31:09.329 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:09.329 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:08.740058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:09.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 msgr2=0x7f1b00057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 0x7f1b00057520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f1b1800b1b0 tx=0x7f1b18015000).stop 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 msgr2=0x7f1b240d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f1b14001e70 tx=0x7f1b14007ae0).stop 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 shutdown_connections 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1b00055060 0x7f1b00057520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b24167ac0 0x7f1b240b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b240b3930 0x7f1b240d0620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 --2- 172.21.15.181:0/4194519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b24166bb0 0x7f1b240d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:09.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 >> 172.21.15.181:0/4194519963 conn(0x7f1b24060bc0 msgr2=0x7f1b241620d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:09.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 shutdown_connections 2020-11-16T22:31:09.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:09.349+0000 7f1b29830700 1 -- 172.21.15.181:0/4194519963 wait complete. 2020-11-16T22:31:09.734 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2020-11-16T22:31:10.641 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:30:53.516445+0000', 'modified': '2020-11-16T22:31:08.740058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6316, 'mds_1': 6313, 'mds_2': 6295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6295': {'gid': 6295, 'name': 'd', 'rank': 2, 'incarnation': 66, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6837/461666765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 461666765}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 461666765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6313': {'gid': 6313, 'name': 'i', 'rank': 1, 'incarnation': 64, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.175:6841/2108826019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 2108826019}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2108826019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6316': {'gid': 6316, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6841/3625693713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6839', 'nonce': 3625693713}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 3625693713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:31:10.642 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:31:10.642 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:31:10.782 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":6295,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.0527082540000001,"mdsmap_epoch":67,"osdmap_epoch":64,"osdmap_epoch_barrier":64,"uptime":21.679786277000002} 2020-11-16T22:31:10.783 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6295, "mdsmap_epoch": 67, "osdmap_epoch": 64, "osdmap_epoch_barrier": 64, "rank_uptime": 3.052708254, "state": "up:active", "uptime": 21.679786277, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:31:10.783 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:31:10.933 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":6313,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.2138580699999997,"mdsmap_epoch":67,"osdmap_epoch":64,"osdmap_epoch_barrier":64,"uptime":21.817695109999999} 2020-11-16T22:31:10.933 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6313, "mdsmap_epoch": 67, "osdmap_epoch": 64, "osdmap_epoch_barrier": 64, "rank_uptime": 4.21385807, "state": "up:active", "uptime": 21.81769511, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:31:10.934 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:31:11.074 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6316,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.564508359000001,"mdsmap_epoch":67,"osdmap_epoch":64,"osdmap_epoch_barrier":64,"uptime":21.958302068999998} 2020-11-16T22:31:11.074 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6316, "mdsmap_epoch": 67, "osdmap_epoch": 64, "osdmap_epoch_barrier": 64, "rank_uptime": 17.564508359, "state": "up:active", "uptime": 21.958302069, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:31:11.075 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:11.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 Processor -- start 2020-11-16T22:31:11.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 -- start start 2020-11-16T22:31:11.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 0x7f333c15e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0cc7d0 con 0x7f333c157a80 2020-11-16T22:31:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c15f0b0 con 0x7f333c15dfd0 2020-11-16T22:31:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f3341aa1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c15dbe0 con 0x7f333c061100 2020-11-16T22:31:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f333bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f333bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40608/0 (socket says 172.21.15.181:40608) 2020-11-16T22:31:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.213+0000 7f333bfff700 1 -- 172.21.15.181:0/3505418879 learned_addr learned my addr 172.21.15.181:0/3505418879 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:11.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f33412a0700 1 --2- 172.21.15.181:0/3505418879 >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 0x7f333c15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333bfff700 1 -- 172.21.15.181:0/3505418879 >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 msgr2=0x7f333c15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:11.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333bfff700 1 --2- 172.21.15.181:0/3505418879 >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 0x7f333c15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333bfff700 1 -- 172.21.15.181:0/3505418879 >> v1:172.21.15.175:6789/0 conn(0x7f333c157a80 legacy=0x7f333c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333bfff700 1 -- 172.21.15.181:0/3505418879 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c1611b0 con 0x7f333c061100 2020-11-16T22:31:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f33412a0700 1 --2- 172.21.15.181:0/3505418879 >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 0x7f333c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:31:11.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333bfff700 1 --2- 172.21.15.181:0/3505418879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3324006fd0 tx=0x7f3324006eb0).ready entity=mon.2 client_cookie=57c8a5640ca35b8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:11.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333b7fe700 1 -- 172.21.15.181:0/3505418879 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324009890 con 0x7f333c061100 2020-11-16T22:31:11.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333b7fe700 1 -- 172.21.15.181:0/3505418879 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33240099f0 con 0x7f333c061100 2020-11-16T22:31:11.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f333b7fe700 1 -- 172.21.15.181:0/3505418879 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324009d70 con 0x7f333c061100 2020-11-16T22:31:11.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- 172.21.15.181:0/3505418879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 msgr2=0x7f333c061580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:11.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 --2- 172.21.15.181:0/3505418879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3324006fd0 tx=0x7f3324006eb0).stop 2020-11-16T22:31:11.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- 172.21.15.181:0/3505418879 shutdown_connections 2020-11-16T22:31:11.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 --2- 172.21.15.181:0/3505418879 >> v2:172.21.15.175:3300/0 conn(0x7f333c15dfd0 0x7f333c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 --2- 172.21.15.181:0/3505418879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c061100 0x7f333c061580 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- 172.21.15.181:0/3505418879 >> 172.21.15.181:0/3505418879 conn(0x7f333c0cf630 msgr2=0x7f333c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:11.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- 172.21.15.181:0/3505418879 shutdown_connections 2020-11-16T22:31:11.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- 172.21.15.181:0/3505418879 wait complete. 2020-11-16T22:31:11.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 Processor -- start 2020-11-16T22:31:11.205 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.217+0000 7f3341aa1700 1 -- start start 2020-11-16T22:31:11.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c157a80 0x7f333c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 0x7f333c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c170610 con 0x7f333c15dfd0 2020-11-16T22:31:11.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c154b20 con 0x7f333c061100 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c170370 con 0x7f333c157a80 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32772/0 (socket says 172.21.15.181:32772) 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f333bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c157a80 0x7f333c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 -- 172.21.15.181:0/4116452745 learned_addr learned my addr 172.21.15.181:0/4116452745 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:11.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f33412a0700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 0x7f333c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 -- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c157a80 msgr2=0x7f333c153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c157a80 0x7f333c153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 -- 172.21.15.181:0/4116452745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 msgr2=0x7f333c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 0x7f333c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 -- 172.21.15.181:0/4116452745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c154d80 con 0x7f333c061100 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f33412a0700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 0x7f333c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:11.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3340a9f700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f332c00c760 tx=0x7f332c007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:11.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00e560 con 0x7f333c061100 2020-11-16T22:31:11.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c00e6c0 con 0x7f333c061100 2020-11-16T22:31:11.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00ea80 con 0x7f333c061100 2020-11-16T22:31:11.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c155020 con 0x7f333c061100 2020-11-16T22:31:11.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.221+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c155760 con 0x7f333c061100 2020-11-16T22:31:11.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f332c010030 con 0x7f333c061100 2020-11-16T22:31:11.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c044f60 con 0x7f333c061100 2020-11-16T22:31:11.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f3338ff9700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 0x7f3318057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:11.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f332c06dc60 con 0x7f333c061100 2020-11-16T22:31:11.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f333bfff700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 0x7f3318057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:11.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.225+0000 7f333bfff700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 0x7f3318057520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f333c0b4af0 tx=0x7f3324015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:11.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.233+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f332c043ec0 con 0x7f333c061100 2020-11-16T22:31:11.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.429+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f333c0b2520 con 0x7f333c061100 2020-11-16T22:31:11.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.429+0000 7f3338ff9700 1 -- 172.21.15.181:0/4116452745 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f332c0474e0 con 0x7f333c061100 2020-11-16T22:31:11.411 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:11.412 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:08.740058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 msgr2=0x7f3318057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:11.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 0x7f3318057520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f333c0b4af0 tx=0x7f3324015000).stop 2020-11-16T22:31:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 msgr2=0x7f333c0ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:11.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f332c00c760 tx=0x7f332c007ba0).stop 2020-11-16T22:31:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 shutdown_connections 2020-11-16T22:31:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3318055060 0x7f3318057520 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c15dfd0 0x7f333c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333c061100 0x7f333c0ce800 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 --2- 172.21.15.181:0/4116452745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333c157a80 0x7f333c153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 >> 172.21.15.181:0/4116452745 conn(0x7f333c0cf630 msgr2=0x7f333c0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 shutdown_connections 2020-11-16T22:31:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:11.433+0000 7f3341aa1700 1 -- 172.21.15.181:0/4116452745 wait complete. 2020-11-16T22:31:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2020-11-16T22:31:11.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:11.621 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:31:11.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:11.685 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:31:14.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:14.138 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:31:14.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:14.175 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:31:14.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:14.212 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:31:14.250 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:14.395 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:31:14.396 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:14.396 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:16.397 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:16.537 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:31:16.537 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:16.538 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:18.539 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:18.678 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:31:18.679 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:18.679 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:20.819 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:31:20.819 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:20.820 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:22.821 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:22.957 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:31:22.957 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:22.958 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:24.959 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:31:25.096 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:31:25.097 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:31:25.098 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:31:25.098 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:31:25.235 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:31:25.236 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:31:25.438 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:31:27.439 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:31:27.577 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:31:27.578 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:31:27.578 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:31:29.521 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:31:29.527 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:31:29.529 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:31:29.579 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:31:29.745 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:31:29.746 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:31:29.746 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:31:31.747 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:31:31.884 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:31:31.885 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:31:32.090 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:31:32.091 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:31:32.227 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:31:32.228 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:31:32.430 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:31:32.430 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 Processor -- start 2020-11-16T22:31:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- start start 2020-11-16T22:31:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b80622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7b8062790 0x7fa7b8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b8160e40 con 0x7fa7b815f5b0 2020-11-16T22:31:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b81609a0 con 0x7fa7b8062790 2020-11-16T22:31:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8160ba0 con 0x7fa7b80615f0 2020-11-16T22:31:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bda6d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa7b815f5b0 0x7fa7b80d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47086/0 (socket says 172.21.15.181:47086) 2020-11-16T22:31:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bda6d700 1 -- 172.21.15.181:0/1338943307 learned_addr learned my addr 172.21.15.181:0/1338943307 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 --2- 172.21.15.181:0/1338943307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b80622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7be26e700 1 --2- 172.21.15.181:0/1338943307 >> v2:172.21.15.181:3300/0 conn(0x7fa7b8062790 0x7fa7b8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bca6b700 1 -- 172.21.15.181:0/1338943307 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672508941 0 0) 0x7fa7b8160e40 con 0x7fa7b815f5b0 2020-11-16T22:31:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bca6b700 1 -- 172.21.15.181:0/1338943307 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a0003240 con 0x7fa7b815f5b0 2020-11-16T22:31:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 -- 172.21.15.181:0/1338943307 >> v2:172.21.15.181:3300/0 conn(0x7fa7b8062790 msgr2=0x7fa7b8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 --2- 172.21.15.181:0/1338943307 >> v2:172.21.15.181:3300/0 conn(0x7fa7b8062790 0x7fa7b8080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:32.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 -- 172.21.15.181:0/1338943307 >> v1:172.21.15.181:6789/0 conn(0x7fa7b815f5b0 legacy=0x7fa7b80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:32.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 -- 172.21.15.181:0/1338943307 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8060970 con 0x7fa7b80615f0 2020-11-16T22:31:32.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bd26c700 1 --2- 172.21.15.181:0/1338943307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b80622a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fa7a800b9a0 tx=0x7fa7a8006df0).ready entity=mon.2 client_cookie=742015f07c822861 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bca6b700 1 -- 172.21.15.181:0/1338943307 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8009680 con 0x7fa7b80615f0 2020-11-16T22:31:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bca6b700 1 -- 172.21.15.181:0/1338943307 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80097e0 con 0x7fa7b80615f0 2020-11-16T22:31:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bca6b700 1 -- 172.21.15.181:0/1338943307 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8009b60 con 0x7fa7b80615f0 2020-11-16T22:31:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/1338943307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 msgr2=0x7fa7b80622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/1338943307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b80622a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fa7a800b9a0 tx=0x7fa7a8006df0).stop 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/1338943307 shutdown_connections 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/1338943307 >> v2:172.21.15.181:3300/0 conn(0x7fa7b8062790 0x7fa7b8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/1338943307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b80622a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/1338943307 >> 172.21.15.181:0/1338943307 conn(0x7fa7b80d69a0 msgr2=0x7fa7b80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.580+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/1338943307 shutdown_connections 2020-11-16T22:31:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/1338943307 wait complete. 2020-11-16T22:31:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 Processor -- start 2020-11-16T22:31:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 -- start start 2020-11-16T22:31:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b816f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8062790 0x7fa7b80d3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b816f730 con 0x7fa7b8062790 2020-11-16T22:31:32.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b816f930 con 0x7fa7b815f5b0 2020-11-16T22:31:32.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bfcd1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b816fbd0 con 0x7fa7b80615f0 2020-11-16T22:31:32.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7be26e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bda6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b816f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7be26e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32792/0 (socket says 172.21.15.181:32792) 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7be26e700 1 -- 172.21.15.181:0/3430349654 learned_addr learned my addr 172.21.15.181:0/3430349654 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.584+0000 7fa7bd26c700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8062790 0x7fa7b80d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 -- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 msgr2=0x7fa7b816f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b816f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:32.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 -- 172.21.15.181:0/3430349654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8062790 msgr2=0x7fa7b80d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:32.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8062790 0x7fa7b80d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:32.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 -- 172.21.15.181:0/3430349654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b816fe70 con 0x7fa7b815f5b0 2020-11-16T22:31:32.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7be26e700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fa7b000ef50 tx=0x7fa7b00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:33.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0009690 con 0x7fa7b815f5b0 2020-11-16T22:31:33.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000cbf0 con 0x7fa7b815f5b0 2020-11-16T22:31:33.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8170110 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00123f0 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b80ba6a0 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b8060b50 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa7b0022070 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 0x7fa790057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7b006e590 con 0x7fa7b815f5b0 2020-11-16T22:31:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.588+0000 7fa7bda6d700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 0x7fa790057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.592+0000 7fa7bda6d700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 0x7fa790057520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa7b4001320 tx=0x7fa7b4002760).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.596+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa7b00447f0 con 0x7fa7b815f5b0 2020-11-16T22:31:33.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7b8069eb0 con 0x7fa7b815f5b0 2020-11-16T22:31:33.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7a5ffb700 1 -- 172.21.15.181:0/3430349654 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fa7b0016070 con 0x7fa7b815f5b0 2020-11-16T22:31:33.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 msgr2=0x7fa790057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:33.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 0x7fa790057520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa7b4001320 tx=0x7fa7b4002760).stop 2020-11-16T22:31:33.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 msgr2=0x7fa7b80d3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:33.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.800+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fa7b000ef50 tx=0x7fa7b00091d0).stop 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 shutdown_connections 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa790055060 0x7fa790057520 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b815f5b0 0x7fa7b80d3510 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b80615f0 0x7fa7b816f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 --2- 172.21.15.181:0/3430349654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7b8062790 0x7fa7b80d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:33.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 >> 172.21.15.181:0/3430349654 conn(0x7fa7b80d69a0 msgr2=0x7fa7b815c080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:33.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 shutdown_connections 2020-11-16T22:31:33.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:32.804+0000 7fa7bfcd1700 1 -- 172.21.15.181:0/3430349654 wait complete. 2020-11-16T22:31:33.392 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 69 2020-11-16T22:31:33.392 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:33.392 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:29.844224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:33.798 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:31:33.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 Processor -- start 2020-11-16T22:31:33.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 -- start start 2020-11-16T22:31:33.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:33.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8c081456c0 0x7f8c08145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:33.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080ce850 con 0x7f8c080b5d60 2020-11-16T22:31:33.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0815f8f0 con 0x7f8c081456c0 2020-11-16T22:31:33.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0ffeb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815fb90 con 0x7f8c080cdf80 2020-11-16T22:31:33.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:33.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40638/0 (socket says 172.21.15.181:40638) 2020-11-16T22:31:33.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 -- 172.21.15.181:0/3469565326 learned_addr learned my addr 172.21.15.181:0/3469565326 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0e588700 1 --2- 172.21.15.181:0/3469565326 >> v2:172.21.15.175:3300/0 conn(0x7f8c081456c0 0x7f8c08145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 -- 172.21.15.181:0/3469565326 >> v2:172.21.15.175:3300/0 conn(0x7f8c081456c0 msgr2=0x7f8c08145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/3469565326 >> v2:172.21.15.175:3300/0 conn(0x7f8c081456c0 0x7f8c08145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 -- 172.21.15.181:0/3469565326 >> v1:172.21.15.175:6789/0 conn(0x7f8c080b5d60 legacy=0x7f8c080b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.952+0000 7f8c0dd87700 1 -- 172.21.15.181:0/3469565326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0815fe30 con 0x7f8c080cdf80 2020-11-16T22:31:34.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/3469565326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f8bf8009d40 tx=0x7f8bf8009250).ready entity=mon.2 client_cookie=c9ef1be1ba65d34e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:34.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0cd85700 1 -- 172.21.15.181:0/3469565326 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800bc40 con 0x7f8c080cdf80 2020-11-16T22:31:34.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0cd85700 1 -- 172.21.15.181:0/3469565326 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf800bda0 con 0x7f8c080cdf80 2020-11-16T22:31:34.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0cd85700 1 -- 172.21.15.181:0/3469565326 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8005c60 con 0x7f8c080cdf80 2020-11-16T22:31:34.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3469565326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 msgr2=0x7f8c080ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3469565326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f8bf8009d40 tx=0x7f8bf8009250).stop 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3469565326 shutdown_connections 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3469565326 >> v2:172.21.15.175:3300/0 conn(0x7f8c081456c0 0x7f8c08145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3469565326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c080cdf80 0x7f8c080ce360 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3469565326 >> 172.21.15.181:0/3469565326 conn(0x7f8c08060b80 msgr2=0x7f8c080d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3469565326 shutdown_connections 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3469565326 wait complete. 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 Processor -- start 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- start start 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c080b5d60 0x7f8c08140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c080c7300 0x7f8c08142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08141370 con 0x7f8c080b5d60 2020-11-16T22:31:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080cdb90 con 0x7f8c080c7300 2020-11-16T22:31:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0ffeb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081415b0 con 0x7f8c081456c0 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40646/0 (socket says 172.21.15.181:40646) 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0d586700 1 -- 172.21.15.181:0/3211161432 learned_addr learned my addr 172.21.15.181:0/3211161432 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0e588700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c080c7300 0x7f8c08142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.956+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c080b5d60 0x7f8c08140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 -- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c080c7300 msgr2=0x7f8c08142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c080c7300 0x7f8c08142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 -- 172.21.15.181:0/3211161432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c080b5d60 msgr2=0x7f8c08140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c080b5d60 0x7f8c08140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 -- 172.21.15.181:0/3211161432 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08141850 con 0x7f8c081456c0 2020-11-16T22:31:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0d586700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f8bfc00c760 tx=0x7f8bfc007c00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:34.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c05ffb700 1 -- 172.21.15.181:0/3211161432 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00f470 con 0x7f8c081456c0 2020-11-16T22:31:34.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c05ffb700 1 -- 172.21.15.181:0/3211161432 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00f5f0 con 0x7f8c081456c0 2020-11-16T22:31:34.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c08141af0 con 0x7f8c081456c0 2020-11-16T22:31:34.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c05ffb700 1 -- 172.21.15.181:0/3211161432 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00f9b0 con 0x7f8c081456c0 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08063d80 con 0x7f8c081456c0 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.960+0000 7f8c05ffb700 1 -- 172.21.15.181:0/3211161432 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8bfc019030 con 0x7f8c081456c0 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c05ffb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 0x7f8bdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c05ffb700 1 -- 172.21.15.181:0/3211161432 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8bfc06d040 con 0x7f8c081456c0 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 0x7f8bdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 0x7f8bdc057520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f8c08066850 tx=0x7f8bf8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:34.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 Processor -- start 2020-11-16T22:31:34.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- start start 2020-11-16T22:31:34.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 0x1ecab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 0x1ec1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d4f4b0 con 0x1ca57b0 2020-11-16T22:31:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e03930 con 0x1ec83e0 2020-11-16T22:31:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cc36f0 con 0x1c37bb0 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0e588700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 0x1ec1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40652/0 (socket says 172.21.15.181:40652) 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0e588700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 0x1ec1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32814/0 (socket says 172.21.15.181:32814) 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 -- 172.21.15.181:0/2682492186 learned_addr learned my addr 172.21.15.181:0/2682492186 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 0x1ecab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 -- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 msgr2=0x1ec1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 0x1ec1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 -- 172.21.15.181:0/2682492186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 msgr2=0x1ecab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 0x1ecab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 -- 172.21.15.181:0/2682492186 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cbe5a0 con 0x1c37bb0 2020-11-16T22:31:34.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 0x1ecab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:34.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c0d586700 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8bfc046f40 tx=0x7f8bfc012000).ready entity=mon.2 client_cookie=3a1a6efcd2704b08 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:34.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8beeffd700 1 -- 172.21.15.181:0/2682492186 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00fd60 con 0x1c37bb0 2020-11-16T22:31:34.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8beeffd700 1 -- 172.21.15.181:0/2682492186 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc046500 con 0x1c37bb0 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8beeffd700 1 -- 172.21.15.181:0/2682492186 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc042e00 con 0x1c37bb0 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 msgr2=0x1ec8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8bfc046f40 tx=0x7f8bfc012000).stop 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.964+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/2682492186 shutdown_connections 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec83e0 0x1ec1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ca57b0 0x1ecab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/2682492186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c37bb0 0x1ec8a80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:34.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/2682492186 >> 172.21.15.181:0/2682492186 conn(0x1e73e40 msgr2=0x1ec47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:34.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/2682492186 shutdown_connections 2020-11-16T22:31:34.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/2682492186 wait complete. 2020-11-16T22:31:34.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- start start 2020-11-16T22:31:34.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 Processor -- start 2020-11-16T22:31:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 0x1ec4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec83e0 0x1f18680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 0x1f18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df06e0 con 0x1ec83e0 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df0820 con 0x1ec2f20 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df1a90 con 0x1ca57b0 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 0x1ec4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 10 client.0 ll_register_callbacks cb 0x1de6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 0x1ec4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40658/0 (socket says 172.21.15.181:40658) 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0dd87700 1 -- 172.21.15.181:0/475394983 learned_addr learned my addr 172.21.15.181:0/475394983 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 0x1f18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0d586700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec83e0 0x1f18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 -- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 msgr2=0x1ec4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 0x1ec4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 -- 172.21.15.181:0/475394983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec83e0 msgr2=0x1f18680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:35.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec83e0 0x1f18680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 -- 172.21.15.181:0/475394983 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1de1280 con 0x1ec2f20 2020-11-16T22:31:35.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c0e588700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 0x1f18b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f8c00013940 tx=0x7f8c00008040).ready entity=mon.1 client_cookie=29c3e4d4d0b99110 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:35.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8bdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000157c0 con 0x1ec2f20 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c00015920 con 0x1ec2f20 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.968+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ea46c0 con 0x1ec2f20 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c00015d30 con 0x1ec2f20 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 10 client.6502 fetch_fsmap learned FSMap version 69 2020-11-16T22:31:35.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x1c16fd0 con 0x1ec2f20 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ea46c0 con 0x1ec2f20 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 69) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f8c00026020 con 0x1ec2f20 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 10 client.6502 fetch_fsmap finished waiting for FSMap version 69 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 10 client.6502 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 20 client.6502 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:35.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1f27730 0x1ef8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 4 client.6502 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:31:35.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1ce1c50 con 0x1f27730 2020-11-16T22:31:35.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1f27730 0x1ef8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 10 client.6502 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1f27730 0x1ef8540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=46415b18c0832d3e server_cookie=89d4dd7ac4e00dd5 in_seq=0 out_seq=0 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 1 -- 172.21.15.181:0/475394983 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1ce1c50 con 0x1f27730 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 10 client.6502 handle_command_reply: tid=0 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 client.6502 shutdown 2020-11-16T22:31:35.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 msgr2=0x1f18b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 0x1f18b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f8c00013940 tx=0x7f8c00008040).stop 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 shutdown_connections 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1f27730 0x1ef8540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ec2f20 0x1f18b70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8bdbfff700 0 client.6502 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca57b0 0x1ec4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/475394983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec83e0 0x1f18680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 >> 172.21.15.181:0/475394983 conn(0x1e73e40 msgr2=0x1ec71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 shutdown_connections 2020-11-16T22:31:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/475394983 wait complete. 2020-11-16T22:31:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.972+0000 7f8c1ea7c740 20 client.6502 trim_cache size 0 max 16384 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 Processor -- start 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 -- start start 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e73e40 0x1efdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 0x1f176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dc8e40 con 0x1e73e40 2020-11-16T22:31:35.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ec7e60 con 0x1ca57b0 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ec29b0 con 0x1ec2f20 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0e588700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 0x1f176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32824/0 (socket says 172.21.15.181:32824) 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0d586700 1 -- 172.21.15.181:0/1548642447 learned_addr learned my addr 172.21.15.181:0/1548642447 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:35.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e73e40 0x1efdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0e588700 1 -- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 msgr2=0x1ec9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0e588700 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0e588700 1 -- 172.21.15.181:0/1548642447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e73e40 msgr2=0x1efdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.992+0000 7f8c0e588700 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e73e40 0x1efdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c0e588700 1 -- 172.21.15.181:0/1548642447 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ec4c80 con 0x1ec2f20 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c0e588700 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 0x1f176c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f8c00027050 tx=0x7f8c00021000).ready entity=mon.2 client_cookie=720eac68fa794e97 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8bef7fe700 1 -- 172.21.15.181:0/1548642447 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000de60 con 0x1ec2f20 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8bef7fe700 1 -- 172.21.15.181:0/1548642447 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c000157c0 con 0x1ec2f20 2020-11-16T22:31:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8bef7fe700 1 -- 172.21.15.181:0/1548642447 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00000d90 con 0x1ec2f20 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 msgr2=0x1f176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 0x1f176c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f8c00027050 tx=0x7f8c00021000).stop 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1548642447 shutdown_connections 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ec2f20 0x1f176c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e73e40 0x1efdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1548642447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1548642447 >> 172.21.15.181:0/1548642447 conn(0x7f8bd4002320 msgr2=0x1f126a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:36.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1548642447 shutdown_connections 2020-11-16T22:31:36.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1548642447 wait complete. 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 -- start start 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:33.996+0000 7f8c1ea7c740 1 Processor -- start 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 0x1f18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec2f20 0x1f16210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f16700 con 0x1ec2f20 2020-11-16T22:31:36.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f7b320 con 0x1ca57b0 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f7b5c0 con 0x1e73e40 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 10 client.0 ll_register_callbacks cb 0x1ec8df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 0x1f18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0dd87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32830/0 (socket says 172.21.15.181:32830) 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0dd87700 1 -- 172.21.15.181:0/1281420112 learned_addr learned my addr 172.21.15.181:0/1281420112 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:36.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0e588700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec2f20 0x1f16210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:36.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 -- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 msgr2=0x1ec76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:36.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 -- 172.21.15.181:0/1281420112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec2f20 msgr2=0x1f16210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:36.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec2f20 0x1f16210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 -- 172.21.15.181:0/1281420112 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd40081d0 con 0x1e73e40 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0d586700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 0x1f18820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f8bfc028040 tx=0x7f8bfc01e000).ready entity=mon.2 client_cookie=757383ce0fd84444 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc046770 con 0x1e73e40 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8bfc00fd20 con 0x1e73e40 2020-11-16T22:31:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bfc046500 con 0x1e73e40 2020-11-16T22:31:36.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc046180 con 0x1e73e40 2020-11-16T22:31:36.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 10 client.6032 fetch_fsmap learned FSMap version 69 2020-11-16T22:31:36.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x1c55720 con 0x1e73e40 2020-11-16T22:31:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc046500 con 0x1e73e40 2020-11-16T22:31:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 69) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f8bfc06b810 con 0x1e73e40 2020-11-16T22:31:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 10 client.6032 fetch_fsmap finished waiting for FSMap version 69 2020-11-16T22:31:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 10 client.6032 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 20 client.6032 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1ec83e0 0x1f1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 4 client.6032 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1ce8f50 con 0x1ec83e0 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.000+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1ec83e0 0x1f1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.004+0000 7f8bdbfff700 10 client.6032 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.004+0000 7f8c0dd87700 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1ec83e0 0x1f1acf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=73b598a5df9cd0eb server_cookie=5777d65f1b57f4c7 in_seq=0 out_seq=0 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8bdbfff700 1 -- 172.21.15.181:0/1281420112 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1ce8f50 con 0x1ec83e0 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8bdbfff700 10 client.6032 handle_command_reply: tid=0 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 client.6032 shutdown 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 msgr2=0x1f18820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 0x1f18820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f8bfc028040 tx=0x7f8bfc01e000).stop 2020-11-16T22:31:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 shutdown_connections 2020-11-16T22:31:36.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x1ec83e0 0x1f1acf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ec2f20 0x1f16210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8bdbfff700 0 client.6032 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:36.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca57b0 0x1ec76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 --2- 172.21.15.181:0/1281420112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e73e40 0x1f18820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 >> 172.21.15.181:0/1281420112 conn(0x7f8bd4002320 msgr2=0x1f126a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 shutdown_connections 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 1 -- 172.21.15.181:0/1281420112 wait complete. 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c1ea7c740 20 client.6032 trim_cache size 0 max 16384 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 msgr2=0x7f8bdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:36.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 0x7f8bdc057520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f8c08066850 tx=0x7f8bf8015000).stop 2020-11-16T22:31:36.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 msgr2=0x7f8c080cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:36.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f8bfc00c760 tx=0x7f8bfc007c00).stop 2020-11-16T22:31:36.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 shutdown_connections 2020-11-16T22:31:36.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8bdc055060 0x7f8bdc057520 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:36.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c080c7300 0x7f8c08142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c080b5d60 0x7f8c08140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 --2- 172.21.15.181:0/3211161432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c081456c0 0x7f8c080cd6a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 >> 172.21.15.181:0/3211161432 conn(0x7f8c08060b80 msgr2=0x7f8c081437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 shutdown_connections 2020-11-16T22:31:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:34.028+0000 7f8c0ffeb700 1 -- 172.21.15.181:0/3211161432 wait complete. 2020-11-16T22:31:37.011 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:31:37.011 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:31:37.011 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:31:37.011 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:31:37.012 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:37.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 Processor -- start 2020-11-16T22:31:37.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 -- start start 2020-11-16T22:31:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180c8db0 con 0x7f3a18161860 2020-11-16T22:31:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a18165030 con 0x7f3a18161480 2020-11-16T22:31:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a17fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180c8330 con 0x7f3a18062960 2020-11-16T22:31:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40680/0 (socket says 172.21.15.181:40680) 2020-11-16T22:31:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 -- 172.21.15.181:0/964315595 learned_addr learned my addr 172.21.15.181:0/964315595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a16ffd700 1 --1- 172.21.15.181:0/964315595 >> v1:172.21.15.181:6789/0 conn(0x7f3a18161480 0x7f3a18164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47144/0 (socket says 172.21.15.181:47144) 2020-11-16T22:31:37.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 -- 172.21.15.181:0/964315595 >> v1:172.21.15.181:6789/0 conn(0x7f3a18161480 legacy=0x7f3a18164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 -- 172.21.15.181:0/964315595 >> v1:172.21.15.175:6789/0 conn(0x7f3a18161860 legacy=0x7f3a18062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 -- 172.21.15.181:0/964315595 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180c85d0 con 0x7f3a18062960 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a177fe700 1 --2- 172.21.15.181:0/964315595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f3a10009d80 tx=0x7f3a10009290).ready entity=mon.2 client_cookie=f7e487e0199ab729 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a15ffb700 1 -- 172.21.15.181:0/964315595 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000bc80 con 0x7f3a18062960 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a15ffb700 1 -- 172.21.15.181:0/964315595 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000bde0 con 0x7f3a18062960 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.163+0000 7f3a15ffb700 1 -- 172.21.15.181:0/964315595 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10011610 con 0x7f3a18062960 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/964315595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 msgr2=0x7f3a180644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 --2- 172.21.15.181:0/964315595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f3a10009d80 tx=0x7f3a10009290).stop 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/964315595 shutdown_connections 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 --2- 172.21.15.181:0/964315595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18062960 0x7f3a180644e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/964315595 >> 172.21.15.181:0/964315595 conn(0x7f3a180d2dc0 msgr2=0x7f3a180ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/964315595 shutdown_connections 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/964315595 wait complete. 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 Processor -- start 2020-11-16T22:31:37.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- start start 2020-11-16T22:31:37.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a18062960 0x7f3a1815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:37.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18161480 0x7f3a180ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:37.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815e240 con 0x7f3a18062960 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815d7c0 con 0x7f3a18161480 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815da60 con 0x7f3a18161860 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a167fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18161480 0x7f3a180ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:37.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40690/0 (socket says 172.21.15.181:40690) 2020-11-16T22:31:37.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 -- 172.21.15.181:0/2795201131 learned_addr learned my addr 172.21.15.181:0/2795201131 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:37.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a16ffd700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a18062960 0x7f3a1815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:37.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 -- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18161480 msgr2=0x7f3a180ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18161480 0x7f3a180ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 -- 172.21.15.181:0/2795201131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a18062960 msgr2=0x7f3a1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a18062960 0x7f3a1815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 -- 172.21.15.181:0/2795201131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1815dd00 con 0x7f3a18161860 2020-11-16T22:31:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a177fe700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f3a10009cd0 tx=0x7f3a10016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10011e30 con 0x7f3a18161860 2020-11-16T22:31:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10010ae0 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1815dfa0 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10010e80 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.167+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180cd460 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a180cd6b0 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3a10017070 con 0x7f3a18161860 2020-11-16T22:31:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a0effd700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 0x7f39ec057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:37.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a16ffd700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 0x7f39ec057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:37.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a10070a30 con 0x7f3a18161860 2020-11-16T22:31:37.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.171+0000 7f3a16ffd700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 0x7f39ec057550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f3a08007270 tx=0x7f3a08007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.179+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3a1004a630 con 0x7f3a18161860 2020-11-16T22:31:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a18044f60 con 0x7f3a18161860 2020-11-16T22:31:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a0effd700 1 -- 172.21.15.181:0/2795201131 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f3a10015070 con 0x7f3a18161860 2020-11-16T22:31:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 msgr2=0x7f39ec057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 0x7f39ec057550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f3a08007270 tx=0x7f3a08007b80).stop 2020-11-16T22:31:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 msgr2=0x7f3a180cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f3a10009cd0 tx=0x7f3a10016000).stop 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 shutdown_connections 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f39ec055090 0x7f39ec057550 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a18161860 0x7f3a180cddc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a18062960 0x7f3a1815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 --2- 172.21.15.181:0/2795201131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18161480 0x7f3a180ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:37.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 >> 172.21.15.181:0/2795201131 conn(0x7f3a180d2dc0 msgr2=0x7f3a18062250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:37.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 shutdown_connections 2020-11-16T22:31:38.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:37.379+0000 7f3a17fff700 1 -- 172.21.15.181:0/2795201131 wait complete. 2020-11-16T22:31:38.174 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 70 2020-11-16T22:31:38.174 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:38.175 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:34.879851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:38.378 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i flush journal 2020-11-16T22:31:38.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.527+0000 7fa1ee97a700 1 Processor -- start 2020-11-16T22:31:38.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.527+0000 7fa1ee97a700 1 -- start start 2020-11-16T22:31:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee97a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1e815b810 0x7fa1e815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee97a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1e815cc00 0x7fa1e815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee97a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e80659d0 con 0x7fa1e815fbe0 2020-11-16T22:31:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee97a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e815d770 con 0x7fa1e815cc00 2020-11-16T22:31:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee97a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e815da10 con 0x7fa1e815b810 2020-11-16T22:31:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1e815b810 0x7fa1e815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1e815b810 0x7fa1e815f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32854/0 (socket says 172.21.15.181:32854) 2020-11-16T22:31:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 -- 172.21.15.181:0/1545366703 learned_addr learned my addr 172.21.15.181:0/1545366703 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ee179700 1 --2- 172.21.15.181:0/1545366703 >> v2:172.21.15.175:3300/0 conn(0x7fa1e815cc00 0x7fa1e815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:38.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ec976700 1 -- 172.21.15.181:0/1545366703 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734268417 0 0) 0x7fa1e80659d0 con 0x7fa1e815fbe0 2020-11-16T22:31:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 -- 172.21.15.181:0/1545366703 >> v2:172.21.15.175:3300/0 conn(0x7fa1e815cc00 msgr2=0x7fa1e815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 --2- 172.21.15.181:0/1545366703 >> v2:172.21.15.175:3300/0 conn(0x7fa1e815cc00 0x7fa1e815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 -- 172.21.15.181:0/1545366703 >> v1:172.21.15.175:6789/0 conn(0x7fa1e815fbe0 legacy=0x7fa1e815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 -- 172.21.15.181:0/1545366703 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e815dcb0 con 0x7fa1e815b810 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ed978700 1 --2- 172.21.15.181:0/1545366703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e815f6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fa1e8066500 tx=0x7fa1e4006eb0).ready entity=mon.1 client_cookie=4431e47f3e5a5642 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ec976700 1 -- 172.21.15.181:0/1545366703 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4009810 con 0x7fa1e815b810 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ec976700 1 -- 172.21.15.181:0/1545366703 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4009970 con 0x7fa1e815b810 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.531+0000 7fa1ec976700 1 -- 172.21.15.181:0/1545366703 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4009cf0 con 0x7fa1e815b810 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- 172.21.15.181:0/1545366703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 msgr2=0x7fa1e815f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/1545366703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e815f6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fa1e8066500 tx=0x7fa1e4006eb0).stop 2020-11-16T22:31:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- 172.21.15.181:0/1545366703 shutdown_connections 2020-11-16T22:31:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/1545366703 >> v2:172.21.15.175:3300/0 conn(0x7fa1e815cc00 0x7fa1e815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/1545366703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e815f6f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- 172.21.15.181:0/1545366703 >> 172.21.15.181:0/1545366703 conn(0x7fa1e80d2dc0 msgr2=0x7fa1e80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- 172.21.15.181:0/1545366703 shutdown_connections 2020-11-16T22:31:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- 172.21.15.181:0/1545366703 wait complete. 2020-11-16T22:31:38.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 Processor -- start 2020-11-16T22:31:38.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.535+0000 7fa1ee97a700 1 -- start start 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e815cc00 0x7fa1e80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e815fbe0 0x7fa1e80ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8147be0 con 0x7fa1e815fbe0 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e816d270 con 0x7fa1e815b810 2020-11-16T22:31:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e816d510 con 0x7fa1e815cc00 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed177700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e815cc00 0x7fa1e80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32858/0 (socket says 172.21.15.181:32858) 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 -- 172.21.15.181:0/2320415624 learned_addr learned my addr 172.21.15.181:0/2320415624 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee179700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e815fbe0 0x7fa1e80ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 -- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e815cc00 msgr2=0x7fa1e80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:38.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e815cc00 0x7fa1e80ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 -- 172.21.15.181:0/2320415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e815fbe0 msgr2=0x7fa1e80ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:39.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e815fbe0 0x7fa1e80ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 -- 172.21.15.181:0/2320415624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e816d7b0 con 0x7fa1e815b810 2020-11-16T22:31:39.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fa1e4006ef0 tx=0x7fa1e4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:39.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1d5ffb700 1 -- 172.21.15.181:0/2320415624 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011dd0 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1d5ffb700 1 -- 172.21.15.181:0/2320415624 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e401e460 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e816da50 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1d5ffb700 1 -- 172.21.15.181:0/2320415624 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e401e800 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.539+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8148ac0 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.543+0000 7fa1d5ffb700 1 -- 172.21.15.181:0/2320415624 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa1e4025030 con 0x7fa1e815b810 2020-11-16T22:31:39.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.543+0000 7fa1d5ffb700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 0x7fa1c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.543+0000 7fa1ed177700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 0x7fa1c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.543+0000 7fa1d5ffb700 1 -- 172.21.15.181:0/2320415624 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1e406f4c0 con 0x7fa1e815b810 2020-11-16T22:31:39.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.543+0000 7fa1ed177700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 0x7fa1c0057520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fa1e80b1480 tx=0x7fa1d8007970).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:39.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 Processor -- start 2020-11-16T22:31:39.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 -- start start 2020-11-16T22:31:39.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14adfd0 0x14a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13394b0 con 0x1221bb0 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13ed930 con 0x128f7b0 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12ad6f0 con 0x14adfd0 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32866/0 (socket says 172.21.15.181:32866) 2020-11-16T22:31:39.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 -- 172.21.15.181:0/2129885294 learned_addr learned my addr 172.21.15.181:0/2129885294 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:39.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ee179700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14adfd0 0x14a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed177700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 -- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14adfd0 msgr2=0x14a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:39.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14adfd0 0x14a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 -- 172.21.15.181:0/2129885294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 msgr2=0x14ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:39.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 -- 172.21.15.181:0/2129885294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12a85a0 con 0x128f7b0 2020-11-16T22:31:39.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed177700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:39.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1ed978700 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fa1e4049150 tx=0x7fa1e4011000).ready entity=mon.1 client_cookie=6cc8922c94c8be31 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:39.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1c6ffd700 1 -- 172.21.15.181:0/2129885294 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4047780 con 0x128f7b0 2020-11-16T22:31:39.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1c6ffd700 1 -- 172.21.15.181:0/2129885294 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e406f0e0 con 0x128f7b0 2020-11-16T22:31:39.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.547+0000 7fa1c6ffd700 1 -- 172.21.15.181:0/2129885294 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40488d0 con 0x128f7b0 2020-11-16T22:31:39.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 msgr2=0x14b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:39.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fa1e4049150 tx=0x7fa1e4011000).stop 2020-11-16T22:31:39.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/2129885294 shutdown_connections 2020-11-16T22:31:39.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14adfd0 0x14a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14b0720 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/2129885294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/2129885294 >> 172.21.15.181:0/2129885294 conn(0x145de40 msgr2=0x14aa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:39.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/2129885294 shutdown_connections 2020-11-16T22:31:39.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/2129885294 wait complete. 2020-11-16T22:31:39.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- start start 2020-11-16T22:31:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 Processor -- start 2020-11-16T22:31:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 0x14e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13da6e0 con 0x14adfd0 2020-11-16T22:31:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13da820 con 0x128f7b0 2020-11-16T22:31:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13dba90 con 0x1221bb0 2020-11-16T22:31:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 10 client.0 ll_register_callbacks cb 0x13d0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40714/0 (socket says 172.21.15.181:40714) 2020-11-16T22:31:39.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 -- 172.21.15.181:0/4064289798 learned_addr learned my addr 172.21.15.181:0/4064289798 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:39.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ee179700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 0x14e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:39.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 -- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 msgr2=0x14db5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:39.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14db5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 -- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 msgr2=0x14e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:39.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 0x14e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 -- 172.21.15.181:0/4064289798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x13cb280 con 0x1221bb0 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ee179700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 0x14e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1ed978700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa1e4047730 tx=0x7fa1e406f000).ready entity=mon.2 client_cookie=7219ee05404d5b60 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4047850 con 0x1221bb0 2020-11-16T22:31:39.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1e40453e0 con 0x1221bb0 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x148e6c0 con 0x1221bb0 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e4048550 con 0x1221bb0 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 10 client.6047 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x1200fd0 con 0x1221bb0 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x148e6c0 con 0x1221bb0 2020-11-16T22:31:39.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fa1e4006030 con 0x1221bb0 2020-11-16T22:31:40.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 10 client.6047 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:40.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 10 client.6047 resolve_mds: resolved ID 'i' to GID 6313 2020-11-16T22:31:40.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 20 client.6047 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14fec40 0x14afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 4 client.6047 mds_command: new command op to 6313 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.551+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 --> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x12cbc50 con 0x14fec40 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1ed177700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14fec40 0x14afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1bbfff700 10 client.6047 ms_handle_connect on v2:172.21.15.175:6840/2108826019 2020-11-16T22:31:40.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1ed177700 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14fec40 0x14afa70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=1e054772e116e0b0 server_cookie=ad7666e3bae2e761 in_seq=0 out_seq=0 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1bbfff700 1 -- 172.21.15.181:0/4064289798 <== mds.1 v2:172.21.15.175:6840/2108826019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x12cbc50 con 0x14fec40 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1bbfff700 10 client.6047 handle_command_reply: tid=0 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 client.6047 shutdown 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 msgr2=0x14debe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa1e4047730 tx=0x7fa1e406f000).stop 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 shutdown_connections 2020-11-16T22:31:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14fec40 0x14afa70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x14adfd0 0x14e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1221bb0 0x14debe0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/4064289798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14db5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1bbfff700 0 client.6047 ms_handle_reset on v2:172.21.15.175:6840/2108826019 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 >> 172.21.15.181:0/4064289798 conn(0x145de40 msgr2=0x14e38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 shutdown_connections 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 1 -- 172.21.15.181:0/4064289798 wait complete. 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.555+0000 7fa1fe66d740 20 client.6047 trim_cache size 0 max 16384 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.571+0000 7fa1fe66d740 1 Processor -- start 2020-11-16T22:31:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.571+0000 7fa1fe66d740 1 -- start start 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13b2e40 con 0x1221bb0 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14aeaf0 con 0x128f7b0 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14f8d60 con 0x145de40 2020-11-16T22:31:40.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ee179700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40724/0 (socket says 172.21.15.181:40724) 2020-11-16T22:31:40.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 -- 172.21.15.181:0/253339 learned_addr learned my addr 172.21.15.181:0/253339 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:40.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed177700 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 -- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 msgr2=0x14a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 -- 172.21.15.181:0/253339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 msgr2=0x14ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 -- 172.21.15.181:0/253339 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b40081d0 con 0x145de40 2020-11-16T22:31:40.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1ed978700 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa1e4049310 tx=0x7fa1e4037000).ready entity=mon.2 client_cookie=c6208e88ee5caa52 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:40.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1c77fe700 1 -- 172.21.15.181:0/253339 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4006580 con 0x145de40 2020-11-16T22:31:40.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1c77fe700 1 -- 172.21.15.181:0/253339 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4048d40 con 0x145de40 2020-11-16T22:31:40.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1c77fe700 1 -- 172.21.15.181:0/253339 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40453e0 con 0x145de40 2020-11-16T22:31:40.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 msgr2=0x14f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:40.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa1e4049310 tx=0x7fa1e4037000).stop 2020-11-16T22:31:40.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- 172.21.15.181:0/253339 shutdown_connections 2020-11-16T22:31:40.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x14a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f8400 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/253339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x14ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- 172.21.15.181:0/253339 >> 172.21.15.181:0/253339 conn(0x7fa1b4002320 msgr2=0x14e0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- 172.21.15.181:0/253339 shutdown_connections 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- 172.21.15.181:0/253339 wait complete. 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 -- start start 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.575+0000 7fa1fe66d740 1 Processor -- start 2020-11-16T22:31:40.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x1502ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x1562d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f7990 con 0x1221bb0 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x150ba80 con 0x128f7b0 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x150bd20 con 0x145de40 2020-11-16T22:31:40.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 10 client.0 ll_register_callbacks cb 0x14ae850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:40.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x1562d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40730/0 (socket says 172.21.15.181:40730) 2020-11-16T22:31:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x1562d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32886/0 (socket says 172.21.15.181:32886) 2020-11-16T22:31:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 -- 172.21.15.181:0/3786201878 learned_addr learned my addr 172.21.15.181:0/3786201878 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed978700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x1502ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 -- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 msgr2=0x1562d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x1562d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 -- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 msgr2=0x1502ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x1502ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 -- 172.21.15.181:0/3786201878 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x123f720 con 0x145de40 2020-11-16T22:31:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed978700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x1502ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ee179700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa1e0017c40 tx=0x7fa1e0008040).ready entity=mon.2 client_cookie=22bf7dbc88b274ee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e001c070 con 0x145de40 2020-11-16T22:31:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1e00159a0 con 0x145de40 2020-11-16T22:31:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1e406f0e0 con 0x145de40 2020-11-16T22:31:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e0015d70 con 0x145de40 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 10 client.6053 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x124ae00 con 0x145de40 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e406f0e0 con 0x145de40 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fa1e00133a0 con 0x145de40 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 10 client.6053 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 10 client.6053 resolve_mds: resolved ID 'i' to GID 6313 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 20 client.6053 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14adfd0 0x14f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 4 client.6053 mds_command: new command op to 6313 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 --> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x12d2f50 con 0x14adfd0 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed978700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14adfd0 0x14f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1bbfff700 10 client.6053 ms_handle_connect on v2:172.21.15.175:6840/2108826019 2020-11-16T22:31:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.579+0000 7fa1ed978700 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14adfd0 0x14f8270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=e96f760783a3403 server_cookie=66984bd596d5ddc in_seq=0 out_seq=0 2020-11-16T22:31:41.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1bbfff700 1 -- 172.21.15.181:0/3786201878 <== mds.1 v2:172.21.15.175:6840/2108826019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x12d2f50 con 0x14adfd0 2020-11-16T22:31:41.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1bbfff700 10 client.6053 handle_command_reply: tid=0 2020-11-16T22:31:41.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 client.6053 shutdown 2020-11-16T22:31:41.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 msgr2=0x14f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa1e0017c40 tx=0x7fa1e0008040).stop 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 shutdown_connections 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x14adfd0 0x14f8270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x145de40 0x14f7d80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1221bb0 0x1502ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1bbfff700 0 client.6053 ms_handle_reset on v2:172.21.15.175:6840/2108826019 2020-11-16T22:31:41.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 --2- 172.21.15.181:0/3786201878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x128f7b0 0x1562d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.603+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 >> 172.21.15.181:0/3786201878 conn(0x7fa1b4002320 msgr2=0x14db390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 shutdown_connections 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1fe66d740 1 -- 172.21.15.181:0/3786201878 wait complete. 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1fe66d740 20 client.6053 trim_cache size 0 max 16384 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 msgr2=0x7fa1c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 0x7fa1c0057520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fa1e80b1480 tx=0x7fa1d8007970).stop 2020-11-16T22:31:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 msgr2=0x7fa1e81476f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fa1e4006ef0 tx=0x7fa1e4015000).stop 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 shutdown_connections 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1c0055060 0x7fa1c0057520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e815fbe0 0x7fa1e80ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e815b810 0x7fa1e81476f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 --2- 172.21.15.181:0/2320415624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e815cc00 0x7fa1e80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 >> 172.21.15.181:0/2320415624 conn(0x7fa1e80d2dc0 msgr2=0x7fa1e80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 shutdown_connections 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:38.607+0000 7fa1ee97a700 1 -- 172.21.15.181:0/2320415624 wait complete. 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:31:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:31:41.595 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:41.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 Processor -- start 2020-11-16T22:31:41.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 -- start start 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 0x7ff0b4167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 0x7ff0b40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b4168390 con 0x7ff0b4166bb0 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b4167770 con 0x7ff0b40b3220 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b8c98700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40b5f00 con 0x7ff0b4167ac0 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 0x7ff0b40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 0x7ff0b4167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 0x7ff0b40b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32894/0 (socket says 172.21.15.181:32894) 2020-11-16T22:31:41.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b1d9b700 1 -- 172.21.15.181:0/793359993 learned_addr learned my addr 172.21.15.181:0/793359993 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 -- 172.21.15.181:0/793359993 >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 msgr2=0x7ff0b40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 --2- 172.21.15.181:0/793359993 >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 0x7ff0b40b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 -- 172.21.15.181:0/793359993 >> v1:172.21.15.175:6789/0 conn(0x7ff0b4166bb0 legacy=0x7ff0b4167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 -- 172.21.15.181:0/793359993 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40b2060 con 0x7ff0b4167ac0 2020-11-16T22:31:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b159a700 1 -- 172.21.15.181:0/793359993 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741705629 0 0) 0x7ff0b4168390 con 0x7ff0b4166bb0 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b259c700 1 --2- 172.21.15.181:0/793359993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 0x7ff0b4167ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff0a400b9a0 tx=0x7ff0a4006df0).ready entity=mon.2 client_cookie=376a9c4e92ce1655 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b159a700 1 -- 172.21.15.181:0/793359993 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40097b0 con 0x7ff0b4167ac0 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b159a700 1 -- 172.21.15.181:0/793359993 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4009910 con 0x7ff0b4167ac0 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.742+0000 7ff0b159a700 1 -- 172.21.15.181:0/793359993 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4009c90 con 0x7ff0b4167ac0 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- 172.21.15.181:0/793359993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 msgr2=0x7ff0b4167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/793359993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 0x7ff0b4167ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff0a400b9a0 tx=0x7ff0a4006df0).stop 2020-11-16T22:31:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- 172.21.15.181:0/793359993 shutdown_connections 2020-11-16T22:31:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/793359993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b4167ac0 0x7ff0b4167ea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/793359993 >> v2:172.21.15.181:3300/0 conn(0x7ff0b40b3220 0x7ff0b40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- 172.21.15.181:0/793359993 >> 172.21.15.181:0/793359993 conn(0x7ff0b4060bc0 msgr2=0x7ff0b40d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- 172.21.15.181:0/793359993 shutdown_connections 2020-11-16T22:31:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- 172.21.15.181:0/793359993 wait complete. 2020-11-16T22:31:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 Processor -- start 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 -- start start 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.746+0000 7ff0b8c98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 0x7ff0b4160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0b4167ac0 0x7ff0b4169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40658f0 con 0x7ff0b4167ac0 2020-11-16T22:31:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4168d40 con 0x7ff0b4166bb0 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4168fe0 con 0x7ff0b40b3220 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40742/0 (socket says 172.21.15.181:40742) 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 -- 172.21.15.181:0/1275137936 learned_addr learned my addr 172.21.15.181:0/1275137936 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b1d9b700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 0x7ff0b4160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b2d9d700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0b4167ac0 0x7ff0b4169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 -- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 msgr2=0x7ff0b4160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 0x7ff0b4160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 -- 172.21.15.181:0/1275137936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0b4167ac0 msgr2=0x7ff0b4169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0b4167ac0 0x7ff0b4169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 -- 172.21.15.181:0/1275137936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4169280 con 0x7ff0b40b3220 2020-11-16T22:31:41.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b1d9b700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 0x7ff0b4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:31:41.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b259c700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff0a4006fd0 tx=0x7ff0a4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:41.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4011e50 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a401e460 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b4169520 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a401e7e0 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b4169a60 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b4069eb0 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff0a4025030 con 0x7ff0b40b3220 2020-11-16T22:31:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 0x7ff084057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0b1d9b700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 0x7ff084057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.750+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff0a406f560 con 0x7ff0b40b3220 2020-11-16T22:31:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.754+0000 7ff0b1d9b700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 0x7ff084057520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff09c00b190 tx=0x7ff09c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.758+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff0a4045870 con 0x7ff0b40b3220 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.974+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b4061100 con 0x7ff0b40b3220 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0aa7fc700 1 -- 172.21.15.181:0/1275137936 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4661 (secure 0 0 0) 0x7ff0a4023070 con 0x7ff0b40b3220 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:34.879851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 msgr2=0x7ff084057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 0x7ff084057520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff09c00b190 tx=0x7ff09c008040).stop 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 msgr2=0x7ff0b4065400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff0a4006fd0 tx=0x7ff0a4015000).stop 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 shutdown_connections 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff084055060 0x7ff084057520 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0b4167ac0 0x7ff0b4169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0b40b3220 0x7ff0b4065400 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 --2- 172.21.15.181:0/1275137936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b4166bb0 0x7ff0b4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 >> 172.21.15.181:0/1275137936 conn(0x7ff0b4060bc0 msgr2=0x7ff0b4167060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 shutdown_connections 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:41.978+0000 7ff0b8c98700 1 -- 172.21.15.181:0/1275137936 wait complete. 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 70 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d flush journal 2020-11-16T22:31:42.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 Processor -- start 2020-11-16T22:31:42.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 -- start start 2020-11-16T22:31:42.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f176c145a30 0x7f176c145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c061b50 con 0x7f176c1492c0 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0618b0 con 0x7f176c0816e0 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f1771900700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0cb930 con 0x7f176c145a30 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f17710ff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f176c0816e0 0x7f176c0cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47216/0 (socket says 172.21.15.181:47216) 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f17710ff700 1 -- 172.21.15.181:0/4127378235 learned_addr learned my addr 172.21.15.181:0/4127378235 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:42.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f17708fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f176c145a30 0x7f176c145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.118+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122889301 0 0) 0x7f176c0618b0 con 0x7f176c0816e0 2020-11-16T22:31:42.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c003240 con 0x7f176c0816e0 2020-11-16T22:31:42.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048238703 0 0) 0x7f176c061b50 con 0x7f176c1492c0 2020-11-16T22:31:42.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f176c0618b0 con 0x7f176c1492c0 2020-11-16T22:31:42.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f17708fe700 1 -- 172.21.15.181:0/4127378235 >> v1:172.21.15.181:6789/0 conn(0x7f176c0816e0 legacy=0x7f176c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f17708fe700 1 -- 172.21.15.181:0/4127378235 >> v1:172.21.15.175:6789/0 conn(0x7f176c1492c0 legacy=0x7f176c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f17708fe700 1 -- 172.21.15.181:0/4127378235 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0cbbd0 con 0x7f176c145a30 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f17708fe700 1 --2- 172.21.15.181:0/4127378235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c145a30 0x7f176c145e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f175400b9a0 tx=0x7f1754006df0).ready entity=mon.1 client_cookie=152329049a278022 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754009680 con 0x7f176c145a30 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17540097e0 con 0x7f176c145a30 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f176b7fe700 1 -- 172.21.15.181:0/4127378235 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754009b60 con 0x7f176c145a30 2020-11-16T22:31:42.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- 172.21.15.181:0/4127378235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c145a30 msgr2=0x7f176c145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 --2- 172.21.15.181:0/4127378235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c145a30 0x7f176c145e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f175400b9a0 tx=0x7f1754006df0).stop 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- 172.21.15.181:0/4127378235 shutdown_connections 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 --2- 172.21.15.181:0/4127378235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c145a30 0x7f176c145e10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- 172.21.15.181:0/4127378235 >> 172.21.15.181:0/4127378235 conn(0x7f176c0d68c0 msgr2=0x7f176c0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- 172.21.15.181:0/4127378235 shutdown_connections 2020-11-16T22:31:42.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- 172.21.15.181:0/4127378235 wait complete. 2020-11-16T22:31:42.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 Processor -- start 2020-11-16T22:31:42.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 -- start start 2020-11-16T22:31:42.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0816e0 0x7f176c130eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.122+0000 7f1771900700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c1492c0 0x7f176c0d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1771900700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c16f2b0 con 0x7f176c0816e0 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1771900700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c16f4e0 con 0x7f176c1492c0 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1771900700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c16f780 con 0x7f176c145a30 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40758/0 (socket says 172.21.15.181:40758) 2020-11-16T22:31:42.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 -- 172.21.15.181:0/1645582423 learned_addr learned my addr 172.21.15.181:0/1645582423 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c1492c0 0x7f176c0d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f17708fe700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0816e0 0x7f176c130eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 -- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c1492c0 msgr2=0x7f176c0d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c1492c0 0x7f176c0d32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 -- 172.21.15.181:0/1645582423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0816e0 msgr2=0x7f176c130eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0816e0 0x7f176c130eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 -- 172.21.15.181:0/1645582423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c16fa20 con 0x7f176c145a30 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f176bfff700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f175c00c6c0 tx=0x7f175c007ae0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1768ff9700 1 -- 172.21.15.181:0/1645582423 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e490 con 0x7f176c145a30 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1768ff9700 1 -- 172.21.15.181:0/1645582423 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c00e5f0 con 0x7f176c145a30 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c16fcc0 con 0x7f176c145a30 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1768ff9700 1 -- 172.21.15.181:0/1645582423 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e990 con 0x7f176c145a30 2020-11-16T22:31:42.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c170200 con 0x7f176c145a30 2020-11-16T22:31:42.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.126+0000 7f1768ff9700 1 -- 172.21.15.181:0/1645582423 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f175c019030 con 0x7f176c145a30 2020-11-16T22:31:42.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f1768ff9700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f1768ff9700 1 -- 172.21.15.181:0/1645582423 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f175c06d290 con 0x7f176c145a30 2020-11-16T22:31:42.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f17708fe700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f17708fe700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1754006e30 tx=0x7f1754015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f17815f3740 1 Processor -- start 2020-11-16T22:31:42.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.130+0000 7f17815f3740 1 -- start start 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x297d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x2974700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x28064b0 con 0x275c7b0 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x28ba930 con 0x26eebb0 2020-11-16T22:31:42.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x277a6f0 con 0x297afd0 2020-11-16T22:31:42.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x2974700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32924/0 (socket says 172.21.15.181:32924) 2020-11-16T22:31:42.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 -- 172.21.15.181:0/2564936233 learned_addr learned my addr 172.21.15.181:0/2564936233 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:42.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17708fe700 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x297d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 -- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 msgr2=0x2974700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x2974700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 -- 172.21.15.181:0/2564936233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 msgr2=0x297d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x297d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 -- 172.21.15.181:0/2564936233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27755a0 con 0x26eebb0 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17708fe700 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x297d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f176bfff700 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f175c001180 tx=0x7f175c012000).ready entity=mon.1 client_cookie=4f76a1ecad282bf8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f1761ffb700 1 -- 172.21.15.181:0/2564936233 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0438c0 con 0x26eebb0 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f1761ffb700 1 -- 172.21.15.181:0/2564936233 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c043a20 con 0x26eebb0 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f1761ffb700 1 -- 172.21.15.181:0/2564936233 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0468b0 con 0x26eebb0 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 msgr2=0x297b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f175c001180 tx=0x7f175c012000).stop 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- 172.21.15.181:0/2564936233 shutdown_connections 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x2974700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x297d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 --2- 172.21.15.181:0/2564936233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x297b670 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- 172.21.15.181:0/2564936233 >> 172.21.15.181:0/2564936233 conn(0x292ae40 msgr2=0x29773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- 172.21.15.181:0/2564936233 shutdown_connections 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- 172.21.15.181:0/2564936233 wait complete. 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 -- start start 2020-11-16T22:31:42.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.134+0000 7f17815f3740 1 Processor -- start 2020-11-16T22:31:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x29db510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x29ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x28a76e0 con 0x275c7b0 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x28a7820 con 0x26eebb0 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x28a8a90 con 0x297afd0 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 10 client.0 ll_register_callbacks cb 0x289dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40774/0 (socket says 172.21.15.181:40774) 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17710ff700 1 -- 172.21.15.181:0/484619634 learned_addr learned my addr 172.21.15.181:0/484619634 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17708fe700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x29db510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x29ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 -- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 msgr2=0x29c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 -- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 msgr2=0x29db510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x29db510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 -- 172.21.15.181:0/484619634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2898280 con 0x275c7b0 2020-11-16T22:31:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17710ff700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f176bfff700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x29ca020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f175c025000 tx=0x7f175c023000).ready entity=mon.0 client_cookie=b2b0725bf188e713 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c046320 con 0x275c7b0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x295b6c0 con 0x275c7b0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mon.0 v2:172.21.15.175:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f175c046480 con 0x275c7b0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c06bc70 con 0x275c7b0 2020-11-16T22:31:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 10 client.6411 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x26cdfd0 con 0x275c7b0 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mon.0 v2:172.21.15.175:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x295b6c0 con 0x275c7b0 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mon.0 v2:172.21.15.175:3300/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f175c045810 con 0x275c7b0 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 10 client.6411 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 10 client.6411 resolve_mds: resolved ID 'd' to GID 6295 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 20 client.6411 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x29aad80 0x29ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 4 client.6411 mds_command: new command op to 6295 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.138+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 --> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2798c50 con 0x29aad80 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17708fe700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x29aad80 0x29ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f173affd700 10 client.6411 ms_handle_connect on v2:172.21.15.181:6836/461666765 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17708fe700 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x29aad80 0x29ddfe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=70c483e6a84ed00b server_cookie=57d5b4de3f558483 in_seq=0 out_seq=0 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f173affd700 1 -- 172.21.15.181:0/484619634 <== mds.2 v2:172.21.15.181:6836/461666765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2798c50 con 0x29aad80 2020-11-16T22:31:42.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f173affd700 10 client.6411 handle_command_reply: tid=0 2020-11-16T22:31:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 client.6411 shutdown 2020-11-16T22:31:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 msgr2=0x29ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x29ca020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f175c025000 tx=0x7f175c023000).stop 2020-11-16T22:31:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 shutdown_connections 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x29aad80 0x29ddfe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297afd0 0x29c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f173affd700 0 client.6411 ms_handle_reset on v2:172.21.15.181:6836/461666765 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26eebb0 0x29db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 --2- 172.21.15.181:0/484619634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x275c7b0 0x29ca020 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 >> 172.21.15.181:0/484619634 conn(0x292ae40 msgr2=0x29ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 shutdown_connections 2020-11-16T22:31:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 1 -- 172.21.15.181:0/484619634 wait complete. 2020-11-16T22:31:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.142+0000 7f17815f3740 20 client.6411 trim_cache size 0 max 16384 2020-11-16T22:31:42.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 Processor -- start 2020-11-16T22:31:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- start start 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x29772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x2979f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x287fe40 con 0x292ae40 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c5760 con 0x275c7b0 2020-11-16T22:31:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x29c5990 con 0x26eebb0 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x2979f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32938/0 (socket says 172.21.15.181:32938) 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x2979f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40782/0 (socket says 172.21.15.181:40782) 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 -- 172.21.15.181:0/1283004449 learned_addr learned my addr 172.21.15.181:0/1283004449 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17708fe700 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x29772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 -- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 msgr2=0x2979f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x2979f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 -- 172.21.15.181:0/1283004449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 msgr2=0x29772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x29772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 -- 172.21.15.181:0/1283004449 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400081d0 con 0x275c7b0 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17710ff700 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f1764007750 tx=0x7f1764006040).ready entity=mon.1 client_cookie=6d71c22fd9426a62 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17627fc700 1 -- 172.21.15.181:0/1283004449 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176401a030 con 0x275c7b0 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17627fc700 1 -- 172.21.15.181:0/1283004449 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764001f50 con 0x275c7b0 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17627fc700 1 -- 172.21.15.181:0/1283004449 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764004a10 con 0x275c7b0 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 msgr2=0x297a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f1764007750 tx=0x7f1764006040).stop 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- 172.21.15.181:0/1283004449 shutdown_connections 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x297a440 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x29772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 --2- 172.21.15.181:0/1283004449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x2979f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- 172.21.15.181:0/1283004449 >> 172.21.15.181:0/1283004449 conn(0x7f1740002320 msgr2=0x297d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- 172.21.15.181:0/1283004449 shutdown_connections 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.162+0000 7f17815f3740 1 -- 172.21.15.181:0/1283004449 wait complete. 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- start start 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 Processor -- start 2020-11-16T22:31:42.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x2a30470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2974dc0 con 0x292ae40 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a30960 con 0x275c7b0 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2974880 con 0x26eebb0 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 10 client.0 ll_register_callbacks cb 0x297ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17708fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32944/0 (socket says 172.21.15.181:32944) 2020-11-16T22:31:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 -- 172.21.15.181:0/2714114768 learned_addr learned my addr 172.21.15.181:0/2714114768 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17708fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40786/0 (socket says 172.21.15.181:40786) 2020-11-16T22:31:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17710ff700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x2a30470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 -- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 msgr2=0x29ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 -- 172.21.15.181:0/2714114768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 msgr2=0x2a30470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x2a30470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 -- 172.21.15.181:0/2714114768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x270c720 con 0x275c7b0 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17708fe700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f176bfff700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f175c0436b0 tx=0x7f175c02e000).ready entity=mon.1 client_cookie=33af98eb4c036ee4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c045810 con 0x275c7b0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f175c046320 con 0x275c7b0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175c043a20 con 0x275c7b0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c045970 con 0x275c7b0 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 10 client.6556 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x2717e00 con 0x275c7b0 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c043a20 con 0x275c7b0 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f175c0448a0 con 0x275c7b0 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 10 client.6556 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 10 client.6556 resolve_mds: resolved ID 'd' to GID 6295 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 20 client.6556 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x297afd0 0x2a15a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 4 client.6556 mds_command: new command op to 6295 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:31:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 --> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x279ff50 con 0x297afd0 2020-11-16T22:31:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17708fe700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x297afd0 0x2a15a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f173affd700 10 client.6556 ms_handle_connect on v2:172.21.15.181:6836/461666765 2020-11-16T22:31:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.166+0000 7f17708fe700 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x297afd0 0x2a15a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=a06b944cebfeb55a server_cookie=8886fa1588ffa24d in_seq=0 out_seq=0 2020-11-16T22:31:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f173affd700 1 -- 172.21.15.181:0/2714114768 <== mds.2 v2:172.21.15.181:6836/461666765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x279ff50 con 0x297afd0 2020-11-16T22:31:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f173affd700 10 client.6556 handle_command_reply: tid=0 2020-11-16T22:31:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 client.6556 shutdown 2020-11-16T22:31:42.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 msgr2=0x2a2d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f175c0436b0 tx=0x7f175c02e000).stop 2020-11-16T22:31:42.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 shutdown_connections 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x297afd0 0x2a15a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x292ae40 0x2a30470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26eebb0 0x29ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 --2- 172.21.15.181:0/2714114768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x275c7b0 0x2a2d030 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 >> 172.21.15.181:0/2714114768 conn(0x7f1740002320 msgr2=0x297d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f173affd700 0 client.6556 ms_handle_reset on v2:172.21.15.181:6836/461666765 2020-11-16T22:31:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 shutdown_connections 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 1 -- 172.21.15.181:0/2714114768 wait complete. 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.190+0000 7f17815f3740 20 client.6556 trim_cache size 0 max 16384 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:31:42.180 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:31:42.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 msgr2=0x7f1748057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1754006e30 tx=0x7f1754015000).stop 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 msgr2=0x7f176c15f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f175c00c6c0 tx=0x7f175c007ae0).stop 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 shutdown_connections 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1748055060 0x7f1748057520 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f176c1492c0 0x7f176c0d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f176c0816e0 0x7f176c130eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 --2- 172.21.15.181:0/1645582423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f176c145a30 0x7f176c15f350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:42.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 >> 172.21.15.181:0/1645582423 conn(0x7f176c0d68c0 msgr2=0x7f176c1604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:42.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 shutdown_connections 2020-11-16T22:31:42.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:42.194+0000 7f1771900700 1 -- 172.21.15.181:0/1645582423 wait complete. 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:31:42.254 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/ 2020-11-16T22:31:42.254 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4 2020-11-16T22:31:42.255 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5 2020-11-16T22:31:42.255 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:31:42.255 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:31:42.255 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:31:42.259 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:42.259 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:42.260 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:42.260 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:42.260 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:31:42.261 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:42.365 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2020-11-16T22:31:42.856 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:42.857 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:42.857 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:42.857 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:42.857 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:31:42.858 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:43.017 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2020-11-16T22:31:43.208 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:43.208 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:43.209 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:43.209 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:43.209 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:31:43.210 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627781 2020-11-16T22:31:43.513 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:43.513 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:43.514 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:43.514 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:43.514 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:31:43.515 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:43.628 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627782 2020-11-16T22:31:43.816 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:31:43.817 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:31:43.817 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:31:43.818 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:31:43.818 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:31:43.818 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:31:43.931 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627783 2020-11-16T22:31:44.120 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:31:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 Processor -- start 2020-11-16T22:31:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 -- start start 2020-11-16T22:31:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 0x55ca2180b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca21764460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca216d51f0 con 0x55ca2177fec0 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca216a3be0 con 0x55ca2177fa60 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca21809d90 con 0x55ca2180c4a0 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49c14e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49c0ce5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 0x55ca2180b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49c14e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40802/0 (socket says 172.21.15.181:40802) 2020-11-16T22:31:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.166+0000 7f49c14e6700 1 -- 172.21.15.181:0/1595117106 learned_addr learned my addr 172.21.15.181:0/1595117106 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c04e4700 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca21764460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 -- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 msgr2=0x55ca217830d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 -- 172.21.15.181:0/1595117106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 msgr2=0x55ca21764460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca21764460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 -- 172.21.15.181:0/1595117106 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca216cd8a0 con 0x55ca2177fa60 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c14e6700 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49c0ce5700 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 0x55ca2180b920 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f49b800a900 tx=0x7f49b8015000).ready entity=mon.1 client_cookie=3c8047a621bfaafd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49bf4e2700 1 -- 172.21.15.181:0/1595117106 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8008910 con 0x55ca2177fa60 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49bf4e2700 1 -- 172.21.15.181:0/1595117106 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8008a70 con 0x55ca2177fa60 2020-11-16T22:31:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49bf4e2700 1 -- 172.21.15.181:0/1595117106 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b801c690 con 0x55ca2177fa60 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 msgr2=0x55ca2180b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 0x55ca2180b920 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f49b800a900 tx=0x7f49b8015000).stop 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/1595117106 shutdown_connections 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2180c4a0 0x55ca217830d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2177fa60 0x55ca2180b920 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/1595117106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca21764460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/1595117106 >> 172.21.15.181:0/1595117106 conn(0x55ca217803e0 msgr2=0x55ca2177f090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/1595117106 shutdown_connections 2020-11-16T22:31:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/1595117106 wait complete. 2020-11-16T22:31:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 Processor -- start 2020-11-16T22:31:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- start start 2020-11-16T22:31:44.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2177fa60 0x55ca2177d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca2177b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 0x55ca2181d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2180be10 con 0x55ca2177fec0 2020-11-16T22:31:44.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2177b760 con 0x55ca2180c4a0 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.170+0000 7f49cf6dfc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca2177af80 con 0x55ca2177fa60 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c0ce5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2177fa60 0x55ca2177d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 0x55ca2181d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c0ce5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2177fa60 0x55ca2177d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40804/0 (socket says 172.21.15.181:40804) 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 0x55ca2181d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32966/0 (socket says 172.21.15.181:32966) 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c0ce5700 1 -- 172.21.15.181:0/514246017 learned_addr learned my addr 172.21.15.181:0/514246017 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c04e4700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca2177b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 -- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2177fa60 msgr2=0x55ca2177d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ca2177fa60 0x55ca2177d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 -- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 msgr2=0x55ca2177b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca2177fec0 0x55ca2177b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 -- 172.21.15.181:0/514246017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca216c9280 con 0x55ca2180c4a0 2020-11-16T22:31:44.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49c14e6700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 0x55ca2181d730 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f49b0005ae0 tx=0x7f49b000c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49bdcdf700 1 -- 172.21.15.181:0/514246017 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000f040 con 0x55ca2180c4a0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49bdcdf700 1 -- 172.21.15.181:0/514246017 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0010940 con 0x55ca2180c4a0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49bdcdf700 1 -- 172.21.15.181:0/514246017 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0010cc0 con 0x55ca2180c4a0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca216a7620 con 0x55ca2180c4a0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca2177b4c0 con 0x55ca2180c4a0 2020-11-16T22:31:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.174+0000 7f49bdcdf700 1 -- 172.21.15.181:0/514246017 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f49b001b020 con 0x55ca2180c4a0 2020-11-16T22:31:44.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49bdcdf700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f49a0053ce0 0x7f49a00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49bdcdf700 1 -- 172.21.15.181:0/514246017 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49b006d490 con 0x55ca2180c4a0 2020-11-16T22:31:44.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49c0ce5700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f49a0053ce0 0x7f49a00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x55ca2180e060 0x55ca21808270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- osd_op(unknown.0.0:1 9.2 9:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x55ca21764540 con 0x55ca2180e060 2020-11-16T22:31:44.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49c0ce5700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f49a0053ce0 0x7f49a00561a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f49b801e040 tx=0x7f49b801c000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:44.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49c04e4700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x55ca2180e060 0x55ca21808270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.178+0000 7f49c04e4700 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x55ca2180e060 0x55ca21808270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:44.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49c04e4700 1 -- 172.21.15.181:0/514246017 <== osd.6 v2:172.21.15.175:6808/15330 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v64'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f49a8009040 con 0x55ca2180e060 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x55ca2180e060 msgr2=0x55ca21808270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x55ca2180e060 0x55ca21808270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f49a0053ce0 msgr2=0x7f49a00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f49a0053ce0 0x7f49a00561a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f49b801e040 tx=0x7f49b801c000).stop 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 msgr2=0x55ca2181d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 --2- 172.21.15.181:0/514246017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca2180c4a0 0x55ca2181d730 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f49b0005ae0 tx=0x7f49b000c040).stop 2020-11-16T22:31:44.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49c14e6700 1 -- 172.21.15.181:0/514246017 reap_dead start 2020-11-16T22:31:44.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 shutdown_connections 2020-11-16T22:31:44.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 >> 172.21.15.181:0/514246017 conn(0x55ca217803e0 msgr2=0x55ca2180b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:44.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 shutdown_connections 2020-11-16T22:31:44.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.182+0000 7f49cf6dfc40 1 -- 172.21.15.181:0/514246017 wait complete. 2020-11-16T22:31:44.171 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:31:44.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 Processor -- start 2020-11-16T22:31:44.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 -- start start 2020-11-16T22:31:44.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 0x56333925d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x5633391edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 0x5633392e7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633391b11f0 con 0x56333925bec0 2020-11-16T22:31:44.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333917fbe0 con 0x5633392e84a0 2020-11-16T22:31:44.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febffa74c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633392ebff0 con 0x56333925ba60 2020-11-16T22:31:44.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf187b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 0x5633392e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 0x56333925d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf187b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 0x5633392e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32980/0 (socket says 172.21.15.181:32980) 2020-11-16T22:31:44.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf187b700 1 -- 172.21.15.181:0/1732872751 learned_addr learned my addr 172.21.15.181:0/1732872751 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:44.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf0879700 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x5633391edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 -- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 msgr2=0x5633392e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:44.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 0x5633392e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 -- 172.21.15.181:0/1732872751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 msgr2=0x5633391edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:44.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x5633391edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 -- 172.21.15.181:0/1732872751 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633391a98a0 con 0x56333925ba60 2020-11-16T22:31:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febf107a700 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 0x56333925d540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7febdc00ad10 tx=0x7febdc007040).ready entity=mon.2 client_cookie=fcf1872fb6027978 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febeb7fe700 1 -- 172.21.15.181:0/1732872751 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00db80 con 0x56333925ba60 2020-11-16T22:31:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febeb7fe700 1 -- 172.21.15.181:0/1732872751 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc00dce0 con 0x56333925ba60 2020-11-16T22:31:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.262+0000 7febeb7fe700 1 -- 172.21.15.181:0/1732872751 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc011600 con 0x56333925ba60 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 msgr2=0x56333925d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 0x56333925d540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7febdc00ad10 tx=0x7febdc007040).stop 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- 172.21.15.181:0/1732872751 shutdown_connections 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633392e84a0 0x5633392e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56333925ba60 0x56333925d540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 --2- 172.21.15.181:0/1732872751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x5633391edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- 172.21.15.181:0/1732872751 >> 172.21.15.181:0/1732872751 conn(0x56333925c3e0 msgr2=0x56333923e980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- 172.21.15.181:0/1732872751 shutdown_connections 2020-11-16T22:31:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- 172.21.15.181:0/1732872751 wait complete. 2020-11-16T22:31:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 Processor -- start 2020-11-16T22:31:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.266+0000 7febffa74c40 1 -- start start 2020-11-16T22:31:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56333925ba60 0x5633392555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x563339259fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 0x5633392e6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339255ab0 con 0x56333925bec0 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633392f9f10 con 0x56333925ba60 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633392f9c70 con 0x5633392e84a0 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf107a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56333925ba60 0x5633392555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 0x5633392e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 0x5633392e6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40830/0 (socket says 172.21.15.181:40830) 2020-11-16T22:31:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 -- 172.21.15.181:0/114055722 learned_addr learned my addr 172.21.15.181:0/114055722 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf107a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56333925ba60 0x5633392555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32984/0 (socket says 172.21.15.181:32984) 2020-11-16T22:31:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf0879700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x563339259fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 -- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56333925ba60 msgr2=0x5633392555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56333925ba60 0x5633392555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 -- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 msgr2=0x563339259fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:45.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56333925bec0 0x563339259fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 -- 172.21.15.181:0/114055722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633391a5280 con 0x5633392e84a0 2020-11-16T22:31:45.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febf187b700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 0x5633392e6380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7febec00c920 tx=0x7febec000b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:45.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febe9ffb700 1 -- 172.21.15.181:0/114055722 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00f620 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563339183620 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febe9ffb700 1 -- 172.21.15.181:0/114055722 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec014030 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febe9ffb700 1 -- 172.21.15.181:0/114055722 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00f9b0 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.270+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633392549a0 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febe9ffb700 1 -- 172.21.15.181:0/114055722 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7febec00fd20 con 0x5633392e84a0 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febe9ffb700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febd4053ce0 0x7febd40561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febe9ffb700 1 -- 172.21.15.181:0/114055722 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7febec06d000 con 0x5633392e84a0 2020-11-16T22:31:45.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febf107a700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febd4053ce0 0x7febd40561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febffa74c40 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x5633392eb850 0x5633392f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- osd_op(unknown.0.0:1 9.2 9:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x56333925a4a0 con 0x5633392eb850 2020-11-16T22:31:45.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febf0879700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x5633392eb850 0x5633392f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febf107a700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febd4053ce0 0x7febd40561a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7febdc00acb0 tx=0x7febdc019000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:45.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.274+0000 7febf0879700 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x5633392eb850 0x5633392f9030 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:45.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febf0879700 1 -- 172.21.15.181:0/114055722 <== osd.6 v2:172.21.15.175:6808/15330 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v64'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febe4000b80 con 0x5633392eb850 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x5633392eb850 msgr2=0x5633392f9030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x5633392eb850 0x5633392f9030 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febd4053ce0 msgr2=0x7febd40561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febd4053ce0 0x7febd40561a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7febdc00acb0 tx=0x7febdc019000).stop 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 msgr2=0x5633392e6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 --2- 172.21.15.181:0/114055722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5633392e84a0 0x5633392e6380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7febec00c920 tx=0x7febec000b80).stop 2020-11-16T22:31:45.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febf187b700 1 -- 172.21.15.181:0/114055722 reap_dead start 2020-11-16T22:31:45.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 shutdown_connections 2020-11-16T22:31:45.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 >> 172.21.15.181:0/114055722 conn(0x56333925c3e0 msgr2=0x56333923e980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:45.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 shutdown_connections 2020-11-16T22:31:45.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:44.282+0000 7febffa74c40 1 -- 172.21.15.181:0/114055722 wait complete. 2020-11-16T22:31:45.474 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:31:45.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.522+0000 7f51c3009c40 1 Processor -- start 2020-11-16T22:31:45.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.522+0000 7f51c3009c40 1 -- start start 2020-11-16T22:31:45.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7a60 0x55db7a0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db79febbe0 con 0x55db7a0c7a60 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7a0c0ab0 con 0x55db7a0c7ec0 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7a0c0d50 con 0x55db7a156930 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b3e0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b3e0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33004/0 (socket says 172.21.15.181:33004) 2020-11-16T22:31:45.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40844/0 (socket says 172.21.15.181:40844) 2020-11-16T22:31:45.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b460f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7a60 0x55db7a0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 -- 172.21.15.181:0/2439210271 learned_addr learned my addr 172.21.15.181:0/2439210271 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:45.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 -- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 msgr2=0x55db7a156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:45.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 -- 172.21.15.181:0/2439210271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7a60 msgr2=0x55db7a0ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:45.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7a60 0x55db7a0ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 -- 172.21.15.181:0/2439210271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7a014480 con 0x55db7a156930 2020-11-16T22:31:45.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b3e0e700 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b4e10700 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f51ac0054e0 tx=0x7f51ac00c040).ready entity=mon.2 client_cookie=956f2dab96845ae6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b2e0c700 1 -- 172.21.15.181:0/2439210271 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00f040 con 0x55db7a156930 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b2e0c700 1 -- 172.21.15.181:0/2439210271 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac008b80 con 0x55db7a156930 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51b2e0c700 1 -- 172.21.15.181:0/2439210271 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac014430 con 0x55db7a156930 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 msgr2=0x55db7a0c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:45.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f51ac0054e0 tx=0x7f51ac00c040).stop 2020-11-16T22:31:45.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- 172.21.15.181:0/2439210271 shutdown_connections 2020-11-16T22:31:45.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6d10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7a60 0x55db7a0ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 --2- 172.21.15.181:0/2439210271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7ec0 0x55db7a156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:45.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- 172.21.15.181:0/2439210271 >> 172.21.15.181:0/2439210271 conn(0x55db7a0c83e0 msgr2=0x55db7a0c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- 172.21.15.181:0/2439210271 shutdown_connections 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- 172.21.15.181:0/2439210271 wait complete. 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 Processor -- start 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.526+0000 7f51c3009c40 1 -- start start 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 0x55db7a0c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7ec0 0x55db7a0c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7a06dfb0 con 0x55db7a0c7ec0 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7a152a20 con 0x55db7a0c7a60 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7a0c5360 con 0x55db7a156930 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 0x55db7a0c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 0x55db7a0c1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33006/0 (socket says 172.21.15.181:33006) 2020-11-16T22:31:45.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 -- 172.21.15.181:0/4187071085 learned_addr learned my addr 172.21.15.181:0/4187071085 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b4e10700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b3e0e700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7ec0 0x55db7a0c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 -- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 msgr2=0x55db7a0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 -- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7ec0 msgr2=0x55db7a0c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db7a0c7ec0 0x55db7a0c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 -- 172.21.15.181:0/4187071085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7a0158a0 con 0x55db7a0c7a60 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b4e10700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55db7a156930 0x55db7a0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b460f700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 0x55db7a0c1a50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f51a4005820 tx=0x7f51a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b1609700 1 -- 172.21.15.181:0/4187071085 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4012ae0 con 0x55db7a0c7a60 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b1609700 1 -- 172.21.15.181:0/4187071085 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4012c40 con 0x55db7a0c7a60 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b1609700 1 -- 172.21.15.181:0/4187071085 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a401e460 con 0x55db7a0c7a60 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55db7a011280 con 0x55db7a0c7a60 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db79fef620 con 0x55db7a0c7a60 2020-11-16T22:31:46.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.530+0000 7f51b1609700 1 -- 172.21.15.181:0/4187071085 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51a401a040 con 0x55db7a0c7a60 2020-11-16T22:31:46.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b1609700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5194053d10 0x7f51940561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b1609700 1 -- 172.21.15.181:0/4187071085 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51a406eef0 con 0x55db7a0c7a60 2020-11-16T22:31:46.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b3e0e700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5194053d10 0x7f51940561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51c3009c40 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55db7a0ab160 0x55db7a06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- osd_op(unknown.0.0:1 9.5 9:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x55db7a153390 con 0x55db7a0ab160 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b3e0e700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5194053d10 0x7f51940561d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x55db7a075e60 tx=0x7f519c007040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b4e10700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55db7a0ab160 0x55db7a06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.534+0000 7f51b4e10700 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55db7a0ab160 0x55db7a06f0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:46.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51b4e10700 1 -- 172.21.15.181:0/4187071085 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v64'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f51ac0098f0 con 0x55db7a0ab160 2020-11-16T22:31:46.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55db7a0ab160 msgr2=0x55db7a06f0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55db7a0ab160 0x55db7a06f0d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5194053d10 msgr2=0x7f51940561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5194053d10 0x7f51940561d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x55db7a075e60 tx=0x7f519c007040).stop 2020-11-16T22:31:46.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 msgr2=0x55db7a0c1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 --2- 172.21.15.181:0/4187071085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db7a0c7a60 0x55db7a0c1a50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f51a4005820 tx=0x7f51a4015000).stop 2020-11-16T22:31:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51b4e10700 1 -- 172.21.15.181:0/4187071085 reap_dead start 2020-11-16T22:31:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 shutdown_connections 2020-11-16T22:31:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 >> 172.21.15.181:0/4187071085 conn(0x55db7a0c83e0 msgr2=0x55db7a059c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 shutdown_connections 2020-11-16T22:31:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:45.538+0000 7f51c3009c40 1 -- 172.21.15.181:0/4187071085 wait complete. 2020-11-16T22:31:46.536 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:31:46.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 Processor -- start 2020-11-16T22:31:46.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 -- start start 2020-11-16T22:31:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 0x5627080f3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 0x5627081e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56270809f190 con 0x5627081bdf50 2020-11-16T22:31:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627080a01f0 con 0x5627081dbd90 2020-11-16T22:31:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56270806ebe0 con 0x5627081bdb70 2020-11-16T22:31:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3bb49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 0x5627081e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3b348700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3bb49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 0x5627081e4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33026/0 (socket says 172.21.15.181:33026) 2020-11-16T22:31:46.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3b348700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40864/0 (socket says 172.21.15.181:40864) 2020-11-16T22:31:46.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3bb49700 1 -- 172.21.15.181:0/1174133084 learned_addr learned my addr 172.21.15.181:0/1174133084 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:46.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.582+0000 7f0d3ab47700 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 0x5627080f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:46.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 -- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 msgr2=0x5627081e4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 0x5627081e4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 -- 172.21.15.181:0/1174133084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 msgr2=0x5627080f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 0x5627080f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 -- 172.21.15.181:0/1174133084 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562708094280 con 0x5627081bdb70 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3ab47700 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 0x5627080f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:46.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d3b348700 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0d2c00b560 tx=0x7f0d2c0016c0).ready entity=mon.2 client_cookie=acee4de8eec37eed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d39b45700 1 -- 172.21.15.181:0/1174133084 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00e430 con 0x5627081bdb70 2020-11-16T22:31:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d39b45700 1 -- 172.21.15.181:0/1174133084 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00e590 con 0x5627081bdb70 2020-11-16T22:31:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d39b45700 1 -- 172.21.15.181:0/1174133084 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00e910 con 0x5627081bdb70 2020-11-16T22:31:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 -- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 msgr2=0x5627081bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:46.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0d2c00b560 tx=0x7f0d2c0016c0).stop 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 -- 172.21.15.181:0/1174133084 shutdown_connections 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081dbd90 0x5627081e4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdb70 0x5627081bc450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/1174133084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081bdf50 0x5627080f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 -- 172.21.15.181:0/1174133084 >> 172.21.15.181:0/1174133084 conn(0x56270814b400 msgr2=0x562708133ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 -- 172.21.15.181:0/1174133084 shutdown_connections 2020-11-16T22:31:46.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.586+0000 7f0d49d42c40 1 -- 172.21.15.181:0/1174133084 wait complete. 2020-11-16T22:31:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 Processor -- start 2020-11-16T22:31:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 -- start start 2020-11-16T22:31:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 0x56270814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdf50 0x5627080f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:46.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081dbd90 0x5627081eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627081eb830 con 0x5627081dbd90 2020-11-16T22:31:47.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627080f1210 con 0x5627081bdb70 2020-11-16T22:31:47.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d49d42c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627080f14b0 con 0x5627081bdf50 2020-11-16T22:31:47.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3b348700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 0x56270814e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3ab47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdf50 0x5627080f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3ab47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdf50 0x5627080f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40872/0 (socket says 172.21.15.181:40872) 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3b348700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 0x56270814e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33028/0 (socket says 172.21.15.181:33028) 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3ab47700 1 -- 172.21.15.181:0/3606945308 learned_addr learned my addr 172.21.15.181:0/3606945308 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081dbd90 0x5627081eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 -- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdf50 msgr2=0x5627080f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5627081bdf50 0x5627080f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 -- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 msgr2=0x56270814e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 0x56270814e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3bb49700 1 -- 172.21.15.181:0/3606945308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562708072620 con 0x5627081dbd90 2020-11-16T22:31:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.590+0000 7f0d3b348700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5627081bdb70 0x56270814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d3bb49700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081dbd90 0x5627081eb340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0d34008340 tx=0x7f0d34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d33fff700 1 -- 172.21.15.181:0/3606945308 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34010bc0 con 0x5627081dbd90 2020-11-16T22:31:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d33fff700 1 -- 172.21.15.181:0/3606945308 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34010d20 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627080f1750 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d33fff700 1 -- 172.21.15.181:0/3606945308 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3401c690 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627080f1c90 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.594+0000 7f0d33fff700 1 -- 172.21.15.181:0/3606945308 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0d34016070 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d33fff700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0d1c053d10 0x7f0d1c0561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d33fff700 1 -- 172.21.15.181:0/3606945308 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d3406ede0 con 0x5627081dbd90 2020-11-16T22:31:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d3b348700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0d1c053d10 0x7f0d1c0561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x5627081e4a50 0x5627081342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d3b348700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0d1c053d10 0x7f0d1c0561d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f0d2c00bbd0 tx=0x7f0d2c014000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:47.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- osd_op(unknown.0.0:1 9.1 9:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x5627081c1480 con 0x5627081e4a50 2020-11-16T22:31:47.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d3ab47700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x5627081e4a50 0x5627081342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.598+0000 7f0d3ab47700 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x5627081e4a50 0x5627081342e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d3ab47700 1 -- 172.21.15.181:0/3606945308 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v64'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0d2400c5e0 con 0x5627081e4a50 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x5627081e4a50 msgr2=0x5627081342e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x5627081e4a50 0x5627081342e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0d1c053d10 msgr2=0x7f0d1c0561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0d1c053d10 0x7f0d1c0561d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f0d2c00bbd0 tx=0x7f0d2c014000).stop 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081dbd90 msgr2=0x5627081eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 --2- 172.21.15.181:0/3606945308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5627081dbd90 0x5627081eb340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0d34008340 tx=0x7f0d34015000).stop 2020-11-16T22:31:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d3bb49700 1 -- 172.21.15.181:0/3606945308 reap_dead start 2020-11-16T22:31:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 shutdown_connections 2020-11-16T22:31:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 >> 172.21.15.181:0/3606945308 conn(0x56270814b400 msgr2=0x5627081db420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 shutdown_connections 2020-11-16T22:31:47.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:46.602+0000 7f0d49d42c40 1 -- 172.21.15.181:0/3606945308 wait complete. 2020-11-16T22:31:47.799 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:31:47.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.845+0000 7f683ada0c40 1 Processor -- start 2020-11-16T22:31:47.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.845+0000 7f683ada0c40 1 -- start start 2020-11-16T22:31:47.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 0x55be1c12bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c1d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1c06cbe0 con 0x55be1c1d54a0 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1c1d72b0 con 0x55be1c148ec0 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f683ada0c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be1c1d40b0 con 0x55be1c148a60 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682c3a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682c3a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40886/0 (socket says 172.21.15.181:40886) 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 0x55be1c12bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682c3a6700 1 -- 172.21.15.181:0/2278160303 learned_addr learned my addr 172.21.15.181:0/2278160303 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:47.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682cba7700 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c1d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:47.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 -- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 msgr2=0x55be1c12d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:47.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 -- 172.21.15.181:0/2278160303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 msgr2=0x55be1c1d8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:47.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c1d8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 -- 172.21.15.181:0/2278160303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be1c095480 con 0x55be1c148ec0 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682c3a6700 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f6823ba5700 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 0x55be1c12bcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x55be1c0db410 tx=0x7f6814013000).ready entity=mon.1 client_cookie=a1379b9e8eb7c9e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682b3a4700 1 -- 172.21.15.181:0/2278160303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814008b50 con 0x55be1c148ec0 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682b3a4700 1 -- 172.21.15.181:0/2278160303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814008cb0 con 0x55be1c148ec0 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.849+0000 7f682b3a4700 1 -- 172.21.15.181:0/2278160303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814012830 con 0x55be1c148ec0 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 msgr2=0x55be1c12bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:47.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 0x55be1c12bcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x55be1c0db410 tx=0x7f6814013000).stop 2020-11-16T22:31:47.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- 172.21.15.181:0/2278160303 shutdown_connections 2020-11-16T22:31:47.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c1d8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148a60 0x55be1c12d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:47.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 --2- 172.21.15.181:0/2278160303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148ec0 0x55be1c12bcc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- 172.21.15.181:0/2278160303 >> 172.21.15.181:0/2278160303 conn(0x55be1c1493e0 msgr2=0x55be1c148030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:48.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- 172.21.15.181:0/2278160303 shutdown_connections 2020-11-16T22:31:48.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- 172.21.15.181:0/2278160303 wait complete. 2020-11-16T22:31:48.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 Processor -- start 2020-11-16T22:31:48.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.853+0000 7f683ada0c40 1 -- start start 2020-11-16T22:31:48.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148a60 0x55be1c12bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 0x55be1c1d7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1c1449d0 con 0x55be1c1d54a0 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1c1d7d20 con 0x55be1c148a60 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be1c1c3770 con 0x55be1c148ec0 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 0x55be1c1d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f682c3a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148a60 0x55be1c12bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 0x55be1c1d7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40892/0 (socket says 172.21.15.181:40892) 2020-11-16T22:31:48.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 -- 172.21.15.181:0/534806923 learned_addr learned my addr 172.21.15.181:0/534806923 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:48.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f682cba7700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 -- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148a60 msgr2=0x55be1c12bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148a60 0x55be1c12bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 -- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 msgr2=0x55be1c147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be1c1d54a0 0x55be1c147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 -- 172.21.15.181:0/534806923 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be1c0968a0 con 0x55be1c148ec0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f682c3a6700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be1c148a60 0x55be1c12bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6823ba5700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 0x55be1c1d7400 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f681401e040 tx=0x7f681401b000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6829ba1700 1 -- 172.21.15.181:0/534806923 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140052f0 con 0x55be1c148ec0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6829ba1700 1 -- 172.21.15.181:0/534806923 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814008e20 con 0x55be1c148ec0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be1c092280 con 0x55be1c148ec0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6829ba1700 1 -- 172.21.15.181:0/534806923 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681401cd10 con 0x55be1c148ec0 2020-11-16T22:31:48.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be1c070620 con 0x55be1c148ec0 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.857+0000 7f6829ba1700 1 -- 172.21.15.181:0/534806923 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f681402e610 con 0x55be1c148ec0 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f6829ba1700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f680c053ce0 0x7f680c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f6829ba1700 1 -- 172.21.15.181:0/534806923 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6814070c40 con 0x55be1c148ec0 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f682c3a6700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f680c053ce0 0x7f680c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f683ada0c40 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55be1c1d4240 0x55be1c1c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- osd_op(unknown.0.0:1 9.5 9:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x55be1c1c2fc0 con 0x55be1c1d4240 2020-11-16T22:31:48.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f682cba7700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55be1c1d4240 0x55be1c1c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f682c3a6700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f680c053ce0 0x7f680c0561a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f681c005e30 tx=0x7f681c00c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.861+0000 7f682cba7700 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55be1c1d4240 0x55be1c1c2ad0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f682cba7700 1 -- 172.21.15.181:0/534806923 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v64'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f682400ab90 con 0x55be1c1d4240 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55be1c1d4240 msgr2=0x55be1c1c2ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55be1c1d4240 0x55be1c1c2ad0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f680c053ce0 msgr2=0x7f680c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f680c053ce0 0x7f680c0561a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f681c005e30 tx=0x7f681c00c040).stop 2020-11-16T22:31:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 msgr2=0x55be1c1d7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 --2- 172.21.15.181:0/534806923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55be1c148ec0 0x55be1c1d7400 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f681401e040 tx=0x7f681401b000).stop 2020-11-16T22:31:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f682cba7700 1 -- 172.21.15.181:0/534806923 reap_dead start 2020-11-16T22:31:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 shutdown_connections 2020-11-16T22:31:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 >> 172.21.15.181:0/534806923 conn(0x55be1c1493e0 msgr2=0x55be1c1497c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:48.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 shutdown_connections 2020-11-16T22:31:48.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:47.865+0000 7f683ada0c40 1 -- 172.21.15.181:0/534806923 wait complete. 2020-11-16T22:31:48.459 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:31:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 Processor -- start 2020-11-16T22:31:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 -- start start 2020-11-16T22:31:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3ead2a60 0x563b3ead0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3ead1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3ea281f0 con 0x563b3ead2a60 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3e9f6be0 con 0x563b3ead2ec0 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f506edaec40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b3eab5a20 con 0x563b3eb5f4a0 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f505fbb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3ead1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40910/0 (socket says 172.21.15.181:40910) 2020-11-16T22:31:48.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 -- 172.21.15.181:0/2056720235 learned_addr learned my addr 172.21.15.181:0/2056720235 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:48.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f50603b4700 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3ead2a60 0x563b3ead0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 -- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 msgr2=0x563b3ead1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:48.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3ead1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 -- 172.21.15.181:0/2056720235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3ead2a60 msgr2=0x563b3ead0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:48.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.505+0000 7f5060bb5700 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3ead2a60 0x563b3ead0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f5060bb5700 1 -- 172.21.15.181:0/2056720235 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b3ea208a0 con 0x563b3eb5f4a0 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f5060bb5700 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f50580019e0 tx=0x7f5058006870).ready entity=mon.2 client_cookie=5472c247229b71e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505ebb1700 1 -- 172.21.15.181:0/2056720235 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058006320 con 0x563b3eb5f4a0 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505ebb1700 1 -- 172.21.15.181:0/2056720235 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058004500 con 0x563b3eb5f4a0 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505ebb1700 1 -- 172.21.15.181:0/2056720235 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580048a0 con 0x563b3eb5f4a0 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 msgr2=0x563b3ead4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f50580019e0 tx=0x7f5058006870).stop 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- 172.21.15.181:0/2056720235 shutdown_connections 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3eb5f4a0 0x563b3ead4540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3ead2a60 0x563b3ead0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- 172.21.15.181:0/2056720235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3ead1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- 172.21.15.181:0/2056720235 >> 172.21.15.181:0/2056720235 conn(0x563b3ead33e0 msgr2=0x563b3eb612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- 172.21.15.181:0/2056720235 shutdown_connections 2020-11-16T22:31:48.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- 172.21.15.181:0/2056720235 wait complete. 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 Processor -- start 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- start start 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3ead2a60 0x563b3eb61600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3eb74bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3eb5f4a0 0x563b3eb591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3eb750e0 con 0x563b3eb5f4a0 2020-11-16T22:31:48.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3ead13d0 con 0x563b3ead2ec0 2020-11-16T22:31:48.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f506edaec40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b3ead1670 con 0x563b3ead2a60 2020-11-16T22:31:48.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3eb74bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3eb74bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33072/0 (socket says 172.21.15.181:33072) 2020-11-16T22:31:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 -- 172.21.15.181:0/3294355990 learned_addr learned my addr 172.21.15.181:0/3294355990 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f50603b4700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3ead2a60 0x563b3eb61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f5060bb5700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3eb5f4a0 0x563b3eb591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 -- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3ead2a60 msgr2=0x563b3eb61600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3ead2a60 0x563b3eb61600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 -- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3eb5f4a0 msgr2=0x563b3eb591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b3eb5f4a0 0x563b3eb591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 -- 172.21.15.181:0/3294355990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b3ea1c280 con 0x563b3ead2ec0 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f50603b4700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563b3ead2a60 0x563b3eb61600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.509+0000 7f505fbb3700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3eb74bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f5050008e00 tx=0x7f505000c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 -- 172.21.15.181:0/3294355990 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000f040 con 0x563b3ead2ec0 2020-11-16T22:31:48.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 -- 172.21.15.181:0/3294355990 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050010cd0 con 0x563b3ead2ec0 2020-11-16T22:31:48.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b3e9fa620 con 0x563b3ead2ec0 2020-11-16T22:31:49.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 -- 172.21.15.181:0/3294355990 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050015680 con 0x563b3ead2ec0 2020-11-16T22:31:49.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b3ead1bb0 con 0x563b3ead2ec0 2020-11-16T22:31:49.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 -- 172.21.15.181:0/3294355990 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5050015a30 con 0x563b3ead2ec0 2020-11-16T22:31:49.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5040053d10 0x7f50400561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f50603b4700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5040053d10 0x7f50400561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.513+0000 7f505d3ae700 1 -- 172.21.15.181:0/3294355990 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f505006da80 con 0x563b3ead2ec0 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.517+0000 7f506edaec40 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x563b3eb62e20 0x563b3ead4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.517+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- osd_op(unknown.0.0:1 9.5 9:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e64) v8 -- 0x563b3eacccf0 con 0x563b3eb62e20 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.517+0000 7f50603b4700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5040053d10 0x7f50400561d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f5048008600 tx=0x7f5048013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:49.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.517+0000 7f5060bb5700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x563b3eb62e20 0x563b3ead4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.517+0000 7f5060bb5700 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x563b3eb62e20 0x563b3ead4a50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f5060bb5700 1 -- 172.21.15.181:0/3294355990 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v64'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5058001600 con 0x563b3eb62e20 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x563b3eb62e20 msgr2=0x563b3ead4a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x563b3eb62e20 0x563b3ead4a50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5040053d10 msgr2=0x7f50400561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5040053d10 0x7f50400561d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f5048008600 tx=0x7f5048013000).stop 2020-11-16T22:31:49.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 msgr2=0x563b3eb74bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 --2- 172.21.15.181:0/3294355990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b3ead2ec0 0x563b3eb74bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f5050008e00 tx=0x7f505000c040).stop 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f5060bb5700 1 -- 172.21.15.181:0/3294355990 reap_dead start 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 shutdown_connections 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 >> 172.21.15.181:0/3294355990 conn(0x563b3ead33e0 msgr2=0x563b3eab5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 shutdown_connections 2020-11-16T22:31:49.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:48.521+0000 7f506edaec40 1 -- 172.21.15.181:0/3294355990 wait complete. 2020-11-16T22:31:49.118 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:49.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.265+0000 7fb6a359e700 1 Processor -- start 2020-11-16T22:31:49.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a359e700 1 -- start start 2020-11-16T22:31:49.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a40cccd0 con 0x7fb6a415f3c0 2020-11-16T22:31:49.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a40ccf70 con 0x7fb6a40615f0 2020-11-16T22:31:49.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a415e030 con 0x7fb6a4156a90 2020-11-16T22:31:49.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40924/0 (socket says 172.21.15.181:40924) 2020-11-16T22:31:49.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 -- 172.21.15.181:0/764682829 learned_addr learned my addr 172.21.15.181:0/764682829 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:49.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a1d9b700 1 --1- 172.21.15.181:0/764682829 >> v1:172.21.15.181:6789/0 conn(0x7fb6a40615f0 0x7fb6a415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47388/0 (socket says 172.21.15.181:47388) 2020-11-16T22:31:49.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 -- 172.21.15.181:0/764682829 >> v1:172.21.15.181:6789/0 conn(0x7fb6a40615f0 legacy=0x7fb6a415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 -- 172.21.15.181:0/764682829 >> v1:172.21.15.175:6789/0 conn(0x7fb6a415f3c0 legacy=0x7fb6a40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:49.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 -- 172.21.15.181:0/764682829 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a415e2d0 con 0x7fb6a4156a90 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a259c700 1 --2- 172.21.15.181:0/764682829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb69400ba20 tx=0x7fb694006eb0).ready entity=mon.2 client_cookie=6ccdfa495a9fad99 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a159a700 1 -- 172.21.15.181:0/764682829 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940098d0 con 0x7fb6a4156a90 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a159a700 1 -- 172.21.15.181:0/764682829 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694009a30 con 0x7fb6a4156a90 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.269+0000 7fb6a159a700 1 -- 172.21.15.181:0/764682829 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694009db0 con 0x7fb6a4156a90 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 -- 172.21.15.181:0/764682829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 msgr2=0x7fb6a4061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 --2- 172.21.15.181:0/764682829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb69400ba20 tx=0x7fb694006eb0).stop 2020-11-16T22:31:49.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 -- 172.21.15.181:0/764682829 shutdown_connections 2020-11-16T22:31:49.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 --2- 172.21.15.181:0/764682829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a4061100 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:49.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 -- 172.21.15.181:0/764682829 >> 172.21.15.181:0/764682829 conn(0x7fb6a40cf630 msgr2=0x7fb6a40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:49.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 -- 172.21.15.181:0/764682829 shutdown_connections 2020-11-16T22:31:49.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.273+0000 7fb6a359e700 1 -- 172.21.15.181:0/764682829 wait complete. 2020-11-16T22:31:49.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 Processor -- start 2020-11-16T22:31:49.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 -- start start 2020-11-16T22:31:49.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a40615f0 0x7fb6a4150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 0x7fb6a414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4150a30 con 0x7fb6a40615f0 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a416b6d0 con 0x7fb6a415f3c0 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a416b970 con 0x7fb6a4156a90 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 0x7fb6a414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a40615f0 0x7fb6a4150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 0x7fb6a414c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33092/0 (socket says 172.21.15.181:33092) 2020-11-16T22:31:49.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a2d9d700 1 -- 172.21.15.181:0/4020827814 learned_addr learned my addr 172.21.15.181:0/4020827814 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:49.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 -- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 msgr2=0x7fb6a414c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 0x7fb6a414c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:49.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 -- 172.21.15.181:0/4020827814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a40615f0 msgr2=0x7fb6a4150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a40615f0 0x7fb6a4150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 -- 172.21.15.181:0/4020827814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a416bc10 con 0x7fb6a4156a90 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.277+0000 7fb6a1d9b700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a414bb80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb69800c6e0 tx=0x7fb698007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6980072e0 con 0x7fb6a4156a90 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698005530 con 0x7fb6a4156a90 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a416beb0 con 0x7fb6a4156a90 2020-11-16T22:31:49.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6980058b0 con 0x7fb6a4156a90 2020-11-16T22:31:49.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a40b79d0 con 0x7fb6a4156a90 2020-11-16T22:31:49.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a40b7b60 con 0x7fb6a4156a90 2020-11-16T22:31:49.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb69801a060 con 0x7fb6a4156a90 2020-11-16T22:31:49.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 0x7fb678057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb69806cff0 con 0x7fb6a4156a90 2020-11-16T22:31:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6a259c700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 0x7fb678057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.281+0000 7fb6a259c700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 0x7fb678057520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb69400b1b0 tx=0x7fb694017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:49.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.285+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb6a40b7b60 con 0x7fb6a4156a90 2020-11-16T22:31:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.493+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6a40b7b60 con 0x7fb6a4156a90 2020-11-16T22:31:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6927fc700 1 -- 172.21.15.181:0/4020827814 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fb6a40b7b60 con 0x7fb6a4156a90 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 msgr2=0x7fb678057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 0x7fb678057520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb69400b1b0 tx=0x7fb694017000).stop 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 msgr2=0x7fb6a414bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a414bb80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb69800c6e0 tx=0x7fb698007b80).stop 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 shutdown_connections 2020-11-16T22:31:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb678055060 0x7fb678057520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:49.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a415f3c0 0x7fb6a414c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6a40615f0 0x7fb6a4150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 --2- 172.21.15.181:0/4020827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a4156a90 0x7fb6a414bb80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 >> 172.21.15.181:0/4020827814 conn(0x7fb6a40cf630 msgr2=0x7fb6a415ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 shutdown_connections 2020-11-16T22:31:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:49.497+0000 7fb6a359e700 1 -- 172.21.15.181:0/4020827814 wait complete. 2020-11-16T22:31:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 70 2020-11-16T22:31:50.079 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:50.079 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:34.879851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:50.282 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 recursive force 2020-11-16T22:31:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 Processor -- start 2020-11-16T22:31:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 -- start start 2020-11-16T22:31:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe53c145a60 0x7fe53c145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53c0cb950 con 0x7fe53c061a30 2020-11-16T22:31:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53c0cbbf0 con 0x7fe53c0615b0 2020-11-16T22:31:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe5413fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c0cbe90 con 0x7fe53c145a60 2020-11-16T22:31:50.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe53c145a60 0x7fe53c145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:50.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe53c0615b0 0x7fe53c080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47404/0 (socket says 172.21.15.181:47404) 2020-11-16T22:31:50.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53a7fc700 1 -- 172.21.15.181:0/1733804194 learned_addr learned my addr 172.21.15.181:0/1733804194 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:50.637 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:31:50.489+0000 7fd9433d0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v2 rc2020-11-16T22:31:14.259728+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@4},l=6327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x562e1e55ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:31:14.259728+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:31:50.637 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:31:50.493+0000 7fd9493dc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v1 rc2020-11-16T22:31:14.259728+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6327=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x562e1e55f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:31:14.259728+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:31:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53affd700 1 -- 172.21.15.181:0/1733804194 >> v1:172.21.15.181:6789/0 conn(0x7fe53c0615b0 legacy=0x7fe53c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe539ffb700 1 -- 172.21.15.181:0/1733804194 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842035101 0 0) 0x7fe53c0cbbf0 con 0x7fe53c0615b0 2020-11-16T22:31:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53affd700 1 -- 172.21.15.181:0/1733804194 >> v1:172.21.15.175:6789/0 conn(0x7fe53c061a30 legacy=0x7fe53c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53affd700 1 -- 172.21.15.181:0/1733804194 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c0cc130 con 0x7fe53c145a60 2020-11-16T22:31:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe53affd700 1 --2- 172.21.15.181:0/1733804194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c145a60 0x7fe53c145e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fe52c00b9a0 tx=0x7fe52c006df0).ready entity=mon.1 client_cookie=8a545c4f27a17545 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe539ffb700 1 -- 172.21.15.181:0/1733804194 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0097b0 con 0x7fe53c145a60 2020-11-16T22:31:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe539ffb700 1 -- 172.21.15.181:0/1733804194 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c009910 con 0x7fe53c145a60 2020-11-16T22:31:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.437+0000 7fe539ffb700 1 -- 172.21.15.181:0/1733804194 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c009c90 con 0x7fe53c145a60 2020-11-16T22:31:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- 172.21.15.181:0/1733804194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c145a60 msgr2=0x7fe53c145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 --2- 172.21.15.181:0/1733804194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c145a60 0x7fe53c145e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fe52c00b9a0 tx=0x7fe52c006df0).stop 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- 172.21.15.181:0/1733804194 shutdown_connections 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 --2- 172.21.15.181:0/1733804194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c145a60 0x7fe53c145e40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- 172.21.15.181:0/1733804194 >> 172.21.15.181:0/1733804194 conn(0x7fe53c0d68e0 msgr2=0x7fe53c0d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- 172.21.15.181:0/1733804194 shutdown_connections 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- 172.21.15.181:0/1733804194 wait complete. 2020-11-16T22:31:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 Processor -- start 2020-11-16T22:31:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- start start 2020-11-16T22:31:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c0615b0 0x7fe53c172710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe53c145a60 0x7fe53c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0d4880 con 0x7fe53c0615b0 2020-11-16T22:31:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c172d60 con 0x7fe53c061a30 2020-11-16T22:31:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe5413fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c173000 con 0x7fe53c145a60 2020-11-16T22:31:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe53c145a60 0x7fe53c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33104/0 (socket says 172.21.15.181:33104) 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c0615b0 0x7fe53c172710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3978091663 learned_addr learned my addr 172.21.15.181:0/3978091663 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe53c145a60 msgr2=0x7fe53c1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe53c145a60 0x7fe53c1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3978091663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c0615b0 msgr2=0x7fe53c172710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c0615b0 0x7fe53c172710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3978091663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c1732a0 con 0x7fe53c061a30 2020-11-16T22:31:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.441+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fe530007fd0 tx=0x7fe5300079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 -- 172.21.15.181:0/3978091663 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300072e0 con 0x7fe53c061a30 2020-11-16T22:31:51.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 -- 172.21.15.181:0/3978091663 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530007440 con 0x7fe53c061a30 2020-11-16T22:31:51.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c0d4ab0 con 0x7fe53c061a30 2020-11-16T22:31:51.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 -- 172.21.15.181:0/3978091663 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530005820 con 0x7fe53c061a30 2020-11-16T22:31:51.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c140480 con 0x7fe53c061a30 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 -- 172.21.15.181:0/3978091663 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe53001a060 con 0x7fe53c061a30 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 0x7fe518057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe522ffd700 1 -- 172.21.15.181:0/3978091663 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe53006d140 con 0x7fe53c061a30 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe53affd700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 0x7fe518057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.445+0000 7fe53affd700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 0x7fe518057520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fe52c00bf10 tx=0x7fe52c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:51.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 Processor -- start 2020-11-16T22:31:51.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 -- start start 2020-11-16T22:31:51.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1328c60 0x15ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1440560 con 0x1396860 2020-11-16T22:31:51.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f49e0 con 0x15af520 2020-11-16T22:31:51.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13b47a0 con 0x1328c60 2020-11-16T22:31:51.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33110/0 (socket says 172.21.15.181:33110) 2020-11-16T22:31:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1221483527 learned_addr learned my addr 172.21.15.181:0/1221483527 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1328c60 0x15ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53affd700 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1328c60 msgr2=0x15ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1328c60 0x15ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1221483527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 msgr2=0x15add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1221483527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13af650 con 0x15af520 2020-11-16T22:31:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe534002d30 tx=0x7fe534007e20).ready entity=mon.1 client_cookie=48d44b82b002ec86 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe513fff700 1 -- 172.21.15.181:0/1221483527 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534002280 con 0x15af520 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe513fff700 1 -- 172.21.15.181:0/1221483527 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5340023e0 con 0x15af520 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe513fff700 1 -- 172.21.15.181:0/1221483527 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534002550 con 0x15af520 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 msgr2=0x1609d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.449+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe534002d30 tx=0x7fe534007e20).stop 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1221483527 shutdown_connections 2020-11-16T22:31:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x1609d40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1221483527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1328c60 0x15ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1221483527 >> 172.21.15.181:0/1221483527 conn(0x1564ef0 msgr2=0x1609630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1221483527 shutdown_connections 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1221483527 wait complete. 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- start start 2020-11-16T22:31:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 Processor -- start 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1328c60 0x1606490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x15d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e1790 con 0x1328c60 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e18d0 con 0x15af520 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14e2b40 con 0x1396860 2020-11-16T22:31:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 10 client.0 ll_register_callbacks cb 0x14d7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x15d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40960/0 (socket says 172.21.15.181:40960) 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3094800193 learned_addr learned my addr 172.21.15.181:0/3094800193 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53affd700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1328c60 0x1606490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 msgr2=0x15d61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x15d61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3094800193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1328c60 msgr2=0x1606490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1328c60 0x1606490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 -- 172.21.15.181:0/3094800193 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x14d2330 con 0x1396860 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fe530043510 tx=0x7fe530010000).ready entity=mon.2 client_cookie=acc241da57dc970f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530043060 con 0x1396860 2020-11-16T22:31:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe530045fc0 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15a9ec0 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe530046390 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 10 client.6140 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x1308080 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15a9ec0 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fe53006b910 con 0x1396860 2020-11-16T22:31:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 10 client.6140 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 10 client.6140 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 20 client.6140 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15ef2c0 0x15cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 4 client.6140 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x13d2d00 con 0x15ef2c0 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.453+0000 7fe53affd700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15ef2c0 0x15cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe5117fa700 10 client.6140 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe53affd700 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15ef2c0 0x15cfe10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4c11c38b94343d73 server_cookie=4a0c4871b0a057dc in_seq=0 out_seq=0 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe5117fa700 1 -- 172.21.15.181:0/3094800193 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x13d2d00 con 0x15ef2c0 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe5117fa700 10 client.6140 handle_command_reply: tid=0 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 client.6140 shutdown 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 msgr2=0x15d5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:51.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fe530043510 tx=0x7fe530010000).stop 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 shutdown_connections 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15ef2c0 0x15cfe10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15af520 0x15d61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1328c60 0x1606490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/3094800193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1396860 0x15d5cf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe5117fa700 0 client.6140 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:51.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 >> 172.21.15.181:0/3094800193 conn(0x1564ef0 msgr2=0x15ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 shutdown_connections 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 1 -- 172.21.15.181:0/3094800193 wait complete. 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.457+0000 7fe54fe8d740 20 client.6140 trim_cache size 0 max 16384 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 Processor -- start 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 -- start start 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x15da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14b9ef0 con 0x1396860 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16093b0 con 0x1328c60 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1608ee0 con 0x1564ef0 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe53affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe53affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40966/0 (socket says 172.21.15.181:40966) 2020-11-16T22:31:51.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe53affd700 1 -- 172.21.15.181:0/4018652423 learned_addr learned my addr 172.21.15.181:0/4018652423 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x15da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.477+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 -- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 msgr2=0x15da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x15da3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 -- 172.21.15.181:0/4018652423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 msgr2=0x15ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 -- 172.21.15.181:0/4018652423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080081d0 con 0x1564ef0 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe53affd700 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe52c006fd0 tx=0x7fe52c031280).ready entity=mon.2 client_cookie=c3b78cfa00360783 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe520ff9700 1 -- 172.21.15.181:0/4018652423 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011d50 con 0x1564ef0 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe520ff9700 1 -- 172.21.15.181:0/4018652423 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c01e430 con 0x1564ef0 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe520ff9700 1 -- 172.21.15.181:0/4018652423 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c01e7b0 con 0x1564ef0 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 msgr2=0x15d9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:51.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe52c006fd0 tx=0x7fe52c031280).stop 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- 172.21.15.181:0/4018652423 shutdown_connections 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x15ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d9f00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/4018652423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x15da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- 172.21.15.181:0/4018652423 >> 172.21.15.181:0/4018652423 conn(0x7fe508002320 msgr2=0x15ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- 172.21.15.181:0/4018652423 shutdown_connections 2020-11-16T22:31:51.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- 172.21.15.181:0/4018652423 wait complete. 2020-11-16T22:31:52.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 -- start start 2020-11-16T22:31:52.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.481+0000 7fe54fe8d740 1 Processor -- start 2020-11-16T22:31:52.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x16094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x166afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1614810 con 0x1396860 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1614a40 con 0x1328c60 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1614ce0 con 0x1564ef0 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 10 client.0 ll_register_callbacks cb 0x15b17d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x16094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40974/0 (socket says 172.21.15.181:40974) 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1296972743 learned_addr learned my addr 172.21.15.181:0/1296972743 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53a7fc700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x166afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 msgr2=0x16094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x16094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1296972743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 msgr2=0x166afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x166afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 -- 172.21.15.181:0/1296972743 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x13467d0 con 0x1564ef0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53b7fe700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fe534009000 tx=0x7fe534017000).ready entity=mon.2 client_cookie=4a0fe0ca089faaf5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534007b50 con 0x1564ef0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe534007930 con 0x1564ef0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5340023e0 con 0x1564ef0 2020-11-16T22:31:52.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5340029b0 con 0x1564ef0 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 10 client.6146 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x1351eb0 con 0x1564ef0 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5340023e0 con 0x1564ef0 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fe53400ed20 con 0x1564ef0 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 10 client.6146 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:52.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 10 client.6146 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 20 client.6146 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15af520 0x15ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 4 client.6146 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x13da000 con 0x15af520 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53affd700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15af520 0x15ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe5117fa700 10 client.6146 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:52.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.485+0000 7fe53affd700 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15af520 0x15ea9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cfb4d38312849269 server_cookie=579afbc921a64535 in_seq=0 out_seq=0 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5117fa700 1 -- 172.21.15.181:0/1296972743 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x13da000 con 0x15af520 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5117fa700 10 client.6146 handle_command_reply: tid=0 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 client.6146 shutdown 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 msgr2=0x15d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fe534009000 tx=0x7fe534017000).stop 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 shutdown_connections 2020-11-16T22:31:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x15af520 0x15ea9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1564ef0 0x15d7f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1328c60 0x16094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 --2- 172.21.15.181:0/1296972743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1396860 0x166afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 >> 172.21.15.181:0/1296972743 conn(0x7fe508002320 msgr2=0x15ec360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:52.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5117fa700 0 client.6146 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 shutdown_connections 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 1 -- 172.21.15.181:0/1296972743 wait complete. 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe54fe8d740 20 client.6146 trim_cache size 0 max 16384 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 msgr2=0x7fe518057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 0x7fe518057520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fe52c00bf10 tx=0x7fe52c015000).stop 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 msgr2=0x7fe53c161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fe530007fd0 tx=0x7fe5300079e0).stop 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 shutdown_connections 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe518055060 0x7fe518057520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe53c145a60 0x7fe53c1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53c0615b0 0x7fe53c172710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 --2- 172.21.15.181:0/3978091663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe53c061a30 0x7fe53c161bb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 >> 172.21.15.181:0/3978091663 conn(0x7fe53c0d68e0 msgr2=0x7fe53c1422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:52.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 shutdown_connections 2020-11-16T22:31:52.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fe5413fc700 1 -- 172.21.15.181:0/3978091663 wait complete. 2020-11-16T22:31:52.289 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:31:50.489+0000 7f71957c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1) n(v4 rc2020-11-16T22:31:14.259728+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@5},l=6327 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5650ff773b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v29>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:31:14.259728+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:31:52.289 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:31:50.493+0000 7f719b7cf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1) n(v3 rc2020-11-16T22:31:14.259728+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6327=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5650ff774100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v29>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:31:14.259728+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:31:52.289 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:31:52.289 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2020-11-16T22:31:52.290 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "7dd00246-57fd-4ba3-937c-4a270ecf192a", 2020-11-16T22:31:52.290 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2020-11-16T22:31:52.290 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:31:52.290 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fd27412f700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@4},l=6327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x559d77a39700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:31:52.494 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:31:50.489+0000 7fd27412f700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/AsLsXsFsx@2},l=6327 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x559d77aea000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:31:52.494 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:52.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 Processor -- start 2020-11-16T22:31:52.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 -- start start 2020-11-16T22:31:52.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc53c062960 0x7fc53c0644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53c0c8db0 con 0x7fc53c161860 2020-11-16T22:31:52.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53c165030 con 0x7fc53c161480 2020-11-16T22:31:52.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc540a0f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c0c8330 con 0x7fc53c062960 2020-11-16T22:31:52.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc53c161480 0x7fc53c164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47442/0 (socket says 172.21.15.181:47442) 2020-11-16T22:31:52.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53b7fe700 1 -- 172.21.15.181:0/3921656342 learned_addr learned my addr 172.21.15.181:0/3921656342 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:52.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53bfff700 1 --2- 172.21.15.181:0/3921656342 >> v2:172.21.15.175:3300/0 conn(0x7fc53c062960 0x7fc53c0644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973185834 0 0) 0x7fc53c165030 con 0x7fc53c161480 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc520003240 con 0x7fc53c161480 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107553323 0 0) 0x7fc53c0c8db0 con 0x7fc53c161860 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc53c165030 con 0x7fc53c161860 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53bfff700 1 -- 172.21.15.181:0/3921656342 >> v1:172.21.15.181:6789/0 conn(0x7fc53c161480 legacy=0x7fc53c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53bfff700 1 -- 172.21.15.181:0/3921656342 >> v1:172.21.15.175:6789/0 conn(0x7fc53c161860 legacy=0x7fc53c062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.645+0000 7fc53bfff700 1 -- 172.21.15.181:0/3921656342 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c0c85d0 con 0x7fc53c062960 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc53bfff700 1 --2- 172.21.15.181:0/3921656342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c062960 0x7fc53c0644e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc53400b9a0 tx=0x7fc534006df0).ready entity=mon.0 client_cookie=59c7d08db37c55bd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534009680 con 0x7fc53c062960 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340097e0 con 0x7fc53c062960 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc53a7fc700 1 -- 172.21.15.181:0/3921656342 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534009b60 con 0x7fc53c062960 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- 172.21.15.181:0/3921656342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c062960 msgr2=0x7fc53c0644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3921656342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c062960 0x7fc53c0644e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc53400b9a0 tx=0x7fc534006df0).stop 2020-11-16T22:31:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- 172.21.15.181:0/3921656342 shutdown_connections 2020-11-16T22:31:52.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3921656342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c062960 0x7fc53c0644e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- 172.21.15.181:0/3921656342 >> 172.21.15.181:0/3921656342 conn(0x7fc53c0d2dc0 msgr2=0x7fc53c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:52.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- 172.21.15.181:0/3921656342 shutdown_connections 2020-11-16T22:31:52.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- 172.21.15.181:0/3921656342 wait complete. 2020-11-16T22:31:52.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 Processor -- start 2020-11-16T22:31:52.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.649+0000 7fc540a0f700 1 -- start start 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c161480 0x7fc53c176ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c161860 0x7fc53c1773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c145ff0 con 0x7fc53c161480 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c146290 con 0x7fc53c161860 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c146530 con 0x7fc53c062960 2020-11-16T22:31:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40986/0 (socket says 172.21.15.181:40986) 2020-11-16T22:31:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 -- 172.21.15.181:0/3223427905 learned_addr learned my addr 172.21.15.181:0/3223427905 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53bfff700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c161860 0x7fc53c1773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53affd700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c161480 0x7fc53c176ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 -- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c161860 msgr2=0x7fc53c1773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c161860 0x7fc53c1773b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 -- 172.21.15.181:0/3223427905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c161480 msgr2=0x7fc53c176ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c161480 0x7fc53c176ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 -- 172.21.15.181:0/3223427905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c1467d0 con 0x7fc53c062960 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc53b7fe700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fc52c00cae0 tx=0x7fc52c007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c005b80 con 0x7fc53c062960 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c005ce0 con 0x7fc53c062960 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c146a70 con 0x7fc53c062960 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007320 con 0x7fc53c062960 2020-11-16T22:31:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c160360 con 0x7fc53c062960 2020-11-16T22:31:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.653+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c044f60 con 0x7fc53c062960 2020-11-16T22:31:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.657+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc52c010030 con 0x7fc53c062960 2020-11-16T22:31:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.657+0000 7fc5337fe700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 0x7fc50c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.657+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc52c06dd90 con 0x7fc53c062960 2020-11-16T22:31:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.657+0000 7fc53affd700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 0x7fc50c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.657+0000 7fc53affd700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 0x7fc50c057520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fc53c0cf930 tx=0x7fc524007990).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.665+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc52c043ff0 con 0x7fc53c062960 2020-11-16T22:31:52.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc53c0671e0 con 0x7fc53c062960 2020-11-16T22:31:52.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc5337fe700 1 -- 172.21.15.181:0/3223427905 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fc52c047610 con 0x7fc53c062960 2020-11-16T22:31:52.853 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:52.854 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:34.879851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:52.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 msgr2=0x7fc50c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 0x7fc50c057520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fc53c0cf930 tx=0x7fc524007990).stop 2020-11-16T22:31:52.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 msgr2=0x7fc53c15fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fc52c00cae0 tx=0x7fc52c007ef0).stop 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 shutdown_connections 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc50c055060 0x7fc50c057520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c161860 0x7fc53c1773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c062960 0x7fc53c15fc50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 --2- 172.21.15.181:0/3223427905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53c161480 0x7fc53c176ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.869+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 >> 172.21.15.181:0/3223427905 conn(0x7fc53c0d2dc0 msgr2=0x7fc53c1472c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:52.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.873+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 shutdown_connections 2020-11-16T22:31:52.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:52.873+0000 7fc540a0f700 1 -- 172.21.15.181:0/3223427905 wait complete. 2020-11-16T22:31:52.857 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 70 2020-11-16T22:31:52.868 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:31:52.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.001+0000 7f0664227700 1 Processor -- start 2020-11-16T22:31:52.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.001+0000 7f0664227700 1 -- start start 2020-11-16T22:31:52.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f065c15f5b0 0x7f065c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c160e40 con 0x7f065c0615f0 2020-11-16T22:31:52.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c1609a0 con 0x7f065c15f5b0 2020-11-16T22:31:52.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c160ba0 con 0x7f065c0d1830 2020-11-16T22:31:52.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:40998/0 (socket says 172.21.15.181:40998) 2020-11-16T22:31:52.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 -- 172.21.15.181:0/3679314823 learned_addr learned my addr 172.21.15.181:0/3679314823 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:52.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0661fc3700 1 --2- 172.21.15.181:0/3679314823 >> v2:172.21.15.181:3300/0 conn(0x7f065c15f5b0 0x7f065c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 -- 172.21.15.181:0/3679314823 >> v2:172.21.15.181:3300/0 conn(0x7f065c15f5b0 msgr2=0x7f065c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 --2- 172.21.15.181:0/3679314823 >> v2:172.21.15.181:3300/0 conn(0x7f065c15f5b0 0x7f065c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 -- 172.21.15.181:0/3679314823 >> v1:172.21.15.175:6789/0 conn(0x7f065c0615f0 legacy=0x7f065c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 -- 172.21.15.181:0/3679314823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c060970 con 0x7f065c0d1830 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f06627c4700 1 --2- 172.21.15.181:0/3679314823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0654009d50 tx=0x7f0654009290).ready entity=mon.2 client_cookie=9f4e36ab1298826 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0660fc1700 1 -- 172.21.15.181:0/3679314823 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065400bbc0 con 0x7f065c0d1830 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0660fc1700 1 -- 172.21.15.181:0/3679314823 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065400bd20 con 0x7f065c0d1830 2020-11-16T22:31:52.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0660fc1700 1 -- 172.21.15.181:0/3679314823 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654011610 con 0x7f065c0d1830 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- 172.21.15.181:0/3679314823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 msgr2=0x7f065c0629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 --2- 172.21.15.181:0/3679314823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0654009d50 tx=0x7f0654009290).stop 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- 172.21.15.181:0/3679314823 shutdown_connections 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 --2- 172.21.15.181:0/3679314823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c0629b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 --2- 172.21.15.181:0/3679314823 >> v2:172.21.15.181:3300/0 conn(0x7f065c15f5b0 0x7f065c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- 172.21.15.181:0/3679314823 >> 172.21.15.181:0/3679314823 conn(0x7f065c0d69a0 msgr2=0x7f065c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:52.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- 172.21.15.181:0/3679314823 shutdown_connections 2020-11-16T22:31:52.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.005+0000 7f0664227700 1 -- 172.21.15.181:0/3679314823 wait complete. 2020-11-16T22:31:52.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 Processor -- start 2020-11-16T22:31:52.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- start start 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065c0615f0 0x7f065c162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c1707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c162780 con 0x7f065c0615f0 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0d6350 con 0x7f065c15f5b0 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1629c0 con 0x7f065c0d1830 2020-11-16T22:31:52.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06617c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c1707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06617c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c1707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41002/0 (socket says 172.21.15.181:41002) 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33162/0 (socket says 172.21.15.181:33162) 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06617c2700 1 -- 172.21.15.181:0/1527724474 learned_addr learned my addr 172.21.15.181:0/1527724474 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065c0615f0 0x7f065c162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 -- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 msgr2=0x7f065c1707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c1707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 -- 172.21.15.181:0/1527724474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065c0615f0 msgr2=0x7f065c162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065c0615f0 0x7f065c162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 -- 172.21.15.181:0/1527724474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c162c60 con 0x7f065c15f5b0 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f06627c4700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0654009d50 tx=0x7f0654016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0649ffb700 1 -- 172.21.15.181:0/1527724474 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065400bbc0 con 0x7f065c15f5b0 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0649ffb700 1 -- 172.21.15.181:0/1527724474 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065400be90 con 0x7f065c15f5b0 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0649ffb700 1 -- 172.21.15.181:0/1527724474 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654010cf0 con 0x7f065c15f5b0 2020-11-16T22:31:52.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c162f00 con 0x7f065c15f5b0 2020-11-16T22:31:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.009+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c1636e0 con 0x7f065c15f5b0 2020-11-16T22:31:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.013+0000 7f0649ffb700 1 -- 172.21.15.181:0/1527724474 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0654017070 con 0x7f065c15f5b0 2020-11-16T22:31:52.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.013+0000 7f0649ffb700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 0x7f0634057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.013+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 0x7f0634057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.013+0000 7f0649ffb700 1 -- 172.21.15.181:0/1527724474 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f065406fdd0 con 0x7f065c15f5b0 2020-11-16T22:31:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.013+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 0x7f0634057520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f0658006fd0 tx=0x7f0658008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:53.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 Processor -- start 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 -- start start 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a33470 0x2a8dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x28c44b0 con 0x27acbb0 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2978930 con 0x281a7b0 2020-11-16T22:31:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x28386f0 con 0x2a33470 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0661fc3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0661fc3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33166/0 (socket says 172.21.15.181:33166) 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0661fc3700 1 -- 172.21.15.181:0/2319769993 learned_addr learned my addr 172.21.15.181:0/2319769993 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f06627c4700 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a33470 0x2a8dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f06617c2700 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.017+0000 7f0661fc3700 1 -- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a33470 msgr2=0x2a8dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0661fc3700 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a33470 0x2a8dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0661fc3700 1 -- 172.21.15.181:0/2319769993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 msgr2=0x2a70530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0661fc3700 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a70530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0661fc3700 1 -- 172.21.15.181:0/2319769993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28335a0 con 0x281a7b0 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0661fc3700 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0658013ad0 tx=0x7f0658014000).ready entity=mon.1 client_cookie=8345a74296bcb99 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f063affd700 1 -- 172.21.15.181:0/2319769993 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800b720 con 0x281a7b0 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f063affd700 1 -- 172.21.15.181:0/2319769993 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065800b880 con 0x281a7b0 2020-11-16T22:31:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f063affd700 1 -- 172.21.15.181:0/2319769993 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800bc40 con 0x281a7b0 2020-11-16T22:31:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 msgr2=0x2a31cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0658013ad0 tx=0x7f0658014000).stop 2020-11-16T22:31:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- 172.21.15.181:0/2319769993 shutdown_connections 2020-11-16T22:31:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2a33470 0x2a8dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x281a7b0 0x2a31cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- 172.21.15.181:0/2319769993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a70530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- 172.21.15.181:0/2319769993 >> 172.21.15.181:0/2319769993 conn(0x29e8e40 msgr2=0x2a8d580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- 172.21.15.181:0/2319769993 shutdown_connections 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- 172.21.15.181:0/2319769993 wait complete. 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- start start 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 Processor -- start 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a33470 0x2a99500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x29656e0 con 0x27acbb0 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2965820 con 0x2a33470 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2966a90 con 0x281a7b0 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f0672cb8740 10 client.0 ll_register_callbacks cb 0x295bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f06617c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f06617c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41016/0 (socket says 172.21.15.181:41016) 2020-11-16T22:31:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f06617c2700 1 -- 172.21.15.181:0/1529693423 learned_addr learned my addr 172.21.15.181:0/1529693423 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.021+0000 7f06627c4700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a33470 0x2a99500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 -- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a33470 msgr2=0x2a99500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a33470 0x2a99500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 -- 172.21.15.181:0/1529693423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 msgr2=0x2a9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 -- 172.21.15.181:0/1529693423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2956280 con 0x281a7b0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f06617c2700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f064c011850 tx=0x7f064c009040).ready entity=mon.2 client_cookie=18a97a085a6ffae8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c003320 con 0x281a7b0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f064c013610 con 0x281a7b0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2a2de10 con 0x281a7b0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f064c01ba30 con 0x281a7b0 2020-11-16T22:31:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 10 client.6161 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x278bfd0 con 0x281a7b0 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2a2de10 con 0x281a7b0 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f064c01bb90 con 0x281a7b0 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 10 client.6161 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 10 client.6161 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 20 client.6161 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a8a7b0 0x2a9b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 4 client.6161 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2856c50 con 0x2a8a7b0 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a8a7b0 0x2a9b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 10 client.6161 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a8a7b0 0x2a9b600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ddeab67dc13eae8b server_cookie=74369c3609ba890f in_seq=0 out_seq=0 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 1 -- 172.21.15.181:0/1529693423 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2856c50 con 0x2a8a7b0 2020-11-16T22:31:53.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 10 client.6161 handle_command_reply: tid=0 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 client.6161 shutdown 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 msgr2=0x2a89300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f064c011850 tx=0x7f064c009040).stop 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 shutdown_connections 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a8a7b0 0x2a9b600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2a33470 0x2a99500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27acbb0 0x2a9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1529693423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a89300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f062ffff700 0 client.6161 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 >> 172.21.15.181:0/1529693423 conn(0x29e8e40 msgr2=0x2a70620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 shutdown_connections 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.025+0000 7f0672cb8740 1 -- 172.21.15.181:0/1529693423 wait complete. 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.029+0000 7f0672cb8740 20 client.6161 trim_cache size 0 max 16384 2020-11-16T22:31:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 Processor -- start 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 -- start start 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 0x2a8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 0x2a9a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x293de40 con 0x29e8e40 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a8a520 con 0x27acbb0 2020-11-16T22:31:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a72c30 con 0x281a7b0 2020-11-16T22:31:53.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41022/0 (socket says 172.21.15.181:41022) 2020-11-16T22:31:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 -- 172.21.15.181:0/1028048291 learned_addr learned my addr 172.21.15.181:0/1028048291 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06617c2700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 0x2a9a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 -- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 msgr2=0x2a9a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 0x2a9a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 0x2a8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 -- 172.21.15.181:0/1028048291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 msgr2=0x2a8a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 0x2a8a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 -- 172.21.15.181:0/1028048291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2a99860 con 0x281a7b0 2020-11-16T22:31:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06617c2700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 0x2a9a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 0x2a8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f06627c4700 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f0654049930 tx=0x7f0654012000).ready entity=mon.2 client_cookie=83f4e456f5a4d90c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f063b7fe700 1 -- 172.21.15.181:0/1028048291 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654036070 con 0x281a7b0 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f063b7fe700 1 -- 172.21.15.181:0/1028048291 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654025ee0 con 0x281a7b0 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f063b7fe700 1 -- 172.21.15.181:0/1028048291 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654049470 con 0x281a7b0 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.049+0000 7f0672cb8740 1 -- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 msgr2=0x2a30d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f0654049930 tx=0x7f0654012000).stop 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 -- 172.21.15.181:0/1028048291 shutdown_connections 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x281a7b0 0x2a30d40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29e8e40 0x2a8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1028048291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27acbb0 0x2a9a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 -- 172.21.15.181:0/1028048291 >> 172.21.15.181:0/1028048291 conn(0x7f0628002320 msgr2=0x2a705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 -- 172.21.15.181:0/1028048291 shutdown_connections 2020-11-16T22:31:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 -- 172.21.15.181:0/1028048291 wait complete. 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 -- start start 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 Processor -- start 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x281a7b0 0x2aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.053+0000 7f0672cb8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 0x2a9e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a31470 con 0x281a7b0 2020-11-16T22:31:53.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ad8f60 con 0x29e8e40 2020-11-16T22:31:53.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a311d0 con 0x27acbb0 2020-11-16T22:31:53.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41028/0 (socket says 172.21.15.181:41028) 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 -- 172.21.15.181:0/1234016628 learned_addr learned my addr 172.21.15.181:0/1234016628 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f06627c4700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 0x2a9e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 10 client.0 ll_register_callbacks cb 0x2a727e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f06617c2700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x281a7b0 0x2aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 -- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 msgr2=0x2a9e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 0x2a9e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 -- 172.21.15.181:0/1234016628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x281a7b0 msgr2=0x2aa0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x281a7b0 0x2aa0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 -- 172.21.15.181:0/1234016628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a9b2e0 con 0x27acbb0 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f06627c4700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 0x2a9e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0661fc3700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0658015010 tx=0x7f0658016000).ready entity=mon.2 client_cookie=8b217ad3d98a578a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065802c0c0 con 0x27acbb0 2020-11-16T22:31:53.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f065800bc00 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f065800b880 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f065802c620 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 10 client.6167 fetch_fsmap learned FSMap version 70 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x7f06280081d0 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065800b880 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 70) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f065802c780 con 0x27acbb0 2020-11-16T22:31:53.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 10 client.6167 fetch_fsmap finished waiting for FSMap version 70 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 10 client.6167 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 20 client.6167 populate_metadata read hostname 'smithi181' 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a33470 0x2a5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 4 client.6167 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x285df50 con 0x2a33470 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f06617c2700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a33470 0x2a5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:53.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 10 client.6167 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f06617c2700 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a33470 0x2a5bda0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee3c6506452b0c3d server_cookie=74225eb028652e3e in_seq=0 out_seq=0 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 1 -- 172.21.15.181:0/1234016628 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x285df50 con 0x2a33470 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 10 client.6167 handle_command_reply: tid=0 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 client.6167 shutdown 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 msgr2=0x2ad8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0658015010 tx=0x7f0658016000).stop 2020-11-16T22:31:53.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 shutdown_connections 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2a33470 0x2a5bda0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29e8e40 0x2a9e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f062ffff700 0 client.6167 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27acbb0 0x2ad8a70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 --2- 172.21.15.181:0/1234016628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x281a7b0 0x2aa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 >> 172.21.15.181:0/1234016628 conn(0x7f0628002320 msgr2=0x2a34180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:53.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 shutdown_connections 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 1 -- 172.21.15.181:0/1234016628 wait complete. 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0672cb8740 20 client.6167 trim_cache size 0 max 16384 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 msgr2=0x7f0634057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 0x7f0634057520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f0658006fd0 tx=0x7f0658008040).stop 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 msgr2=0x7f065c170cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.057+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0654009d50 tx=0x7f0654016000).stop 2020-11-16T22:31:53.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 shutdown_connections 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0634055060 0x7f0634057520 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065c15f5b0 0x7f065c170cb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f065c0615f0 0x7f065c162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 --2- 172.21.15.181:0/1527724474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f065c0d1830 0x7f065c1707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 >> 172.21.15.181:0/1527724474 conn(0x7f065c0d69a0 msgr2=0x7f065c061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 shutdown_connections 2020-11-16T22:31:53.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:53.061+0000 7f0664227700 1 -- 172.21.15.181:0/1527724474 wait complete. 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "7dd00246-57fd-4ba3-937c-4a270ecf192a": { 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:31:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "7dd00246-57fd-4ba3-937c-4a270ecf192a", 2020-11-16T22:31:53.848 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:31:53.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:31:53.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:31:53.848 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:31:53.848 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:31:55.701 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:31:55.716+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:31:58.850 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:31:58.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.992+0000 7fa3e1195700 1 Processor -- start 2020-11-16T22:31:58.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.992+0000 7fa3e1195700 1 -- start start 2020-11-16T22:31:58.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3dc1614b0 0x7fa3dc161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc1641e0 0x7fa3dc164650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc165160 con 0x7fa3dc0c8880 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc164ec0 con 0x7fa3dc1614b0 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0c7c00 con 0x7fa3dc1641e0 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc1641e0 0x7fa3dc164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc1641e0 0x7fa3dc164650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33198/0 (socket says 172.21.15.181:33198) 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 -- 172.21.15.181:0/1470783551 learned_addr learned my addr 172.21.15.181:0/1470783551 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/1470783551 >> v2:172.21.15.175:3300/0 conn(0x7fa3dc1614b0 0x7fa3dc161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 -- 172.21.15.181:0/1470783551 >> v2:172.21.15.175:3300/0 conn(0x7fa3dc1614b0 msgr2=0x7fa3dc161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 --2- 172.21.15.181:0/1470783551 >> v2:172.21.15.175:3300/0 conn(0x7fa3dc1614b0 0x7fa3dc161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 -- 172.21.15.181:0/1470783551 >> v1:172.21.15.175:6789/0 conn(0x7fa3dc0c8880 legacy=0x7fa3dc1656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 -- 172.21.15.181:0/1470783551 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0c7ea0 con 0x7fa3dc1641e0 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e0994700 1 --2- 172.21.15.181:0/1470783551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc164650 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa3dc0d0450 tx=0x7fa3d4009290).ready entity=mon.1 client_cookie=5fbc7f6e21b3ca3e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3daffd700 1 -- 172.21.15.181:0/1470783551 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400bbc0 con 0x7fa3dc1641e0 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3daffd700 1 -- 172.21.15.181:0/1470783551 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400bd20 con 0x7fa3dc1641e0 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3daffd700 1 -- 172.21.15.181:0/1470783551 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4011610 con 0x7fa3dc1641e0 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- 172.21.15.181:0/1470783551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1641e0 msgr2=0x7fa3dc164650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 --2- 172.21.15.181:0/1470783551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc164650 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa3dc0d0450 tx=0x7fa3d4009290).stop 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- 172.21.15.181:0/1470783551 shutdown_connections 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 --2- 172.21.15.181:0/1470783551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc164650 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 --2- 172.21.15.181:0/1470783551 >> v2:172.21.15.175:3300/0 conn(0x7fa3dc1614b0 0x7fa3dc161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- 172.21.15.181:0/1470783551 >> 172.21.15.181:0/1470783551 conn(0x7fa3dc0cf630 msgr2=0x7fa3dc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- 172.21.15.181:0/1470783551 shutdown_connections 2020-11-16T22:31:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:58.996+0000 7fa3e1195700 1 -- 172.21.15.181:0/1470783551 wait complete. 2020-11-16T22:31:58.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 Processor -- start 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- start start 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 0x7fa3dc0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b5b20 con 0x7fa3dc1641e0 2020-11-16T22:31:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b5d80 con 0x7fa3dc1614b0 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b6020 con 0x7fa3dc0c8880 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41042/0 (socket says 172.21.15.181:41042) 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3dbfff700 1 -- 172.21.15.181:0/283601395 learned_addr learned my addr 172.21.15.181:0/283601395 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 0x7fa3dc0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e0994700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 -- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 msgr2=0x7fa3dc0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:58.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 -- 172.21.15.181:0/283601395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3dc1641e0 msgr2=0x7fa3dc0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc0c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 -- 172.21.15.181:0/283601395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0b62c0 con 0x7fa3dc1614b0 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 0x7fa3dc0c8f30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa3d000c710 tx=0x7fa3d0007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:58.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0007020 con 0x7fa3dc1614b0 2020-11-16T22:31:58.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0007180 con 0x7fa3dc1614b0 2020-11-16T22:31:58.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00064f0 con 0x7fa3dc1614b0 2020-11-16T22:31:58.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc0b6560 con 0x7fa3dc1614b0 2020-11-16T22:31:58.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.000+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc0b6aa0 con 0x7fa3dc1614b0 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc0b7430 con 0x7fa3dc1614b0 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa3d0010030 con 0x7fa3dc1614b0 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3bbfff700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 0x7fa3b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa3d006d2b0 con 0x7fa3dc1614b0 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 0x7fa3b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:58.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.004+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 0x7fa3b4057520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa3cc00bb50 tx=0x7fa3cc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:58.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.012+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa3d0043510 con 0x7fa3dc1614b0 2020-11-16T22:31:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.216+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3dc044f60 con 0x7fa3dc1614b0 2020-11-16T22:31:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.216+0000 7fa3bbfff700 1 -- 172.21.15.181:0/283601395 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7fa3d0001e60 con 0x7fa3dc1614b0 2020-11-16T22:31:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:31:59.204 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:31:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 msgr2=0x7fa3b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 0x7fa3b4057520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa3cc00bb50 tx=0x7fa3cc008040).stop 2020-11-16T22:31:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 msgr2=0x7fa3dc0c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 0x7fa3dc0c8f30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa3d000c710 tx=0x7fa3d0007b90).stop 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 shutdown_connections 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa3b4055060 0x7fa3b4057520 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3dc1641e0 0x7fa3dc0c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3dc0c8880 0x7fa3dc0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 --2- 172.21.15.181:0/283601395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc1614b0 0x7fa3dc0c8f30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 >> 172.21.15.181:0/283601395 conn(0x7fa3dc0cf630 msgr2=0x7fa3dc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 shutdown_connections 2020-11-16T22:31:59.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.220+0000 7fa3e1195700 1 -- 172.21.15.181:0/283601395 wait complete. 2020-11-16T22:31:59.208 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:31:59.224 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:31:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 Processor -- start 2020-11-16T22:31:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 -- start start 2020-11-16T22:31:59.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcd20161480 0x7fcd20164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 0x7fcd20062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd200c8db0 con 0x7fcd20161860 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200c8330 con 0x7fcd20161480 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd255f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200c85d0 con 0x7fcd20164e10 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd24df2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcd20161860 0x7fcd20062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47512/0 (socket says 172.21.15.181:47512) 2020-11-16T22:31:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd24df2700 1 -- 172.21.15.181:0/3964249439 learned_addr learned my addr 172.21.15.181:0/3964249439 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:31:59.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 0x7fcd20062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3964249439 >> v2:172.21.15.175:3300/0 conn(0x7fcd20161480 0x7fcd20164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.360+0000 7fcd1effd700 1 -- 172.21.15.181:0/3964249439 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428142027 0 0) 0x7fcd200c8db0 con 0x7fcd20161860 2020-11-16T22:31:59.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1effd700 1 -- 172.21.15.181:0/3964249439 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd00003240 con 0x7fcd20161860 2020-11-16T22:31:59.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3964249439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 msgr2=0x7fcd20062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:59.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 0x7fcd20062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3964249439 >> v1:172.21.15.181:6789/0 conn(0x7fcd20161860 legacy=0x7fcd20062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3964249439 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200c8870 con 0x7fcd20161480 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 0x7fcd20062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd20164920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcd200c8260 tx=0x7fcd100091d0).ready entity=mon.0 client_cookie=aed5fd650a88aeb9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1effd700 1 -- 172.21.15.181:0/3964249439 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000b9c0 con 0x7fcd20161480 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1effd700 1 -- 172.21.15.181:0/3964249439 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000bb20 con 0x7fcd20161480 2020-11-16T22:31:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1effd700 1 -- 172.21.15.181:0/3964249439 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000bea0 con 0x7fcd20161480 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- 172.21.15.181:0/3964249439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 msgr2=0x7fcd20164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd20164920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcd200c8260 tx=0x7fcd100091d0).stop 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- 172.21.15.181:0/3964249439 shutdown_connections 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd20164920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3964249439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20164e10 0x7fcd20062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- 172.21.15.181:0/3964249439 >> 172.21.15.181:0/3964249439 conn(0x7fcd200d2dc0 msgr2=0x7fcd200ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- 172.21.15.181:0/3964249439 shutdown_connections 2020-11-16T22:31:59.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- 172.21.15.181:0/3964249439 wait complete. 2020-11-16T22:31:59.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 Processor -- start 2020-11-16T22:31:59.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- start start 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd201748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 0x7fcd20145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20145990 con 0x7fcd20161480 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201464c0 con 0x7fcd20161860 2020-11-16T22:31:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd255f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20146760 con 0x7fcd20173f00 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41058/0 (socket says 172.21.15.181:41058) 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd24df2700 1 -- 172.21.15.181:0/3280044127 learned_addr learned my addr 172.21.15.181:0/3280044127 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 0x7fcd20145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.364+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd201748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 -- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 msgr2=0x7fcd20145e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 0x7fcd20145e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 -- 172.21.15.181:0/3280044127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 msgr2=0x7fcd201748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd201748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 -- 172.21.15.181:0/3280044127 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20146a00 con 0x7fcd20173f00 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 0x7fcd20145e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd24df2700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fcd1800c710 tx=0x7fcd180079c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 -- 172.21.15.181:0/3280044127 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180072e0 con 0x7fcd20173f00 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 -- 172.21.15.181:0/3280044127 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18007440 con 0x7fcd20173f00 2020-11-16T22:31:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd20146ca0 con 0x7fcd20173f00 2020-11-16T22:31:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 -- 172.21.15.181:0/3280044127 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18005800 con 0x7fcd20173f00 2020-11-16T22:31:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd20174340 con 0x7fcd20173f00 2020-11-16T22:31:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 -- 172.21.15.181:0/3280044127 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcd18010070 con 0x7fcd20173f00 2020-11-16T22:31:59.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 0x7fccf8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fccfffff700 1 -- 172.21.15.181:0/3280044127 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd1806d380 con 0x7fcd20173f00 2020-11-16T22:31:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 0x7fccf8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.368+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 0x7fccf8057520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fcd10009fd0 tx=0x7fcd10016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 Processor -- start 2020-11-16T22:31:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 -- start start 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a47b0 0x26c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x26c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x254e4b0 con 0x24a47b0 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2602930 con 0x2436bb0 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24c26f0 con 0x26c2fd0 2020-11-16T22:31:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x26c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41068/0 (socket says 172.21.15.181:41068) 2020-11-16T22:31:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x26c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33224/0 (socket says 172.21.15.181:33224) 2020-11-16T22:31:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a47b0 0x26c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 -- 172.21.15.181:0/1162089018 learned_addr learned my addr 172.21.15.181:0/1162089018 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 -- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 msgr2=0x26c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x26c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 -- 172.21.15.181:0/1162089018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a47b0 msgr2=0x26c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a47b0 0x26c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.372+0000 7fcd24df2700 1 -- 172.21.15.181:0/1162089018 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24bd5a0 con 0x26c2fd0 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd24df2700 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fcd180059b0 tx=0x7fcd18009000).ready entity=mon.2 client_cookie=bfdbde01f82c0491 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fccfd7fa700 1 -- 172.21.15.181:0/1162089018 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1804d070 con 0x26c2fd0 2020-11-16T22:31:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fccfd7fa700 1 -- 172.21.15.181:0/1162089018 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18045710 con 0x26c2fd0 2020-11-16T22:31:59.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fccfd7fa700 1 -- 172.21.15.181:0/1162089018 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18005e00 con 0x26c2fd0 2020-11-16T22:31:59.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 msgr2=0x26bc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:31:59.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fcd180059b0 tx=0x7fcd18009000).stop 2020-11-16T22:31:59.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- 172.21.15.181:0/1162089018 shutdown_connections 2020-11-16T22:31:59.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x26c2fd0 0x26bc700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24a47b0 0x26c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- 172.21.15.181:0/1162089018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x26c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- 172.21.15.181:0/1162089018 >> 172.21.15.181:0/1162089018 conn(0x2672e40 msgr2=0x26bf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- 172.21.15.181:0/1162089018 shutdown_connections 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- 172.21.15.181:0/1162089018 wait complete. 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- start start 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 Processor -- start 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 0x27131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c2fd0 0x26dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ef6e0 con 0x26c2fd0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ef820 con 0x2436bb0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25f0a90 con 0x24a47b0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd352e6740 10 client.0 ll_register_callbacks cb 0x25e5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 0x27131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33228/0 (socket says 172.21.15.181:33228) 2020-11-16T22:31:59.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3910366947 learned_addr learned my addr 172.21.15.181:0/3910366947 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:31:59.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 0x27131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41072/0 (socket says 172.21.15.181:41072) 2020-11-16T22:31:59.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd24df2700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c2fd0 0x26dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:31:59.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 msgr2=0x27131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 0x27131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3910366947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c2fd0 msgr2=0x26dd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c2fd0 0x26dd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 -- 172.21.15.181:0/3910366947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25e0280 con 0x2436bb0 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fcd200c8f40 tx=0x7fcd1001e000).ready entity=mon.1 client_cookie=201dc83c86137b3b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:31:59.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.376+0000 7fccf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:31:59.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10009940 con 0x2436bb0 2020-11-16T22:31:59.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd1001d730 con 0x2436bb0 2020-11-16T22:31:59.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26a36c0 con 0x2436bb0 2020-11-16T22:32:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd10032b30 con 0x2436bb0 2020-11-16T22:32:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 10 client.6679 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x2415fd0 con 0x2436bb0 2020-11-16T22:32:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26a36c0 con 0x2436bb0 2020-11-16T22:32:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fcd10032c90 con 0x2436bb0 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 10 client.6679 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 10 client.6679 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 20 client.6679 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26bf900 0x2723f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 4 client.6679 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x24e0c50 con 0x26bf900 2020-11-16T22:32:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26bf900 0x2723f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 10 client.6679 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26bf900 0x2723f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=53b04e55690dc1a8 server_cookie=132f8e1ae1773a23 in_seq=0 out_seq=0 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 1 -- 172.21.15.181:0/3910366947 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x24e0c50 con 0x26bf900 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 10 client.6679 handle_command_reply: tid=0 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 client.6679 shutdown 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 msgr2=0x2711a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fcd200c8f40 tx=0x7fcd1001e000).stop 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 shutdown_connections 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26bf900 0x2723f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26c2fd0 0x26dd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fccf67fc700 0 client.6679 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2436bb0 0x2711a50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3910366947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24a47b0 0x27131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 >> 172.21.15.181:0/3910366947 conn(0x2672e40 msgr2=0x26f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 shutdown_connections 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 1 -- 172.21.15.181:0/3910366947 wait complete. 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.380+0000 7fcd352e6740 20 client.6679 trim_cache size 0 max 16384 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.400+0000 7fcd352e6740 1 Processor -- start 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.400+0000 7fcd352e6740 1 -- start start 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 0x26f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 0x26bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x2725e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c7e40 con 0x2672e40 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26bd200 con 0x24a47b0 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2713a30 con 0x2436bb0 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 0x26bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x2725e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 0x26bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41080/0 (socket says 172.21.15.181:41080) 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd1f7fe700 1 -- 172.21.15.181:0/762596123 learned_addr learned my addr 172.21.15.181:0/762596123 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 0x26f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 -- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 msgr2=0x26bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 0x26bf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 -- 172.21.15.181:0/762596123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 msgr2=0x26f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 0x26f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 -- 172.21.15.181:0/762596123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec008290 con 0x24a47b0 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 0x26f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd24df2700 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x2725e80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x26c3480 tx=0x7fcd1806d000).ready entity=mon.1 client_cookie=64fcef7b1faada63 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fccfdffb700 1 -- 172.21.15.181:0/762596123 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18046c00 con 0x24a47b0 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fccfdffb700 1 -- 172.21.15.181:0/762596123 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd180469a0 con 0x24a47b0 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fccfdffb700 1 -- 172.21.15.181:0/762596123 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1806bd60 con 0x24a47b0 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 msgr2=0x2725e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:00.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x2725e80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x26c3480 tx=0x7fcd1806d000).stop 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- 172.21.15.181:0/762596123 shutdown_connections 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x2725e80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2672e40 0x26f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 --2- 172.21.15.181:0/762596123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2436bb0 0x26bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- 172.21.15.181:0/762596123 >> 172.21.15.181:0/762596123 conn(0x7fccec002320 msgr2=0x26b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- 172.21.15.181:0/762596123 shutdown_connections 2020-11-16T22:32:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- 172.21.15.181:0/762596123 wait complete. 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.404+0000 7fcd352e6740 1 -- start start 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 Processor -- start 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 0x26c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2672e40 0x2725e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26c3b90 con 0x2436bb0 2020-11-16T22:32:00.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2778410 con 0x24a47b0 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x27786b0 con 0x2672e40 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33244/0 (socket says 172.21.15.181:33244) 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 10 client.0 ll_register_callbacks cb 0x26c3d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 -- 172.21.15.181:0/3533949942 learned_addr learned my addr 172.21.15.181:0/3533949942 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd24df2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2672e40 0x2725e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 0x26c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 -- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2672e40 msgr2=0x2725e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2672e40 0x2725e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 -- 172.21.15.181:0/3533949942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 msgr2=0x26c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 0x26c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 -- 172.21.15.181:0/3533949942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2454720 con 0x24a47b0 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 0x26c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1f7fe700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fcd200d2230 tx=0x7fcd08019040).ready entity=mon.1 client_cookie=7a4fbc0028425f5a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0801e710 con 0x24a47b0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd0801e870 con 0x24a47b0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd18045710 con 0x24a47b0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd0801ec40 con 0x24a47b0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 10 client.6685 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x245fe00 con 0x24a47b0 2020-11-16T22:32:00.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18045710 con 0x24a47b0 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fcd08025020 con 0x24a47b0 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 10 client.6685 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 10 client.6685 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 20 client.6685 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26c2fd0 0x2726370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 4 client.6685 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:32:00.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x24e7f50 con 0x26c2fd0 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.408+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26c2fd0 0x2726370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fccf67fc700 10 client.6685 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd1ffff700 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26c2fd0 0x2726370 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b9e95b42272a349d server_cookie=aaf1a1c48fa30cd6 in_seq=0 out_seq=0 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fccf67fc700 1 -- 172.21.15.181:0/3533949942 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x24e7f50 con 0x26c2fd0 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fccf67fc700 10 client.6685 handle_command_reply: tid=0 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 client.6685 shutdown 2020-11-16T22:32:00.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 msgr2=0x26c23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fcd200d2230 tx=0x7fcd08019040).stop 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 shutdown_connections 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x26c2fd0 0x2726370 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2672e40 0x2725e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2436bb0 0x26c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 --2- 172.21.15.181:0/3533949942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24a47b0 0x26c23c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 >> 172.21.15.181:0/3533949942 conn(0x7fccec002320 msgr2=0x26b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:00.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fccf67fc700 0 client.6685 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 shutdown_connections 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 1 -- 172.21.15.181:0/3533949942 wait complete. 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd352e6740 20 client.6685 trim_cache size 0 max 16384 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 msgr2=0x7fccf8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 0x7fccf8057520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fcd10009fd0 tx=0x7fcd10016000).stop 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 msgr2=0x7fcd20149810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:00.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fcd1800c710 tx=0x7fcd180079c0).stop 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 shutdown_connections 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fccf8055060 0x7fccf8057520 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd20173f00 0x7fcd20149810 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd20161480 0x7fcd201748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 --2- 172.21.15.181:0/3280044127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd20161860 0x7fcd20145e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 >> 172.21.15.181:0/3280044127 conn(0x7fcd200d2dc0 msgr2=0x7fcd201472c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:00.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 shutdown_connections 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:31:59.412+0000 7fcd255f3700 1 -- 172.21.15.181:0/3280044127 wait complete. 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: "7dd00246-57fd-4ba3-937c-4a270ecf192a": { 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:32:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "7dd00246-57fd-4ba3-937c-4a270ecf192a", 2020-11-16T22:32:00.205 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:32:00.205 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:00.205 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:00.205 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:32:00.205 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:32:00.207 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:00.213 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:00.216 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:04.701 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:32:04.719+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:32:05.207 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:05.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 Processor -- start 2020-11-16T22:32:05.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- start start 2020-11-16T22:32:05.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a700cf390 0x7f2a700cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a70163970 0x7f2a70163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a70065190 con 0x7f2a700600f0 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a700cfc60 con 0x7f2a70163970 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a70065430 con 0x7f2a700cf390 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a700cf390 0x7f2a700cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a700cf390 0x7f2a700cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33250/0 (socket says 172.21.15.181:33250) 2020-11-16T22:32:05.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 -- 172.21.15.181:0/141273452 learned_addr learned my addr 172.21.15.181:0/141273452 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:05.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6dd9b700 1 --2- 172.21.15.181:0/141273452 >> v2:172.21.15.175:3300/0 conn(0x7f2a70163970 0x7f2a70163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 -- 172.21.15.181:0/141273452 >> v2:172.21.15.175:3300/0 conn(0x7f2a70163970 msgr2=0x7f2a70163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:05.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 --2- 172.21.15.181:0/141273452 >> v2:172.21.15.175:3300/0 conn(0x7f2a70163970 0x7f2a70163df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 -- 172.21.15.181:0/141273452 >> v1:172.21.15.175:6789/0 conn(0x7f2a700600f0 legacy=0x7f2a70060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 -- 172.21.15.181:0/141273452 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700656d0 con 0x7f2a700cf390 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6e59c700 1 --2- 172.21.15.181:0/141273452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a700cf770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f2a700622a0 tx=0x7f2a58006eb0).ready entity=mon.1 client_cookie=6d953d7d8b9cb76d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6d59a700 1 -- 172.21.15.181:0/141273452 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58009810 con 0x7f2a700cf390 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6d59a700 1 -- 172.21.15.181:0/141273452 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a58009970 con 0x7f2a700cf390 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6d59a700 1 -- 172.21.15.181:0/141273452 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58009cf0 con 0x7f2a700cf390 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- 172.21.15.181:0/141273452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 msgr2=0x7f2a700cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/141273452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a700cf770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f2a700622a0 tx=0x7f2a58006eb0).stop 2020-11-16T22:32:05.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- 172.21.15.181:0/141273452 shutdown_connections 2020-11-16T22:32:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/141273452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a700cf770 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/141273452 >> v2:172.21.15.175:3300/0 conn(0x7f2a70163970 0x7f2a70163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- 172.21.15.181:0/141273452 >> 172.21.15.181:0/141273452 conn(0x7f2a700cfe90 msgr2=0x7f2a700d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- 172.21.15.181:0/141273452 shutdown_connections 2020-11-16T22:32:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.351+0000 7f2a6f59e700 1 -- 172.21.15.181:0/141273452 wait complete. 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 Processor -- start 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- start start 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a7013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a70163970 0x7f2a7015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70062b00 con 0x7f2a70163970 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7013e680 con 0x7f2a700cf390 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7013e920 con 0x7f2a700600f0 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a7013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41098/0 (socket says 172.21.15.181:41098) 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 -- 172.21.15.181:0/1751802504 learned_addr learned my addr 172.21.15.181:0/1751802504 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:05.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a7013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33258/0 (socket says 172.21.15.181:33258) 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6ed9d700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a70163970 0x7f2a7015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 -- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 msgr2=0x7f2a7013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a7013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 -- 172.21.15.181:0/1751802504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a70163970 msgr2=0x7f2a7015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a70163970 0x7f2a7015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 -- 172.21.15.181:0/1751802504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7013ebc0 con 0x7f2a700600f0 2020-11-16T22:32:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6e59c700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f2a70065570 tx=0x7f2a58015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58011d50 con 0x7f2a700600f0 2020-11-16T22:32:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5801e460 con 0x7f2a700600f0 2020-11-16T22:32:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7013ee60 con 0x7f2a700600f0 2020-11-16T22:32:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5801e820 con 0x7f2a700600f0 2020-11-16T22:32:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.355+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7013fa90 con 0x7f2a700600f0 2020-11-16T22:32:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2a58025030 con 0x7f2a700600f0 2020-11-16T22:32:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7013f300 con 0x7f2a700600f0 2020-11-16T22:32:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a667fc700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 0x7f2a40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a5806f3c0 con 0x7f2a700600f0 2020-11-16T22:32:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a6dd9b700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 0x7f2a40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.359+0000 7f2a6dd9b700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 0x7f2a40057520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f2a7013e120 tx=0x7f2a60006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.363+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2a7013f300 con 0x7f2a700600f0 2020-11-16T22:32:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a70044f60 con 0x7f2a700600f0 2020-11-16T22:32:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a667fc700 1 -- 172.21.15.181:0/1751802504 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f2a58045620 con 0x7f2a700600f0 2020-11-16T22:32:05.555 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:05.555 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 msgr2=0x7f2a40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 0x7f2a40057520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f2a7013e120 tx=0x7f2a60006df0).stop 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 msgr2=0x7f2a70062610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f2a70065570 tx=0x7f2a58015000).stop 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 shutdown_connections 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2a40055060 0x7f2a40057520 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a70163970 0x7f2a7015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a700600f0 0x7f2a70062610 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 --2- 172.21.15.181:0/1751802504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a700cf390 0x7f2a7013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.567+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 >> 172.21.15.181:0/1751802504 conn(0x7f2a700cfe90 msgr2=0x7f2a70060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.571+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 shutdown_connections 2020-11-16T22:32:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.571+0000 7f2a6f59e700 1 -- 172.21.15.181:0/1751802504 wait complete. 2020-11-16T22:32:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:05.569 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:32:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 Processor -- start 2020-11-16T22:32:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 -- start start 2020-11-16T22:32:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4c4160530 0x7ff4c4061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c4062700 con 0x7ff4c4063620 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c40629a0 con 0x7ff4c4160530 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c97c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4062c40 con 0x7ff4c4142660 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41108/0 (socket says 172.21.15.181:41108) 2020-11-16T22:32:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c3fff700 1 -- 172.21.15.181:0/2915728287 learned_addr learned my addr 172.21.15.181:0/2915728287 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.711+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/2915728287 >> v2:172.21.15.175:3300/0 conn(0x7ff4c4160530 0x7ff4c4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c3fff700 1 -- 172.21.15.181:0/2915728287 >> v2:172.21.15.175:3300/0 conn(0x7ff4c4160530 msgr2=0x7ff4c4061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2915728287 >> v2:172.21.15.175:3300/0 conn(0x7ff4c4160530 0x7ff4c4061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c3fff700 1 -- 172.21.15.181:0/2915728287 >> v1:172.21.15.175:6789/0 conn(0x7ff4c4063620 legacy=0x7ff4c4063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c3fff700 1 -- 172.21.15.181:0/2915728287 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4062ee0 con 0x7ff4c4142660 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2915728287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7ff4b4006fd0 tx=0x7ff4b4006eb0).ready entity=mon.2 client_cookie=a81d7c982f4a8592 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c37fe700 1 -- 172.21.15.181:0/2915728287 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40098a0 con 0x7ff4c4142660 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c37fe700 1 -- 172.21.15.181:0/2915728287 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b4009a00 con 0x7ff4c4142660 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c37fe700 1 -- 172.21.15.181:0/2915728287 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4009d80 con 0x7ff4c4142660 2020-11-16T22:32:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- 172.21.15.181:0/2915728287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 msgr2=0x7ff4c4142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/2915728287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7ff4b4006fd0 tx=0x7ff4b4006eb0).stop 2020-11-16T22:32:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- 172.21.15.181:0/2915728287 shutdown_connections 2020-11-16T22:32:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/2915728287 >> v2:172.21.15.175:3300/0 conn(0x7ff4c4160530 0x7ff4c4061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/2915728287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4142660 0x7ff4c4142a40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- 172.21.15.181:0/2915728287 >> 172.21.15.181:0/2915728287 conn(0x7ff4c40d2dc0 msgr2=0x7ff4c40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- 172.21.15.181:0/2915728287 shutdown_connections 2020-11-16T22:32:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- 172.21.15.181:0/2915728287 wait complete. 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 Processor -- start 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.715+0000 7ff4c97c8700 1 -- start start 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 0x7ff4c4141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c4160530 0x7ff4c41417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cb000 con 0x7ff4c4160530 2020-11-16T22:32:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cb2a0 con 0x7ff4c4142660 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cb540 con 0x7ff4c4063620 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41112/0 (socket says 172.21.15.181:41112) 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c3fff700 1 -- 172.21.15.181:0/1675328905 learned_addr learned my addr 172.21.15.181:0/1675328905 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 0x7ff4c4141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c4160530 0x7ff4c41417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 msgr2=0x7ff4c40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/1675328905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c4160530 msgr2=0x7ff4c41417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c4160530 0x7ff4c41417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/1675328905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40cb7e0 con 0x7ff4c4142660 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 0x7ff4c4141300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff4ac007fd0 tx=0x7ff4ac007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c0ff9700 1 -- 172.21.15.181:0/1675328905 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f470 con 0x7ff4c4142660 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c0ff9700 1 -- 172.21.15.181:0/1675328905 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00f5d0 con 0x7ff4c4142660 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c0ff9700 1 -- 172.21.15.181:0/1675328905 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f950 con 0x7ff4c4142660 2020-11-16T22:32:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c40cba80 con 0x7ff4c4142660 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.719+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4143360 con 0x7ff4c4142660 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c0ff9700 1 -- 172.21.15.181:0/1675328905 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff4ac019030 con 0x7ff4c4142660 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 Processor -- start 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- start start 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c0ff9700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 0x7ff4a0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c0ff9700 1 -- 172.21.15.181:0/1675328905 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4ac06cfe0 con 0x7ff4c4142660 2020-11-16T22:32:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 0x7ff4a0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 0x7ff4a0057520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff4bc0092a0 tx=0x7ff4bc0091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x25f7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24804b0 con 0x23d67b0 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2534930 con 0x2368bb0 2020-11-16T22:32:06.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23f46f0 con 0x25f4fd0 2020-11-16T22:32:06.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41122/0 (socket says 172.21.15.181:41122) 2020-11-16T22:32:06.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 -- 172.21.15.181:0/2498239353 learned_addr learned my addr 172.21.15.181:0/2498239353 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:06.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x25f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 -- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 msgr2=0x25f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 -- 172.21.15.181:0/2498239353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 msgr2=0x25f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x25f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 -- 172.21.15.181:0/2498239353 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23ef5a0 con 0x25f4fd0 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff4bc00ff20 tx=0x7ff4bc014000).ready entity=mon.2 client_cookie=486e2ab4cfb40691 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4b97fa700 1 -- 172.21.15.181:0/2498239353 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc013ba0 con 0x25f4fd0 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4b97fa700 1 -- 172.21.15.181:0/2498239353 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc013d00 con 0x25f4fd0 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4b97fa700 1 -- 172.21.15.181:0/2498239353 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01e610 con 0x25f4fd0 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 msgr2=0x25ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff4bc00ff20 tx=0x7ff4bc014000).stop 2020-11-16T22:32:06.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2498239353 shutdown_connections 2020-11-16T22:32:06.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25ee700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x25f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2498239353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2498239353 >> 172.21.15.181:0/2498239353 conn(0x25a4e40 msgr2=0x25f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:06.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2498239353 shutdown_connections 2020-11-16T22:32:06.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.727+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2498239353 wait complete. 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- start start 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 Processor -- start 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x2645100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25216e0 con 0x23d67b0 2020-11-16T22:32:06.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2521820 con 0x2368bb0 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2522a90 con 0x25f4fd0 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 10 client.0 ll_register_callbacks cb 0x2517ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c8fc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33284/0 (socket says 172.21.15.181:33284) 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3720871283 learned_addr learned my addr 172.21.15.181:0/3720871283 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:06.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x2645100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 msgr2=0x25f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3720871283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 msgr2=0x2645100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x2645100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3720871283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2512280 con 0x2368bb0 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7ff4b4011fd0 tx=0x7ff4b4015000).ready entity=mon.1 client_cookie=cbdf7694c849a609 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:32:06.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401e430 con 0x2368bb0 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25d56c0 con 0x2368bb0 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4b401e590 con 0x2368bb0 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b401e960 con 0x2368bb0 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 10 client.6700 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x2347fd0 con 0x2368bb0 2020-11-16T22:32:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25d56c0 con 0x2368bb0 2020-11-16T22:32:06.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7ff4b4026020 con 0x2368bb0 2020-11-16T22:32:06.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 10 client.6700 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:06.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 10 client.6700 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:06.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 20 client.6700 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x260f630 0x26252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 4 client.6700 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2412c50 con 0x260f630 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x260f630 0x26252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff49a7fc700 10 client.6700 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.731+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x260f630 0x26252f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4a50535d95e2445a server_cookie=1702de971ebbf5e7 in_seq=0 out_seq=0 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff49a7fc700 1 -- 172.21.15.181:0/3720871283 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2412c50 con 0x260f630 2020-11-16T22:32:06.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff49a7fc700 10 client.6700 handle_command_reply: tid=0 2020-11-16T22:32:06.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 client.6700 shutdown 2020-11-16T22:32:06.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 msgr2=0x25f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:06.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7ff4b4011fd0 tx=0x7ff4b4015000).stop 2020-11-16T22:32:06.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 shutdown_connections 2020-11-16T22:32:06.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x260f630 0x26252f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25f4fd0 0x25f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2368bb0 0x25f1530 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff49a7fc700 0 client.6700 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3720871283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d67b0 0x2645100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 >> 172.21.15.181:0/3720871283 conn(0x25a4e40 msgr2=0x25f6080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 shutdown_connections 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3720871283 wait complete. 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.735+0000 7ff4d94bb740 20 client.6700 trim_cache size 0 max 16384 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 Processor -- start 2020-11-16T22:32:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 -- start start 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x2644260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d67b0 0x263f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24f9e40 con 0x2368bb0 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f6380 con 0x23d67b0 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2664a80 con 0x25a4e40 2020-11-16T22:32:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41136/0 (socket says 172.21.15.181:41136) 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3323824740 learned_addr learned my addr 172.21.15.181:0/3323824740 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d67b0 0x263f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x2644260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d67b0 msgr2=0x263f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d67b0 0x263f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3323824740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 msgr2=0x2644260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.751+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x2644260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4c3fff700 1 -- 172.21.15.181:0/3323824740 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490008160 con 0x25a4e40 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff4b400b000 tx=0x7ff4b4021040).ready entity=mon.2 client_cookie=cfd02bc818976577 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4b9ffb700 1 -- 172.21.15.181:0/3323824740 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401e960 con 0x25a4e40 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4b9ffb700 1 -- 172.21.15.181:0/3323824740 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b4011e30 con 0x25a4e40 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4b9ffb700 1 -- 172.21.15.181:0/3323824740 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40098a0 con 0x25a4e40 2020-11-16T22:32:07.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 msgr2=0x2657d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:07.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff4b400b000 tx=0x7ff4b4021040).stop 2020-11-16T22:32:07.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3323824740 shutdown_connections 2020-11-16T22:32:07.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d67b0 0x263f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25a4e40 0x2657d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/3323824740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x2644260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3323824740 >> 172.21.15.181:0/3323824740 conn(0x7ff490002320 msgr2=0x25ee770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3323824740 shutdown_connections 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- 172.21.15.181:0/3323824740 wait complete. 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 -- start start 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.755+0000 7ff4d94bb740 1 Processor -- start 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x26551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25a4e40 0x26a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2626940 con 0x2368bb0 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26a7000 con 0x25a4e40 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25f2ee0 con 0x23d67b0 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 10 client.0 ll_register_callbacks cb 0x25f5a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:07.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41142/0 (socket says 172.21.15.181:41142) 2020-11-16T22:32:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x26551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/2263212859 learned_addr learned my addr 172.21.15.181:0/2263212859 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:07.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4c8fc7700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25a4e40 0x26a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25a4e40 msgr2=0x26a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25a4e40 0x26a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/2263212859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 msgr2=0x26551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x26551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 -- 172.21.15.181:0/2263212859 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2386720 con 0x23d67b0 2020-11-16T22:32:07.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4bb7fe700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff4ac043420 tx=0x7ff4ac003040).ready entity=mon.2 client_cookie=ac43029c95b794a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac001ea0 con 0x23d67b0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4ac00fe10 con 0x23d67b0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4bc013d00 con 0x23d67b0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ac046620 con 0x23d67b0 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 10 client.6209 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:07.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x2391e00 con 0x23d67b0 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc013d00 con 0x23d67b0 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7ff4ac042d10 con 0x23d67b0 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 10 client.6209 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 10 client.6209 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 20 client.6209 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x25f4fd0 0x2644c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:07.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 4 client.6209 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x2419f50 con 0x25f4fd0 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x25f4fd0 0x2644c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4c3fff700 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x25f4fd0 0x2644c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=803d74ae047b1362 server_cookie=aeac248514755dce in_seq=0 out_seq=0 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 10 client.6209 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 1 -- 172.21.15.181:0/2263212859 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x2419f50 con 0x25f4fd0 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 10 client.6209 handle_command_reply: tid=0 2020-11-16T22:32:07.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 client.6209 shutdown 2020-11-16T22:32:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 msgr2=0x2626450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff4ac043420 tx=0x7ff4ac003040).stop 2020-11-16T22:32:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 shutdown_connections 2020-11-16T22:32:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x25f4fd0 0x2644c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25a4e40 0x26a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff49a7fc700 0 client.6209 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2368bb0 0x26551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 --2- 172.21.15.181:0/2263212859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23d67b0 0x2626450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 >> 172.21.15.181:0/2263212859 conn(0x7ff490002320 msgr2=0x25ee740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 shutdown_connections 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 1 -- 172.21.15.181:0/2263212859 wait complete. 2020-11-16T22:32:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.759+0000 7ff4d94bb740 20 client.6209 trim_cache size 0 max 16384 2020-11-16T22:32:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 msgr2=0x7ff4a0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 0x7ff4a0057520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff4bc0092a0 tx=0x7ff4bc0091d0).stop 2020-11-16T22:32:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 msgr2=0x7ff4c4141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 0x7ff4c4141300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff4ac007fd0 tx=0x7ff4ac007b50).stop 2020-11-16T22:32:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 shutdown_connections 2020-11-16T22:32:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff4a0055060 0x7ff4a0057520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c4160530 0x7ff4c41417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c4063620 0x7ff4c40caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 --2- 172.21.15.181:0/1675328905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c4142660 0x7ff4c4141300 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 >> 172.21.15.181:0/1675328905 conn(0x7ff4c40d2dc0 msgr2=0x7ff4c415ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 shutdown_connections 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:05.763+0000 7ff4c97c8700 1 -- 172.21.15.181:0/1675328905 wait complete. 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:32:07.971 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:32:07.972 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:32:07.972 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-11-16T22:32:07.972 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 Processor -- start 2020-11-16T22:32:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 -- start start 2020-11-16T22:32:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63740b6c20 0x7f6374066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6374064050 con 0x7f637415e270 2020-11-16T22:32:08.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63740642f0 con 0x7f6374063b80 2020-11-16T22:32:08.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f637b982700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6374064590 con 0x7f63740b6c20 2020-11-16T22:32:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6379f1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63740b6c20 0x7f6374066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6378f1d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6374063b80 0x7f637415f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47610/0 (socket says 172.21.15.181:47610) 2020-11-16T22:32:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6378f1d700 1 -- 172.21.15.181:0/2576266821 learned_addr learned my addr 172.21.15.181:0/2576266821 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670578092 0 0) 0x7f63740642f0 con 0x7f6374063b80 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6358003240 con 0x7f6374063b80 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432749136 0 0) 0x7f6374064050 con 0x7f637415e270 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63740642f0 con 0x7f637415e270 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1595834077 0 0) 0x7f6358003240 con 0x7f6374063b80 2020-11-16T22:32:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6379f1f700 1 -- 172.21.15.181:0/2576266821 >> v1:172.21.15.181:6789/0 conn(0x7f6374063b80 legacy=0x7f637415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6379f1f700 1 -- 172.21.15.181:0/2576266821 >> v1:172.21.15.175:6789/0 conn(0x7f637415e270 legacy=0x7f63740667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6379f1f700 1 -- 172.21.15.181:0/2576266821 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374064830 con 0x7f63740b6c20 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6379f1f700 1 --2- 172.21.15.181:0/2576266821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63740b6c20 0x7f6374066ec0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f636c009c10 tx=0x7f636c0091d0).ready entity=mon.1 client_cookie=388c1e530c05fa4b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700014e0 con 0x7f63740b6c20 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c00b730 con 0x7f63740b6c20 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.115+0000 7f6367fff700 1 -- 172.21.15.181:0/2576266821 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00bab0 con 0x7f63740b6c20 2020-11-16T22:32:08.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- 172.21.15.181:0/2576266821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63740b6c20 msgr2=0x7f6374066ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 --2- 172.21.15.181:0/2576266821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63740b6c20 0x7f6374066ec0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f636c009c10 tx=0x7f636c0091d0).stop 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- 172.21.15.181:0/2576266821 shutdown_connections 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 --2- 172.21.15.181:0/2576266821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63740b6c20 0x7f6374066ec0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- 172.21.15.181:0/2576266821 >> 172.21.15.181:0/2576266821 conn(0x7f63740d4790 msgr2=0x7f63740d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- 172.21.15.181:0/2576266821 shutdown_connections 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- 172.21.15.181:0/2576266821 wait complete. 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 Processor -- start 2020-11-16T22:32:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.119+0000 7f637b982700 1 -- start start 2020-11-16T22:32:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 0x7f6374142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63740b6c20 0x7f63740a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 0x7f63740a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741653a0 con 0x7f63740b6c20 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374170400 con 0x7f6374063b80 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63741706a0 con 0x7f637415e270 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637971e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 0x7f6374142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 0x7f63740a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637971e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 0x7f6374142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33312/0 (socket says 172.21.15.181:33312) 2020-11-16T22:32:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637971e700 1 -- 172.21.15.181:0/3299663753 learned_addr learned my addr 172.21.15.181:0/3299663753 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6378f1d700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63740b6c20 0x7f63740a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 -- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 msgr2=0x7f6374142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 0x7f6374142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 -- 172.21.15.181:0/3299663753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63740b6c20 msgr2=0x7f63740a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63740b6c20 0x7f63740a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 -- 172.21.15.181:0/3299663753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374170940 con 0x7f637415e270 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f6379f1f700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 0x7f63740a0590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f63740d1200 tx=0x7f636c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c01d450 con 0x7f637415e270 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c01d5b0 con 0x7f637415e270 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6374170be0 con 0x7f637415e270 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c01d930 con 0x7f637415e270 2020-11-16T22:32:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6374142d70 con 0x7f637415e270 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374142fc0 con 0x7f637415e270 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f636c023020 con 0x7f637415e270 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 0x7f634c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f636c06fba0 con 0x7f637415e270 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.123+0000 7f637971e700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 0x7f634c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.127+0000 7f637971e700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 0x7f634c057550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f63680033e0 tx=0x7f6368007ae0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.131+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f636c045e80 con 0x7f637415e270 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6374170220 con 0x7f637415e270 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f63657fa700 1 -- 172.21.15.181:0/3299663753 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f636c02c090 con 0x7f637415e270 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 msgr2=0x7f634c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 0x7f634c057550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f63680033e0 tx=0x7f6368007ae0).stop 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 msgr2=0x7f63740a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 0x7f63740a0590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f63740d1200 tx=0x7f636c016000).stop 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 shutdown_connections 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f634c055090 0x7f634c057550 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f637415e270 0x7f63740a0590 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374063b80 0x7f6374142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 --2- 172.21.15.181:0/3299663753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63740b6c20 0x7f63740a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 >> 172.21.15.181:0/3299663753 conn(0x7f63740d4790 msgr2=0x7f63740b9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 shutdown_connections 2020-11-16T22:32:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.335+0000 7f637b982700 1 -- 172.21.15.181:0/3299663753 wait complete. 2020-11-16T22:32:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:08.523 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:08.523 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:08.524 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls 2020-11-16T22:32:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 Processor -- start 2020-11-16T22:32:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 -- start start 2020-11-16T22:32:08.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fce340816e0 0x7fce34145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 0x7fce340cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce34061b50 con 0x7fce34145a30 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce340618b0 con 0x7fce340816e0 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce340cb930 con 0x7fce341492c0 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 0x7fce340cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fce34145a30 0x7fce34080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47624/0 (socket says 172.21.15.181:47624) 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3259c700 1 -- 172.21.15.181:0/4120211561 learned_addr learned my addr 172.21.15.181:0/4120211561 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce31d9b700 1 --2- 172.21.15.181:0/4120211561 >> v2:172.21.15.175:3300/0 conn(0x7fce340816e0 0x7fce34145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3159a700 1 -- 172.21.15.181:0/4120211561 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337992971 0 0) 0x7fce34061b50 con 0x7fce34145a30 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce3159a700 1 -- 172.21.15.181:0/4120211561 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c003240 con 0x7fce34145a30 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce32d9d700 1 -- 172.21.15.181:0/4120211561 >> v2:172.21.15.175:3300/0 conn(0x7fce340816e0 msgr2=0x7fce34145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce32d9d700 1 --2- 172.21.15.181:0/4120211561 >> v2:172.21.15.175:3300/0 conn(0x7fce340816e0 0x7fce34145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce32d9d700 1 -- 172.21.15.181:0/4120211561 >> v1:172.21.15.181:6789/0 conn(0x7fce34145a30 legacy=0x7fce34080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.675+0000 7fce32d9d700 1 -- 172.21.15.181:0/4120211561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340cbbd0 con 0x7fce341492c0 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce32d9d700 1 --2- 172.21.15.181:0/4120211561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 0x7fce340cc5b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fce2c009c90 tx=0x7fce2c0091d0).ready entity=mon.2 client_cookie=92560c97d553cd96 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3159a700 1 -- 172.21.15.181:0/4120211561 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00b9c0 con 0x7fce341492c0 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3159a700 1 -- 172.21.15.181:0/4120211561 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c00bb20 con 0x7fce341492c0 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3159a700 1 -- 172.21.15.181:0/4120211561 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00bea0 con 0x7fce341492c0 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- 172.21.15.181:0/4120211561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 msgr2=0x7fce340cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 --2- 172.21.15.181:0/4120211561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 0x7fce340cc5b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fce2c009c90 tx=0x7fce2c0091d0).stop 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- 172.21.15.181:0/4120211561 shutdown_connections 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 --2- 172.21.15.181:0/4120211561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce341492c0 0x7fce340cc5b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 --2- 172.21.15.181:0/4120211561 >> v2:172.21.15.175:3300/0 conn(0x7fce340816e0 0x7fce34145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- 172.21.15.181:0/4120211561 >> 172.21.15.181:0/4120211561 conn(0x7fce340d68c0 msgr2=0x7fce340d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- 172.21.15.181:0/4120211561 shutdown_connections 2020-11-16T22:32:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- 172.21.15.181:0/4120211561 wait complete. 2020-11-16T22:32:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 Processor -- start 2020-11-16T22:32:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.679+0000 7fce3359e700 1 -- start start 2020-11-16T22:32:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce340816e0 0x7fce34170810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 0x7fce3413e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34171250 con 0x7fce340816e0 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce341714f0 con 0x7fce341492c0 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce34171790 con 0x7fce34145a30 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 0x7fce3413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33326/0 (socket says 172.21.15.181:33326) 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce32d9d700 1 -- 172.21.15.181:0/3805058731 learned_addr learned my addr 172.21.15.181:0/3805058731 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3259c700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce340816e0 0x7fce34170810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 -- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 msgr2=0x7fce34164880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 -- 172.21.15.181:0/3805058731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce340816e0 msgr2=0x7fce34170810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce340816e0 0x7fce34170810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 -- 172.21.15.181:0/3805058731 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34171a30 con 0x7fce34145a30 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce32d9d700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 0x7fce3413e860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fce2400eba0 tx=0x7fce24006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce2a7fc700 1 -- 172.21.15.181:0/3805058731 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400c820 con 0x7fce34145a30 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce2a7fc700 1 -- 172.21.15.181:0/3805058731 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400c980 con 0x7fce34145a30 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce34171cd0 con 0x7fce34145a30 2020-11-16T22:32:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce2a7fc700 1 -- 172.21.15.181:0/3805058731 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400cd00 con 0x7fce34145a30 2020-11-16T22:32:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.683+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34172410 con 0x7fce34145a30 2020-11-16T22:32:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce2a7fc700 1 -- 172.21.15.181:0/3805058731 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce24012600 con 0x7fce34145a30 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce2a7fc700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 0x7fce08057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce2a7fc700 1 -- 172.21.15.181:0/3805058731 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce2406e000 con 0x7fce34145a30 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce3259c700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 0x7fce08057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce3259c700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 0x7fce08057520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fce200055a0 tx=0x7fce20009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce47a85740 1 Processor -- start 2020-11-16T22:32:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.687+0000 7fce47a85740 1 -- start start 2020-11-16T22:32:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c0d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2c0afd0 0x2c04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a964b0 con 0x29ec7b0 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b4a930 con 0x2c0afd0 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a0a6f0 con 0x297ebb0 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41178/0 (socket says 172.21.15.181:41178) 2020-11-16T22:32:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 -- 172.21.15.181:0/1562940473 learned_addr learned my addr 172.21.15.181:0/1562940473 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce32d9d700 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2c0afd0 0x2c04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce3259c700 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c0d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 -- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2c0afd0 msgr2=0x2c04700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2c0afd0 0x2c04700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 -- 172.21.15.181:0/1562940473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 msgr2=0x2c0d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c0d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 -- 172.21.15.181:0/1562940473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2a055a0 con 0x297ebb0 2020-11-16T22:32:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fce24047b90 tx=0x7fce24014000).ready entity=mon.2 client_cookie=b2e8286983cc6fb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce0f7fe700 1 -- 172.21.15.181:0/1562940473 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24044630 con 0x297ebb0 2020-11-16T22:32:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce0f7fe700 1 -- 172.21.15.181:0/1562940473 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24044790 con 0x297ebb0 2020-11-16T22:32:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce0f7fe700 1 -- 172.21.15.181:0/1562940473 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24047620 con 0x297ebb0 2020-11-16T22:32:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 msgr2=0x2c0b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fce24047b90 tx=0x7fce24014000).stop 2020-11-16T22:32:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- 172.21.15.181:0/1562940473 shutdown_connections 2020-11-16T22:32:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2c0afd0 0x2c04700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 --2- 172.21.15.181:0/1562940473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0b670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- 172.21.15.181:0/1562940473 >> 172.21.15.181:0/1562940473 conn(0x2bbae40 msgr2=0x2c073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- 172.21.15.181:0/1562940473 shutdown_connections 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- 172.21.15.181:0/1562940473 wait complete. 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 -- start start 2020-11-16T22:32:08.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.691+0000 7fce47a85740 1 Processor -- start 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x297ebb0 0x2c3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b376e0 con 0x29ec7b0 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b37820 con 0x297ebb0 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b38a90 con 0x2c0afd0 2020-11-16T22:32:08.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 10 client.0 ll_register_callbacks cb 0x2b2dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x297ebb0 0x2c3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41186/0 (socket says 172.21.15.181:41186) 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 -- 172.21.15.181:0/347059111 learned_addr learned my addr 172.21.15.181:0/347059111 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce31d9b700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 -- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x297ebb0 msgr2=0x2c3ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x297ebb0 0x2c3ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 -- 172.21.15.181:0/347059111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 msgr2=0x2c6a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c6a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 -- 172.21.15.181:0/347059111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2b28280 con 0x2c0afd0 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce31d9b700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce32d9d700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fce2c011fd0 tx=0x7fce2c016000).ready entity=mon.2 client_cookie=3fdb08760aa5da8e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:32:08.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c023070 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce2c01d660 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2beb6c0 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce2c01da30 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 10 client.6224 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x295dfd0 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2beb6c0 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fce2c030020 con 0x2c0afd0 2020-11-16T22:32:08.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 10 client.6224 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 10 client.6224 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 20 client.6224 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c08f80 0x2c07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 4 client.6224 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2a28c50 con 0x2c08f80 2020-11-16T22:32:08.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce3259c700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c08f80 0x2c07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce0cff9700 10 client.6224 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:08.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.695+0000 7fce3259c700 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c08f80 0x2c07370 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3a2d2e996fe8c3d8 server_cookie=ba61c57ee855e015 in_seq=0 out_seq=0 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce0cff9700 1 -- 172.21.15.181:0/347059111 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2a28c50 con 0x2c08f80 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce0cff9700 10 client.6224 handle_command_reply: tid=0 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 client.6224 shutdown 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 msgr2=0x2c58e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fce2c011fd0 tx=0x7fce2c016000).stop 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 shutdown_connections 2020-11-16T22:32:08.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c08f80 0x2c07370 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2c0afd0 0x2c58e50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x297ebb0 0x2c3ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 --2- 172.21.15.181:0/347059111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29ec7b0 0x2c6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce0cff9700 0 client.6224 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 >> 172.21.15.181:0/347059111 conn(0x2bbae40 msgr2=0x2c0bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 shutdown_connections 2020-11-16T22:32:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 1 -- 172.21.15.181:0/347059111 wait complete. 2020-11-16T22:32:08.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.699+0000 7fce47a85740 20 client.6224 trim_cache size 0 max 16384 2020-11-16T22:32:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 Processor -- start 2020-11-16T22:32:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 -- start start 2020-11-16T22:32:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2bbae40 0x2c050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2c3b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b0fe40 con 0x2bbae40 2020-11-16T22:32:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c5c4a0 con 0x29ec7b0 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2c3ea30 con 0x297ebb0 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2c3b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41194/0 (socket says 172.21.15.181:41194) 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2c3b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33348/0 (socket says 172.21.15.181:33348) 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 -- 172.21.15.181:0/1738178182 learned_addr learned my addr 172.21.15.181:0/1738178182 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce3259c700 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2bbae40 0x2c050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 -- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 msgr2=0x2c3b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2c3b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 -- 172.21.15.181:0/1738178182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2bbae40 msgr2=0x2c050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2bbae40 0x2c050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 -- 172.21.15.181:0/1738178182 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0081d0 con 0x297ebb0 2020-11-16T22:32:08.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce31d9b700 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fce24047e50 tx=0x7fce2401e070).ready entity=mon.2 client_cookie=b8a32108d9137c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce0ffff700 1 -- 172.21.15.181:0/1738178182 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24044630 con 0x297ebb0 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce0ffff700 1 -- 172.21.15.181:0/1738178182 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24019df0 con 0x297ebb0 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.711+0000 7fce0ffff700 1 -- 172.21.15.181:0/1738178182 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24043f20 con 0x297ebb0 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 msgr2=0x2c0d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fce24047e50 tx=0x7fce2401e070).stop 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- 172.21.15.181:0/1738178182 shutdown_connections 2020-11-16T22:32:08.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2c3b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2bbae40 0x2c050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 --2- 172.21.15.181:0/1738178182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x297ebb0 0x2c0d2a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- 172.21.15.181:0/1738178182 >> 172.21.15.181:0/1738178182 conn(0x7fcdfc002320 msgr2=0x2bff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- 172.21.15.181:0/1738178182 shutdown_connections 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- 172.21.15.181:0/1738178182 wait complete. 2020-11-16T22:32:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 -- start start 2020-11-16T22:32:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.715+0000 7fce47a85740 1 Processor -- start 2020-11-16T22:32:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x297ebb0 0x2cb7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2bbae40 0x2cc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c082e0 con 0x297ebb0 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c08580 con 0x29ec7b0 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2c08820 con 0x2bbae40 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2bbae40 0x2cc09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33356/0 (socket says 172.21.15.181:33356) 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 10 client.0 ll_register_callbacks cb 0x2c0bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 -- 172.21.15.181:0/3193054323 learned_addr learned my addr 172.21.15.181:0/3193054323 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce3259c700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x297ebb0 0x2cb7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 -- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2bbae40 msgr2=0x2cc09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2bbae40 0x2cc09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 -- 172.21.15.181:0/3193054323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x297ebb0 msgr2=0x2cb7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x297ebb0 0x2cb7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 -- 172.21.15.181:0/3193054323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x299c720 con 0x29ec7b0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce31d9b700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fce24046510 tx=0x7fce24031070).ready entity=mon.1 client_cookie=3d86706d94658c35 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24044630 con 0x29ec7b0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce24012890 con 0x29ec7b0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce24044790 con 0x29ec7b0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce24012c60 con 0x29ec7b0 2020-11-16T22:32:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 10 client.6733 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x29a7e00 con 0x29ec7b0 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24044790 con 0x29ec7b0 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fce24019310 con 0x29ec7b0 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 10 client.6733 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 10 client.6733 resolve_mds: resolved ID 'a' to GID 6316 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 20 client.6733 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:08.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c0afd0 0x2c3e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 4 client.6733 mds_command: new command op to 6316 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:32:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 --> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x2a2ff50 con 0x2c0afd0 2020-11-16T22:32:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce3259c700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c0afd0 0x2c3e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 10 client.6733 ms_handle_connect on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce3259c700 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c0afd0 0x2c3e3e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=508df27e6c202d4b server_cookie=b0e37a578961355 in_seq=0 out_seq=0 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 1 -- 172.21.15.181:0/3193054323 <== mds.0 v2:172.21.15.181:6839/3625693713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+272 (crc 0 0 0) 0x2a2ff50 con 0x2c0afd0 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce0cff9700 10 client.6733 handle_command_reply: tid=0 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.719+0000 7fce47a85740 1 client.6733 shutdown 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 msgr2=0x2cc04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fce24046510 tx=0x7fce24031070).stop 2020-11-16T22:32:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 shutdown_connections 2020-11-16T22:32:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:6839/3625693713,v1:172.21.15.181:6841/3625693713] conn(0x2c0afd0 0x2c3e3e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce0cff9700 0 client.6733 ms_handle_reset on v2:172.21.15.181:6839/3625693713 2020-11-16T22:32:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2bbae40 0x2cc09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x297ebb0 0x2cb7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 --2- 172.21.15.181:0/3193054323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29ec7b0 0x2cc04f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 >> 172.21.15.181:0/3193054323 conn(0x7fcdfc002320 msgr2=0x2c64200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 shutdown_connections 2020-11-16T22:32:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 1 -- 172.21.15.181:0/3193054323 wait complete. 2020-11-16T22:32:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce47a85740 20 client.6733 trim_cache size 0 max 16384 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 msgr2=0x7fce08057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 0x7fce08057520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fce200055a0 tx=0x7fce20009040).stop 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 msgr2=0x7fce3413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 0x7fce3413e860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fce2400eba0 tx=0x7fce24006df0).stop 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 shutdown_connections 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce08055060 0x7fce08057520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce341492c0 0x7fce34164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce340816e0 0x7fce34170810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 --2- 172.21.15.181:0/3805058731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce34145a30 0x7fce3413e860 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 >> 172.21.15.181:0/3805058731 conn(0x7fce340d68c0 msgr2=0x7fce340d47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 shutdown_connections 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:08.723+0000 7fce3359e700 1 -- 172.21.15.181:0/3805058731 wait complete. 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:32:08.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 1724883395, 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627778, 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3" 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:08.921 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:08.922 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 3304846807, 2020-11-16T22:32:08.922 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627779, 2020-11-16T22:32:08.922 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3/d4" 2020-11-16T22:32:08.922 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:08.922 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:32:08.923 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.067+0000 7f2b1ffff700 1 Processor -- start 2020-11-16T22:32:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.067+0000 7f2b1ffff700 1 -- start start 2020-11-16T22:32:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.067+0000 7f2b1ffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b20061100 0x7f2b200615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.067+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 0x7f2b2015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.067+0000 7f2b1ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b200cc7d0 con 0x7f2b20157a80 2020-11-16T22:32:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2015f0b0 con 0x7f2b20061100 2020-11-16T22:32:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2015dbe0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2b20157a80 0x7f2b200cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47666/0 (socket says 172.21.15.181:47666) 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1effd700 1 -- 172.21.15.181:0/3161081654 learned_addr learned my addr 172.21.15.181:0/3161081654 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1f7fe700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 0x7f2b2015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 --2- 172.21.15.181:0/3161081654 >> v2:172.21.15.175:3300/0 conn(0x7f2b20061100 0x7f2b200615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1dffb700 1 -- 172.21.15.181:0/3161081654 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500684871 0 0) 0x7f2b200cc7d0 con 0x7f2b20157a80 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1dffb700 1 -- 172.21.15.181:0/3161081654 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b00003240 con 0x7f2b20157a80 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 -- 172.21.15.181:0/3161081654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 msgr2=0x7f2b2015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 0x7f2b2015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 -- 172.21.15.181:0/3161081654 >> v1:172.21.15.181:6789/0 conn(0x7f2b20157a80 legacy=0x7f2b200cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 -- 172.21.15.181:0/3161081654 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b201611b0 con 0x7f2b20061100 2020-11-16T22:32:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1f7fe700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 0x7f2b2015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1e7fc700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200615b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f2b0800b9a0 tx=0x7f2b08006df0).ready entity=mon.0 client_cookie=373e8f567b6f6e78 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1dffb700 1 -- 172.21.15.181:0/3161081654 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08009680 con 0x7f2b20061100 2020-11-16T22:32:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1dffb700 1 -- 172.21.15.181:0/3161081654 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b080097e0 con 0x7f2b20061100 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1dffb700 1 -- 172.21.15.181:0/3161081654 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08009b60 con 0x7f2b20061100 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- 172.21.15.181:0/3161081654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 msgr2=0x7f2b200615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200615b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f2b0800b9a0 tx=0x7f2b08006df0).stop 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- 172.21.15.181:0/3161081654 shutdown_connections 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2015dfd0 0x7f2b2015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/3161081654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200615b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- 172.21.15.181:0/3161081654 >> 172.21.15.181:0/3161081654 conn(0x7f2b200cf630 msgr2=0x7f2b200cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:09.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- 172.21.15.181:0/3161081654 shutdown_connections 2020-11-16T22:32:09.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- 172.21.15.181:0/3161081654 wait complete. 2020-11-16T22:32:09.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 Processor -- start 2020-11-16T22:32:09.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.071+0000 7f2b1ffff700 1 -- start start 2020-11-16T22:32:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2016f330 con 0x7f2b20061100 2020-11-16T22:32:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2016f5d0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2016f870 con 0x7f2b20157a80 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33372/0 (socket says 172.21.15.181:33372) 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 -- 172.21.15.181:0/834646543 learned_addr learned my addr 172.21.15.181:0/834646543 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41212/0 (socket says 172.21.15.181:41212) 2020-11-16T22:32:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1effd700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 -- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 msgr2=0x7f2b200b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 -- 172.21.15.181:0/834646543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 msgr2=0x7f2b200b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 -- 172.21.15.181:0/834646543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2016fb10 con 0x7f2b2015dfd0 2020-11-16T22:32:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1e7fc700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1effd700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1f7fe700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f2b1800ec20 tx=0x7f2b18006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800c9b0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800cb10 con 0x7f2b2015dfd0 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2016fdb0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180123f0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.075+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b20149040 con 0x7f2b2015dfd0 2020-11-16T22:32:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b20044f60 con 0x7f2b2015dfd0 2020-11-16T22:32:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2b180127e0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b16ffd700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 0x7f2afc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b1806e2e0 con 0x7f2b2015dfd0 2020-11-16T22:32:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b1effd700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 0x7f2afc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.079+0000 7f2b1effd700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 0x7f2afc057520 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f2b10005920 tx=0x7f2b10009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.087+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2b18044540 con 0x7f2b2015dfd0 2020-11-16T22:32:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b20149290 con 0x7f2b2015dfd0 2020-11-16T22:32:09.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b16ffd700 1 -- 172.21.15.181:0/834646543 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f2b18047b60 con 0x7f2b2015dfd0 2020-11-16T22:32:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:09.274 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:09.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 msgr2=0x7f2afc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 0x7f2afc057520 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f2b10005920 tx=0x7f2b10009040).stop 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 msgr2=0x7f2b20148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f2b1800ec20 tx=0x7f2b18006df0).stop 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 shutdown_connections 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2afc055060 0x7f2afc057520 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b2015dfd0 0x7f2b20148b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b20061100 0x7f2b200b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 --2- 172.21.15.181:0/834646543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b20157a80 0x7f2b200b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 >> 172.21.15.181:0/834646543 conn(0x7f2b200cf630 msgr2=0x7f2b200cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 shutdown_connections 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.287+0000 7f2b1ffff700 1 -- 172.21.15.181:0/834646543 wait complete. 2020-11-16T22:32:09.277 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:09.289 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i damage ls 2020-11-16T22:32:09.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.427+0000 7f4ea62e2700 1 Processor -- start 2020-11-16T22:32:09.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.427+0000 7f4ea62e2700 1 -- start start 2020-11-16T22:32:09.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ea0142a40 0x7f4ea0063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea0063b10 con 0x7f4ea0160530 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea0062700 con 0x7f4ea0142660 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea00629a0 con 0x7f4ea0142a40 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ea0142a40 0x7f4ea0063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ea0142a40 0x7f4ea0063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33378/0 (socket says 172.21.15.181:33378) 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 -- 172.21.15.181:0/2758206749 learned_addr learned my addr 172.21.15.181:0/2758206749 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:09.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea52e0700 1 --1- 172.21.15.181:0/2758206749 >> v1:172.21.15.181:6789/0 conn(0x7f4ea0142660 0x7f4ea0063ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47680/0 (socket says 172.21.15.181:47680) 2020-11-16T22:32:09.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 -- 172.21.15.181:0/2758206749 >> v1:172.21.15.181:6789/0 conn(0x7f4ea0142660 legacy=0x7f4ea0063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 -- 172.21.15.181:0/2758206749 >> v1:172.21.15.175:6789/0 conn(0x7f4ea0160530 legacy=0x7f4ea0061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 -- 172.21.15.181:0/2758206749 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0062c40 con 0x7f4ea0142a40 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/2758206749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0142a40 0x7f4ea0063620 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f4e9c009d50 tx=0x7f4e9c009290).ready entity=mon.1 client_cookie=fc95acdd1abd25c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4e97fff700 1 -- 172.21.15.181:0/2758206749 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00bc80 con 0x7f4ea0142a40 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4e97fff700 1 -- 172.21.15.181:0/2758206749 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00bde0 con 0x7f4ea0142a40 2020-11-16T22:32:09.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4e97fff700 1 -- 172.21.15.181:0/2758206749 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011610 con 0x7f4ea0142a40 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- 172.21.15.181:0/2758206749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0142a40 msgr2=0x7f4ea0063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/2758206749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0142a40 0x7f4ea0063620 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f4e9c009d50 tx=0x7f4e9c009290).stop 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- 172.21.15.181:0/2758206749 shutdown_connections 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/2758206749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0142a40 0x7f4ea0063620 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- 172.21.15.181:0/2758206749 >> 172.21.15.181:0/2758206749 conn(0x7f4ea00d2dc0 msgr2=0x7f4ea00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- 172.21.15.181:0/2758206749 shutdown_connections 2020-11-16T22:32:09.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.431+0000 7f4ea62e2700 1 -- 172.21.15.181:0/2758206749 wait complete. 2020-11-16T22:32:09.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 Processor -- start 2020-11-16T22:32:09.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- start start 2020-11-16T22:32:09.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea0142660 0x7f4ea0141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea0142a40 0x7f4ea0060290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00cae20 con 0x7f4ea0142660 2020-11-16T22:32:09.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0170a40 con 0x7f4ea0160530 2020-11-16T22:32:09.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0141780 con 0x7f4ea0142a40 2020-11-16T22:32:09.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea4adf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea0142a40 0x7f4ea0060290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33382/0 (socket says 172.21.15.181:33382) 2020-11-16T22:32:09.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/1556986235 learned_addr learned my addr 172.21.15.181:0/1556986235 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:09.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea0142660 0x7f4ea0141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea0142a40 msgr2=0x7f4ea0060290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea0142a40 0x7f4ea0060290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/1556986235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea0142660 msgr2=0x7f4ea0141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:09.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea0142660 0x7f4ea0141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:09.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/1556986235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00cab80 con 0x7f4ea0160530 2020-11-16T22:32:09.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f4e9800c710 tx=0x7f4e98007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4e957fa700 1 -- 172.21.15.181:0/1556986235 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800e490 con 0x7f4ea0160530 2020-11-16T22:32:09.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4e957fa700 1 -- 172.21.15.181:0/1556986235 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800e5f0 con 0x7f4ea0160530 2020-11-16T22:32:09.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea0170c30 con 0x7f4ea0160530 2020-11-16T22:32:09.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4e957fa700 1 -- 172.21.15.181:0/1556986235 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800e970 con 0x7f4ea0160530 2020-11-16T22:32:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.435+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea0171410 con 0x7f4ea0160530 2020-11-16T22:32:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.439+0000 7f4e957fa700 1 -- 172.21.15.181:0/1556986235 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4e98010030 con 0x7f4ea0160530 2020-11-16T22:32:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.439+0000 7f4e957fa700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 0x7f4e78057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.439+0000 7f4e957fa700 1 -- 172.21.15.181:0/1556986235 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e9806d920 con 0x7f4ea0160530 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.439+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 0x7f4e78057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.439+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 0x7f4e78057520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f4e90007270 tx=0x7f4e90007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 Processor -- start 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 -- start start 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x206b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2068fd0 0x2062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ef44b0 con 0x2068fd0 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa8930 con 0x1ddcbb0 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e686f0 con 0x1e4a7b0 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33388/0 (socket says 172.21.15.181:33388) 2020-11-16T22:32:10.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 -- 172.21.15.181:0/3518688207 learned_addr learned my addr 172.21.15.181:0/3518688207 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:10.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x206b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2068fd0 0x2062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 -- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 msgr2=0x206b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x206b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 -- 172.21.15.181:0/3518688207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2068fd0 msgr2=0x2062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2068fd0 0x2062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 -- 172.21.15.181:0/3518688207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e635a0 con 0x1ddcbb0 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x206b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f4ea0146160 tx=0x7f4e9c016000).ready entity=mon.1 client_cookie=6e551ced72699ea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4e7e7fc700 1 -- 172.21.15.181:0/3518688207 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011610 con 0x1ddcbb0 2020-11-16T22:32:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4e7e7fc700 1 -- 172.21.15.181:0/3518688207 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c011a50 con 0x1ddcbb0 2020-11-16T22:32:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.443+0000 7f4e7e7fc700 1 -- 172.21.15.181:0/3518688207 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011bc0 con 0x1ddcbb0 2020-11-16T22:32:10.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 msgr2=0x2069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:10.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f4ea0146160 tx=0x7f4e9c016000).stop 2020-11-16T22:32:10.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/3518688207 shutdown_connections 2020-11-16T22:32:10.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2068fd0 0x2062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x206b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/3518688207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ddcbb0 0x2069670 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/3518688207 >> 172.21.15.181:0/3518688207 conn(0x2018e40 msgr2=0x20653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/3518688207 shutdown_connections 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/3518688207 wait complete. 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- start start 2020-11-16T22:32:10.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 Processor -- start 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e4a7b0 0x20ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f956e0 con 0x1e4a7b0 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f95820 con 0x2068fd0 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f96a90 con 0x1ddcbb0 2020-11-16T22:32:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4eb5fd5740 10 client.0 ll_register_callbacks cb 0x1f8bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:10.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea52e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33398/0 (socket says 172.21.15.181:33398) 2020-11-16T22:32:10.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2775911582 learned_addr learned my addr 172.21.15.181:0/2775911582 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:10.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e4a7b0 0x20ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 msgr2=0x20681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e4a7b0 msgr2=0x20ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e4a7b0 0x20ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.447+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2775911582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f86280 con 0x2068fd0 2020-11-16T22:32:10.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f4e98043d40 tx=0x7f4e98011000).ready entity=mon.1 client_cookie=ea40cb22deb121f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:10.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:32:10.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98046d30 con 0x2068fd0 2020-11-16T22:32:10.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e98046e90 con 0x2068fd0 2020-11-16T22:32:10.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20496c0 con 0x2068fd0 2020-11-16T22:32:10.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e98043840 con 0x2068fd0 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 10 client.6754 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x1dbbfd0 con 0x2068fd0 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20496c0 con 0x2068fd0 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f4e980467a0 con 0x2068fd0 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 10 client.6754 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 10 client.6754 resolve_mds: resolved ID 'i' to GID 6313 2020-11-16T22:32:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 20 client.6754 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x20b8260 0x20849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 4 client.6754 mds_command: new command op to 6313 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 --> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1e86c50 con 0x20b8260 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x20b8260 0x20849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 10 client.6754 ms_handle_connect on v2:172.21.15.175:6840/2108826019 2020-11-16T22:32:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x20b8260 0x20849b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=1c28a8a1720bc1b8 server_cookie=cefadb3632b5f4f in_seq=0 out_seq=0 2020-11-16T22:32:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/2775911582 <== mds.1 v2:172.21.15.175:6840/2108826019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1e86c50 con 0x20b8260 2020-11-16T22:32:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4e6f7fe700 10 client.6754 handle_command_reply: tid=0 2020-11-16T22:32:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 client.6754 shutdown 2020-11-16T22:32:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 msgr2=0x20ba980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f4e98043d40 tx=0x7f4e98011000).stop 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 shutdown_connections 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x20b8260 0x20849b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.451+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2068fd0 0x20ba980 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2775911582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e4a7b0 0x20ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4e6f7fe700 0 client.6754 ms_handle_reset on v2:172.21.15.175:6840/2108826019 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 >> 172.21.15.181:0/2775911582 conn(0x2018e40 msgr2=0x209e900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 shutdown_connections 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2775911582 wait complete. 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.455+0000 7f4eb5fd5740 20 client.6754 trim_cache size 0 max 16384 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 Processor -- start 2020-11-16T22:32:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- start start 2020-11-16T22:32:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 0x20681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ddcbb0 0x209c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f6de40 con 0x1ddcbb0 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x20b35f0 con 0x2018e40 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2063d90 con 0x1e4a7b0 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea52e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 0x20681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41246/0 (socket says 172.21.15.181:41246) 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea52e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 0x20681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33402/0 (socket says 172.21.15.181:33402) 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2223114668 learned_addr learned my addr 172.21.15.181:0/2223114668 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ddcbb0 0x209c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 msgr2=0x20681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:11.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 0x20681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2223114668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ddcbb0 msgr2=0x209c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:11.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ddcbb0 0x209c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 -- 172.21.15.181:0/2223114668 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740081d0 con 0x1e4a7b0 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f4e98007dc0 tx=0x7f4e98023000).ready entity=mon.2 client_cookie=27d3cac4e3fc1819 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4e7effd700 1 -- 172.21.15.181:0/2223114668 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98043840 con 0x1e4a7b0 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4e7effd700 1 -- 172.21.15.181:0/2223114668 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e980467a0 con 0x1e4a7b0 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4e7effd700 1 -- 172.21.15.181:0/2223114668 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98046910 con 0x1e4a7b0 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 msgr2=0x209cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f4e98007dc0 tx=0x7f4e98023000).stop 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2223114668 shutdown_connections 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e4a7b0 0x209cd20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2018e40 0x20681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/2223114668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ddcbb0 0x209c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2223114668 >> 172.21.15.181:0/2223114668 conn(0x7f4e74002320 msgr2=0x2065590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2223114668 shutdown_connections 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.471+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/2223114668 wait complete. 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 -- start start 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 Processor -- start 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 0x2069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x211f3a0 con 0x2018e40 2020-11-16T22:32:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ba9f0 con 0x1e4a7b0 2020-11-16T22:32:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x211e480 con 0x1ddcbb0 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33408/0 (socket says 172.21.15.181:33408) 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 -- 172.21.15.181:0/1409961515 learned_addr learned my addr 172.21.15.181:0/1409961515 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 10 client.0 ll_register_callbacks cb 0x20695b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 -- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 msgr2=0x20fd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20fd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 -- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 msgr2=0x2069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:11.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 0x2069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 0x2069fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 -- 172.21.15.181:0/1409961515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1dfa720 con 0x1e4a7b0 2020-11-16T22:32:11.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:32:11.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea5ae1700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 0x2069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:32:11.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4ea4adf700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f4e9c011000 tx=0x7f4e9c01f070).ready entity=mon.1 client_cookie=cb299d4a8587f7eb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:11.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4e6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011610 con 0x1e4a7b0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e9c00b750 con 0x1e4a7b0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.475+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e9c011a50 con 0x1e4a7b0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e9c026420 con 0x1e4a7b0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 10 client.6760 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x1e05e00 con 0x1e4a7b0 2020-11-16T22:32:11.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c011a50 con 0x1e4a7b0 2020-11-16T22:32:11.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f4e9c026580 con 0x1e4a7b0 2020-11-16T22:32:11.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 10 client.6760 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:11.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 10 client.6760 resolve_mds: resolved ID 'i' to GID 6313 2020-11-16T22:32:11.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 20 client.6760 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:11.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x2068fd0 0x209aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:11.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 4 client.6760 mds_command: new command op to 6313 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:32:11.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 --> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x1e8df50 con 0x2068fd0 2020-11-16T22:32:11.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x2068fd0 0x209aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:11.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 10 client.6760 ms_handle_connect on v2:172.21.15.175:6840/2108826019 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea52e0700 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x2068fd0 0x209aa60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=f64cb6f1f3ce86bb server_cookie=f5517ff61e0dace1 in_seq=0 out_seq=0 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 1 -- 172.21.15.181:0/1409961515 <== mds.1 v2:172.21.15.175:6840/2108826019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+283 (crc 0 0 0) 0x1e8df50 con 0x2068fd0 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 10 client.6760 handle_command_reply: tid=0 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 client.6760 shutdown 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 msgr2=0x20ba060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f4e9c011000 tx=0x7f4e9c01f070).stop 2020-11-16T22:32:11.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 shutdown_connections 2020-11-16T22:32:11.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:6840/2108826019,v1:172.21.15.175:6841/2108826019] conn(0x2068fd0 0x209aa60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2018e40 0x2069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ddcbb0 0x20fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:11.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 --2- 172.21.15.181:0/1409961515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e4a7b0 0x20ba060 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4e6f7fe700 0 client.6760 ms_handle_reset on v2:172.21.15.175:6840/2108826019 2020-11-16T22:32:12.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 >> 172.21.15.181:0/1409961515 conn(0x7f4e74002320 msgr2=0x2067fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:12.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 shutdown_connections 2020-11-16T22:32:12.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 1 -- 172.21.15.181:0/1409961515 wait complete. 2020-11-16T22:32:12.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4eb5fd5740 20 client.6760 trim_cache size 0 max 16384 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 msgr2=0x7f4e78057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 0x7f4e78057520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f4e90007270 tx=0x7f4e90007b80).stop 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 msgr2=0x7f4ea0060780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f4e9800c710 tx=0x7f4e98007b30).stop 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 shutdown_connections 2020-11-16T22:32:12.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4e78055060 0x7f4e78057520 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea0160530 0x7f4ea0060780 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ea0142660 0x7f4ea0141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 --2- 172.21.15.181:0/1556986235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea0142a40 0x7f4ea0060290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.479+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 >> 172.21.15.181:0/1556986235 conn(0x7f4ea00d2dc0 msgr2=0x7f4ea015d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:12.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.483+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 shutdown_connections 2020-11-16T22:32:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:09.483+0000 7f4ea62e2700 1 -- 172.21.15.181:0/1556986235 wait complete. 2020-11-16T22:32:12.294 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:32:12.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:12.294 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 757831454, 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627780, 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3/d4/d5" 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:12.295 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:12.296 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 1194764723, 2020-11-16T22:32:12.296 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627781, 2020-11-16T22:32:12.296 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2020-11-16T22:32:12.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:12.296 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:32:12.297 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:12.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 Processor -- start 2020-11-16T22:32:12.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 -- start start 2020-11-16T22:32:12.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbc98160690 0x7fbc98066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:12.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9815e2e0 con 0x7fbc980b2190 2020-11-16T22:32:12.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc980ce460 con 0x7fbc9815de80 2020-11-16T22:32:12.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d935700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc98066a60 con 0x7fbc98160690 2020-11-16T22:32:12.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc97fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbc9815de80 0x7fbc9815ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47722/0 (socket says 172.21.15.181:47722) 2020-11-16T22:32:12.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc97fff700 1 -- 172.21.15.181:0/92655098 learned_addr learned my addr 172.21.15.181:0/92655098 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:12.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc9d134700 1 --2- 172.21.15.181:0/92655098 >> v2:172.21.15.175:3300/0 conn(0x7fbc98160690 0x7fbc98066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:12.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687792176 0 0) 0x7fbc980ce460 con 0x7fbc9815de80 2020-11-16T22:32:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc7c003240 con 0x7fbc9815de80 2020-11-16T22:32:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299396584 0 0) 0x7fbc9815e2e0 con 0x7fbc980b2190 2020-11-16T22:32:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.443+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc980ce460 con 0x7fbc980b2190 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3494252468 0 0) 0x7fbc7c003240 con 0x7fbc9815de80 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 >> v2:172.21.15.175:3300/0 conn(0x7fbc98160690 msgr2=0x7fbc98066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 --2- 172.21.15.181:0/92655098 >> v2:172.21.15.175:3300/0 conn(0x7fbc98160690 0x7fbc98066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 >> v1:172.21.15.175:6789/0 conn(0x7fbc980b2190 legacy=0x7fbc980cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980ce700 con 0x7fbc9815de80 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7fbc840014e0 con 0x7fbc9815de80 2020-11-16T22:32:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 >> v1:172.21.15.181:6789/0 conn(0x7fbc9815de80 legacy=0x7fbc9815ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc7c003910 0x7fbc7c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 0x7fbc7c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c004e50 0x7fbc7c00d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98066a60 con 0x7fbc7c003910 2020-11-16T22:32:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c00de30 con 0x7fbc7c004e50 2020-11-16T22:32:12.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c00df70 con 0x7fbc7c0044e0 2020-11-16T22:32:12.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d134700 1 --2- 172.21.15.181:0/92655098 >> v2:172.21.15.175:3300/0 conn(0x7fbc98160690 0x7fbc98066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:12.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 0x7fbc7c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:12.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9c933700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc7c003910 0x7fbc7c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d134700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c004e50 0x7fbc7c00d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 -- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c004e50 msgr2=0x7fbc7c00d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c004e50 0x7fbc7c00d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 -- 172.21.15.181:0/92655098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc7c003910 msgr2=0x7fbc7c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc7c003910 0x7fbc7c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:12.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 -- 172.21.15.181:0/92655098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c003240 con 0x7fbc7c0044e0 2020-11-16T22:32:12.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d134700 1 -- 172.21.15.181:0/92655098 reap_dead start 2020-11-16T22:32:12.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc97fff700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 0x7fbc7c004960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fbc8400c160 tx=0x7fbc9815a510).ready entity=mon.2 client_cookie=e50e626f928b3fd2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:12.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400cd50 con 0x7fbc7c0044e0 2020-11-16T22:32:12.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc840070d0 con 0x7fbc7c0044e0 2020-11-16T22:32:13.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc977fe700 1 -- 172.21.15.181:0/92655098 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007450 con 0x7fbc7c0044e0 2020-11-16T22:32:13.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 -- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 msgr2=0x7fbc7c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 0x7fbc7c004960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fbc8400c160 tx=0x7fbc9815a510).stop 2020-11-16T22:32:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 -- 172.21.15.181:0/92655098 shutdown_connections 2020-11-16T22:32:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 --2- 172.21.15.181:0/92655098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7c0044e0 0x7fbc7c004960 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 -- 172.21.15.181:0/92655098 >> 172.21.15.181:0/92655098 conn(0x7fbc980cfe90 msgr2=0x7fbc980d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 -- 172.21.15.181:0/92655098 shutdown_connections 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.447+0000 7fbc9d935700 1 -- 172.21.15.181:0/92655098 wait complete. 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 Processor -- start 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- start start 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c0044e0 0x7fbc9815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc980d2a30 0x7fbc98134030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980d2e70 con 0x7fbc980d2a30 2020-11-16T22:32:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9815b810 con 0x7fbc7c0044e0 2020-11-16T22:32:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9815bab0 con 0x7fbc9815b330 2020-11-16T22:32:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:13.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9c933700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c0044e0 0x7fbc9815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:13.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41272/0 (socket says 172.21.15.181:41272) 2020-11-16T22:32:13.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 -- 172.21.15.181:0/3000056950 learned_addr learned my addr 172.21.15.181:0/3000056950 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:13.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d134700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc980d2a30 0x7fbc98134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 -- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c0044e0 msgr2=0x7fbc9815ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c0044e0 0x7fbc9815ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 -- 172.21.15.181:0/3000056950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc980d2a30 msgr2=0x7fbc98134030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc980d2a30 0x7fbc98134030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 -- 172.21.15.181:0/3000056950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9815bd50 con 0x7fbc9815b330 2020-11-16T22:32:13.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc97fff700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fbc84007740 tx=0x7fbc84015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:13.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84006e80 con 0x7fbc9815b330 2020-11-16T22:32:13.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8400cea0 con 0x7fbc9815b330 2020-11-16T22:32:13.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007450 con 0x7fbc9815b330 2020-11-16T22:32:13.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9815bff0 con 0x7fbc9815b330 2020-11-16T22:32:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.451+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9815c530 con 0x7fbc9815b330 2020-11-16T22:32:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98044f60 con 0x7fbc9815b330 2020-11-16T22:32:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbc84026020 con 0x7fbc9815b330 2020-11-16T22:32:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc94ff9700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 0x7fbc70057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbc8406f080 con 0x7fbc9815b330 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc9c933700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 0x7fbc70057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.455+0000 7fbc9c933700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 0x7fbc70057550 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fbc8c007ad0 tx=0x7fbc8c007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.463+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbc840452e0 con 0x7fbc9815b330 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.659+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc9805c4a0 con 0x7fbc9815b330 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.659+0000 7fbc94ff9700 1 -- 172.21.15.181:0/3000056950 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7fbc84048900 con 0x7fbc9815b330 2020-11-16T22:32:13.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 msgr2=0x7fbc70057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:13.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 0x7fbc70057550 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fbc8c007ad0 tx=0x7fbc8c007b00).stop 2020-11-16T22:32:13.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 msgr2=0x7fbc980d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:13.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fbc84007740 tx=0x7fbc84015000).stop 2020-11-16T22:32:13.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 shutdown_connections 2020-11-16T22:32:13.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbc70055090 0x7fbc70057550 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc980d2a30 0x7fbc98134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7c0044e0 0x7fbc9815ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 --2- 172.21.15.181:0/3000056950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc9815b330 0x7fbc980d2540 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 >> 172.21.15.181:0/3000056950 conn(0x7fbc980cfe90 msgr2=0x7fbc9815fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 shutdown_connections 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:12.663+0000 7fbc9d935700 1 -- 172.21.15.181:0/3000056950 wait complete. 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:13.667 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:13.668 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:13.870 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls 2020-11-16T22:32:14.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 Processor -- start 2020-11-16T22:32:14.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 -- start start 2020-11-16T22:32:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8ffc0b2900 0x7f8ffc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc15d990 con 0x7f8ffc0cf6b0 2020-11-16T22:32:14.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c8910 con 0x7f8ffc0b2900 2020-11-16T22:32:14.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f90025da700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0b3460 con 0x7f8ffc15d500 2020-11-16T22:32:14.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41282/0 (socket says 172.21.15.181:41282) 2020-11-16T22:32:14.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.015+0000 7f9001dd9700 1 -- 172.21.15.181:0/221754908 learned_addr learned my addr 172.21.15.181:0/221754908 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:14.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90015d8700 1 --2- 172.21.15.181:0/221754908 >> v2:172.21.15.175:3300/0 conn(0x7f8ffc0b2900 0x7f8ffc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f9001dd9700 1 -- 172.21.15.181:0/221754908 >> v2:172.21.15.175:3300/0 conn(0x7f8ffc0b2900 msgr2=0x7f8ffc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f9001dd9700 1 --2- 172.21.15.181:0/221754908 >> v2:172.21.15.175:3300/0 conn(0x7f8ffc0b2900 0x7f8ffc0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f9001dd9700 1 -- 172.21.15.181:0/221754908 >> v1:172.21.15.175:6789/0 conn(0x7f8ffc0cf6b0 legacy=0x7f8ffc15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f9001dd9700 1 -- 172.21.15.181:0/221754908 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0b31d0 con 0x7f8ffc15d500 2020-11-16T22:32:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f9001dd9700 1 --2- 172.21.15.181:0/221754908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f8ff4009d50 tx=0x7f8ff4009290).ready entity=mon.2 client_cookie=6f7af8012675ede2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f8ff3fff700 1 -- 172.21.15.181:0/221754908 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400bbc0 con 0x7f8ffc15d500 2020-11-16T22:32:14.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f8ff3fff700 1 -- 172.21.15.181:0/221754908 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400bd20 con 0x7f8ffc15d500 2020-11-16T22:32:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f8ff3fff700 1 -- 172.21.15.181:0/221754908 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4011610 con 0x7f8ffc15d500 2020-11-16T22:32:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- 172.21.15.181:0/221754908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 msgr2=0x7f8ffc0c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 --2- 172.21.15.181:0/221754908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f8ff4009d50 tx=0x7f8ff4009290).stop 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- 172.21.15.181:0/221754908 shutdown_connections 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 --2- 172.21.15.181:0/221754908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc0c7250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 --2- 172.21.15.181:0/221754908 >> v2:172.21.15.175:3300/0 conn(0x7f8ffc0b2900 0x7f8ffc0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- 172.21.15.181:0/221754908 >> 172.21.15.181:0/221754908 conn(0x7f8ffc0d2dc0 msgr2=0x7f8ffc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- 172.21.15.181:0/221754908 shutdown_connections 2020-11-16T22:32:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- 172.21.15.181:0/221754908 wait complete. 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 Processor -- start 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.019+0000 7f90025da700 1 -- start start 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0b2900 0x7f8ffc144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ffc0cf6b0 0x7f8ffc061a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc150a10 con 0x7f8ffc0b2900 2020-11-16T22:32:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc14f5b0 con 0x7f8ffc0cf6b0 2020-11-16T22:32:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc14f850 con 0x7f8ffc15d500 2020-11-16T22:32:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41288/0 (socket says 172.21.15.181:41288) 2020-11-16T22:32:14.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 -- 172.21.15.181:0/4143877525 learned_addr learned my addr 172.21.15.181:0/4143877525 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:14.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9000dd7700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ffc0cf6b0 0x7f8ffc061a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90015d8700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0b2900 0x7f8ffc144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 -- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ffc0cf6b0 msgr2=0x7f8ffc061a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ffc0cf6b0 0x7f8ffc061a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 -- 172.21.15.181:0/4143877525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0b2900 msgr2=0x7f8ffc144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0b2900 0x7f8ffc144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:14.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 -- 172.21.15.181:0/4143877525 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc14faf0 con 0x7f8ffc15d500 2020-11-16T22:32:14.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f9001dd9700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f8ff4009fd0 tx=0x7f8ff4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:14.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 -- 172.21.15.181:0/4143877525 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400bbc0 con 0x7f8ffc15d500 2020-11-16T22:32:14.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 -- 172.21.15.181:0/4143877525 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400be90 con 0x7f8ffc15d500 2020-11-16T22:32:14.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc150030 con 0x7f8ffc15d500 2020-11-16T22:32:14.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 -- 172.21.15.181:0/4143877525 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4010d10 con 0x7f8ffc15d500 2020-11-16T22:32:14.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc150570 con 0x7f8ffc15d500 2020-11-16T22:32:14.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 -- 172.21.15.181:0/4143877525 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8ff4017070 con 0x7f8ffc15d500 2020-11-16T22:32:14.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 0x7f8fd4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f8ff17fa700 1 -- 172.21.15.181:0/4143877525 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ff406fe70 con 0x7f8ffc15d500 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90015d8700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 0x7f8fd4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.023+0000 7f90015d8700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 0x7f8fd4057520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ff8006fd0 tx=0x7f8ff8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 Processor -- start 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 -- start start 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1ae0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1ade670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 0x1ad7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x19694b0 con 0x1addfd0 2020-11-16T22:32:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1d930 con 0x18bf7b0 2020-11-16T22:32:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x18dd6f0 con 0x1851bb0 2020-11-16T22:32:14.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1ae0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1ae0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33450/0 (socket says 172.21.15.181:33450) 2020-11-16T22:32:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f90015d8700 1 -- 172.21.15.181:0/812986681 learned_addr learned my addr 172.21.15.181:0/812986681 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.027+0000 7f9000dd7700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1ade670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9001dd9700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 0x1ad7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 -- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 msgr2=0x1ae0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1ae0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 -- 172.21.15.181:0/812986681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 msgr2=0x1ad7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 0x1ad7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 -- 172.21.15.181:0/812986681 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18d85a0 con 0x1851bb0 2020-11-16T22:32:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9001dd9700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 0x1ad7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:15.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f9000dd7700 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1ade670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f8ffc162120 tx=0x7f8fec007b90).ready entity=mon.2 client_cookie=9dd35788c4e6a1a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:15.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f8fde7fc700 1 -- 172.21.15.181:0/812986681 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007030 con 0x1851bb0 2020-11-16T22:32:15.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f8fde7fc700 1 -- 172.21.15.181:0/812986681 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec007190 con 0x1851bb0 2020-11-16T22:32:15.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f8fde7fc700 1 -- 172.21.15.181:0/812986681 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00f900 con 0x1851bb0 2020-11-16T22:32:15.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 msgr2=0x1ade670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:15.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1ade670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f8ffc162120 tx=0x7f8fec007b90).stop 2020-11-16T22:32:15.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- 172.21.15.181:0/812986681 shutdown_connections 2020-11-16T22:32:15.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1addfd0 0x1ad7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1ae0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 --2- 172.21.15.181:0/812986681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1ade670 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- 172.21.15.181:0/812986681 >> 172.21.15.181:0/812986681 conn(0x1a8de40 msgr2=0x1ada3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- 172.21.15.181:0/812986681 shutdown_connections 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- 172.21.15.181:0/812986681 wait complete. 2020-11-16T22:32:15.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 -- start start 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.031+0000 7f90122cd740 1 Processor -- start 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x18bf7b0 0x1ad9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1addfd0 0x1ada370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a0a6e0 con 0x18bf7b0 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a0a820 con 0x1addfd0 2020-11-16T22:32:15.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a0ba90 con 0x1851bb0 2020-11-16T22:32:15.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 10 client.0 ll_register_callbacks cb 0x1a00ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:15.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41298/0 (socket says 172.21.15.181:41298) 2020-11-16T22:32:15.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 -- 172.21.15.181:0/3277199755 learned_addr learned my addr 172.21.15.181:0/3277199755 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f9001dd9700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1addfd0 0x1ada370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f9000dd7700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x18bf7b0 0x1ad9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 -- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1addfd0 msgr2=0x1ada370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1addfd0 0x1ada370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 -- 172.21.15.181:0/3277199755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x18bf7b0 msgr2=0x1ad9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:15.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x18bf7b0 0x1ad9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 -- 172.21.15.181:0/3277199755 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19fb280 con 0x1851bb0 2020-11-16T22:32:15.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90015d8700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f8ff8011e10 tx=0x7f8ff8013000).ready entity=mon.2 client_cookie=48a47c562b2d43f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:15.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:32:15.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8012a30 con 0x1851bb0 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ff8012b90 con 0x1851bb0 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1abe6c0 con 0x1851bb0 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff8021450 con 0x1851bb0 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 10 client.6263 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x1830fd0 con 0x1851bb0 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1abe6c0 con 0x1851bb0 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f8ff80215b0 con 0x1851bb0 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 10 client.6263 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 10 client.6263 resolve_mds: resolved ID 'd' to GID 6295 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 20 client.6263 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1b40630 0x1b40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 4 client.6263 mds_command: new command op to 6295 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 --> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x18fbc50 con 0x1b40630 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f9000dd7700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1b40630 0x1b40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f8fcb7fe700 10 client.6263 ms_handle_connect on v2:172.21.15.181:6836/461666765 2020-11-16T22:32:15.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.035+0000 7f9000dd7700 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1b40630 0x1b40a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=1359bb7482dd22ad server_cookie=797af01786673225 in_seq=0 out_seq=0 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/3277199755 <== mds.2 v2:172.21.15.181:6836/461666765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x18fbc50 con 0x1b40630 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f8fcb7fe700 10 client.6263 handle_command_reply: tid=0 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 client.6263 shutdown 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 msgr2=0x1b0e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f8ff8011e10 tx=0x7f8ff8013000).stop 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 shutdown_connections 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1b40630 0x1b40a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1addfd0 0x1ada370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1851bb0 0x1b0e610 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f8fcb7fe700 0 client.6263 ms_handle_reset on v2:172.21.15.181:6836/461666765 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 --2- 172.21.15.181:0/3277199755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x18bf7b0 0x1ad9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 >> 172.21.15.181:0/3277199755 conn(0x1a8de40 msgr2=0x1b11270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 shutdown_connections 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 1 -- 172.21.15.181:0/3277199755 wait complete. 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.039+0000 7f90122cd740 20 client.6263 trim_cache size 0 max 16384 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 Processor -- start 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 -- start start 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x19e2e40 con 0x1851bb0 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b3e3d0 con 0x18bf7b0 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ad8910 con 0x1a8de40 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90015d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41308/0 (socket says 172.21.15.181:41308) 2020-11-16T22:32:16.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33468/0 (socket says 172.21.15.181:33468) 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90015d8700 1 -- 172.21.15.181:0/3713893398 learned_addr learned my addr 172.21.15.181:0/3713893398 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9000dd7700 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 -- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 msgr2=0x1abe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 -- 172.21.15.181:0/3713893398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 msgr2=0x1b27b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b27b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 -- 172.21.15.181:0/3713893398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00081d0 con 0x18bf7b0 2020-11-16T22:32:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f90015d8700 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f9001dd9700 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f8ff4049a90 tx=0x7f8ff4012000).ready entity=mon.1 client_cookie=988ec6b2a8f570c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f8fdeffd700 1 -- 172.21.15.181:0/3713893398 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40496f0 con 0x18bf7b0 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f8fdeffd700 1 -- 172.21.15.181:0/3713893398 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff4019c50 con 0x18bf7b0 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.055+0000 7f8fdeffd700 1 -- 172.21.15.181:0/3713893398 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4049280 con 0x18bf7b0 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 msgr2=0x1b28030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f8ff4049a90 tx=0x7f8ff4012000).stop 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- 172.21.15.181:0/3713893398 shutdown_connections 2020-11-16T22:32:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x18bf7b0 0x1b28030 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1a8de40 0x1abe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 --2- 172.21.15.181:0/3713893398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b27b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- 172.21.15.181:0/3713893398 >> 172.21.15.181:0/3713893398 conn(0x7f8fd0002320 msgr2=0x1b28d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- 172.21.15.181:0/3713893398 shutdown_connections 2020-11-16T22:32:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- 172.21.15.181:0/3713893398 wait complete. 2020-11-16T22:32:16.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 -- start start 2020-11-16T22:32:16.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.059+0000 7f90122cd740 1 Processor -- start 2020-11-16T22:32:16.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a8de40 0x1ad2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0ebf0 con 0x1851bb0 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b33700 con 0x1a8de40 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b339a0 con 0x18bf7b0 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 10 client.0 ll_register_callbacks cb 0x1ade850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:32:16.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41312/0 (socket says 172.21.15.181:41312) 2020-11-16T22:32:16.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 -- 172.21.15.181:0/2228537304 learned_addr learned my addr 172.21.15.181:0/2228537304 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:16.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9001dd9700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a8de40 0x1ad2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90015d8700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:16.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 -- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a8de40 msgr2=0x1ad2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:16.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a8de40 0x1ad2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 -- 172.21.15.181:0/2228537304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 msgr2=0x1b11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 -- 172.21.15.181:0/2228537304 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x186f720 con 0x18bf7b0 2020-11-16T22:32:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f9000dd7700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f8ffc062a10 tx=0x7f8fec025040).ready entity=mon.2 client_cookie=d0e89b836ad463b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec02b070 con 0x18bf7b0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8fec0260c0 con 0x18bf7b0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fec007190 con 0x18bf7b0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fec00f900 con 0x18bf7b0 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 10 client.6269 fetch_fsmap learned FSMap version 73 2020-11-16T22:32:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x187ae00 con 0x18bf7b0 2020-11-16T22:32:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec007190 con 0x18bf7b0 2020-11-16T22:32:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 73) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f8fec003600 con 0x18bf7b0 2020-11-16T22:32:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 10 client.6269 fetch_fsmap finished waiting for FSMap version 73 2020-11-16T22:32:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 10 client.6269 resolve_mds: resolved ID 'd' to GID 6295 2020-11-16T22:32:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 20 client.6269 populate_metadata read hostname 'smithi181' 2020-11-16T22:32:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1addfd0 0x1afbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 4 client.6269 mds_command: new command op to 6295 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:32:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 --> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x1902f50 con 0x1addfd0 2020-11-16T22:32:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90015d8700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1addfd0 0x1afbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 10 client.6269 ms_handle_connect on v2:172.21.15.181:6836/461666765 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90015d8700 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1addfd0 0x1afbe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=7ec139a6845ca7e4 server_cookie=90e783e7db6e393e in_seq=0 out_seq=0 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 1 -- 172.21.15.181:0/2228537304 <== mds.2 v2:172.21.15.181:6836/461666765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+296 (crc 0 0 0) 0x1902f50 con 0x1addfd0 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 10 client.6269 handle_command_reply: tid=0 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 client.6269 shutdown 2020-11-16T22:32:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 msgr2=0x1b28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f8ffc062a10 tx=0x7f8fec025040).stop 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 shutdown_connections 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:6836/461666765,v1:172.21.15.181:6837/461666765] conn(0x1addfd0 0x1afbe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a8de40 0x1ad2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f8fcb7fe700 0 client.6269 ms_handle_reset on v2:172.21.15.181:6836/461666765 2020-11-16T22:32:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.063+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1851bb0 0x1b11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90122cd740 1 --2- 172.21.15.181:0/2228537304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x18bf7b0 0x1b28fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 >> 172.21.15.181:0/2228537304 conn(0x7f8fd0002320 msgr2=0x1b2e930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 shutdown_connections 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90122cd740 1 -- 172.21.15.181:0/2228537304 wait complete. 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90122cd740 20 client.6269 trim_cache size 0 max 16384 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 msgr2=0x7f8fd4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 0x7f8fd4057520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ff8006fd0 tx=0x7f8ff8006df0).stop 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 msgr2=0x7f8ffc061f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f8ff4009fd0 tx=0x7f8ff4016000).stop 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 shutdown_connections 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8fd4055060 0x7f8fd4057520 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ffc15d500 0x7f8ffc061f30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0b2900 0x7f8ffc144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 --2- 172.21.15.181:0/4143877525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ffc0cf6b0 0x7f8ffc061a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 >> 172.21.15.181:0/4143877525 conn(0x7f8ffc0d2dc0 msgr2=0x7f8ffc065390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 shutdown_connections 2020-11-16T22:32:17.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:14.067+0000 7f90025da700 1 -- 172.21.15.181:0/4143877525 wait complete. 2020-11-16T22:32:17.482 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:32:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 1197180725, 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627783, 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "damage_type": "backtrace", 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 3064901440, 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 1099511627782, 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:17.484 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:32:17.485 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:32:17.485 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:17.522 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:17.539 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:17.557 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:32:17.557 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:32:17.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:32:17.558 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:32:17.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:32:17.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:17.645 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:32:17.666 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:32:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 Processor -- start 2020-11-16T22:32:17.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 -- start start 2020-11-16T22:32:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed080116c0 0x7fed080f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 0x7fed0805b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:17.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed080e9e70 con 0x7fed080116c0 2020-11-16T22:32:18.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0805def0 con 0x7fed08006f40 2020-11-16T22:32:18.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1629f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed08005d80 con 0x7fed08011aa0 2020-11-16T22:32:18.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed15a9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 0x7fed0805b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed14a9c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed14a9c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41320/0 (socket says 172.21.15.181:41320) 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed15a9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 0x7fed0805b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33478/0 (socket says 172.21.15.181:33478) 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed14a9c700 1 -- 172.21.15.181:0/3152480636 learned_addr learned my addr 172.21.15.181:0/3152480636 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.814+0000 7fed1529d700 1 --2- 172.21.15.181:0/3152480636 >> v2:172.21.15.175:3300/0 conn(0x7fed080116c0 0x7fed080f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 -- 172.21.15.181:0/3152480636 >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 msgr2=0x7fed0805b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 --2- 172.21.15.181:0/3152480636 >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 0x7fed0805b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 -- 172.21.15.181:0/3152480636 >> v2:172.21.15.175:3300/0 conn(0x7fed080116c0 msgr2=0x7fed080f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:18.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 --2- 172.21.15.181:0/3152480636 >> v2:172.21.15.175:3300/0 conn(0x7fed080116c0 0x7fed080f0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 -- 172.21.15.181:0/3152480636 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08006020 con 0x7fed08011aa0 2020-11-16T22:32:18.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed14a9c700 1 --2- 172.21.15.181:0/3152480636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fed080132c0 tx=0x7fed04006eb0).ready entity=mon.2 client_cookie=a00b6ab79aeb70e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:18.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed03fff700 1 -- 172.21.15.181:0/3152480636 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04009810 con 0x7fed08011aa0 2020-11-16T22:32:18.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed03fff700 1 -- 172.21.15.181:0/3152480636 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04009970 con 0x7fed08011aa0 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed03fff700 1 -- 172.21.15.181:0/3152480636 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04009cf0 con 0x7fed08011aa0 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- 172.21.15.181:0/3152480636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 msgr2=0x7fed0805da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 --2- 172.21.15.181:0/3152480636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fed080132c0 tx=0x7fed04006eb0).stop 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- 172.21.15.181:0/3152480636 shutdown_connections 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 --2- 172.21.15.181:0/3152480636 >> v2:172.21.15.181:3300/0 conn(0x7fed08006f40 0x7fed0805b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 --2- 172.21.15.181:0/3152480636 >> v2:172.21.15.175:3300/0 conn(0x7fed080116c0 0x7fed080f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 --2- 172.21.15.181:0/3152480636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08011aa0 0x7fed0805da00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- 172.21.15.181:0/3152480636 >> 172.21.15.181:0/3152480636 conn(0x7fed080661c0 msgr2=0x7fed080665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- 172.21.15.181:0/3152480636 shutdown_connections 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- 172.21.15.181:0/3152480636 wait complete. 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 Processor -- start 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.818+0000 7fed1629f700 1 -- start start 2020-11-16T22:32:18.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 0x7fed080e9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:18.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:18.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed080d45f0 0x7fed08065240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed080ff540 con 0x7fed080d45f0 2020-11-16T22:32:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed080ff730 con 0x7fed080116c0 2020-11-16T22:32:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed080ff9d0 con 0x7fed08006f40 2020-11-16T22:32:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1529d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 0x7fed080e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33482/0 (socket says 172.21.15.181:33482) 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1529d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 0x7fed080e9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41322/0 (socket says 172.21.15.181:41322) 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 -- 172.21.15.181:0/739386002 learned_addr learned my addr 172.21.15.181:0/739386002 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed15a9e700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed080d45f0 0x7fed08065240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 -- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 msgr2=0x7fed080e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 0x7fed080e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 -- 172.21.15.181:0/739386002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed080d45f0 msgr2=0x7fed08065240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed080d45f0 0x7fed08065240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 -- 172.21.15.181:0/739386002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080ffc70 con 0x7fed080116c0 2020-11-16T22:32:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed14a9c700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fed04006fd0 tx=0x7fed04015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04009cb0 con 0x7fed080116c0 2020-11-16T22:32:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04011c80 con 0x7fed080116c0 2020-11-16T22:32:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed080fff10 con 0x7fed080116c0 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0401e460 con 0x7fed080116c0 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed08100450 con 0x7fed080116c0 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.822+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed080d49d0 con 0x7fed080116c0 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.826+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fed04025030 con 0x7fed080116c0 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.826+0000 7fed017fa700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 0x7fece8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.826+0000 7fed1529d700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 0x7fece8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.826+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fed0406fc00 con 0x7fed080116c0 2020-11-16T22:32:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.826+0000 7fed1529d700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 0x7fece8057520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fed0c006fd0 tx=0x7fed0c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.830+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fed04045e80 con 0x7fed080116c0 2020-11-16T22:32:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.986+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fed08004fb0 con 0x7fed080116c0 2020-11-16T22:32:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed017fa700 1 -- 172.21.15.181:0/739386002 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed0402d020 con 0x7fed080116c0 2020-11-16T22:32:18.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 msgr2=0x7fece8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 0x7fece8057520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fed0c006fd0 tx=0x7fed0c006df0).stop 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 msgr2=0x7fed080d4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fed04006fd0 tx=0x7fed04015000).stop 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 shutdown_connections 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fece8055060 0x7fece8057520 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed080d45f0 0x7fed08065240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08006f40 0x7fed080e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 --2- 172.21.15.181:0/739386002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed080116c0 0x7fed080d4100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 >> 172.21.15.181:0/739386002 conn(0x7fed080661c0 msgr2=0x7fed080666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 shutdown_connections 2020-11-16T22:32:18.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:17.994+0000 7fed1629f700 1 -- 172.21.15.181:0/739386002 wait complete. 2020-11-16T22:32:18.833 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2020-11-16T22:32:19.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.178+0000 7f1e3a904700 1 Processor -- start 2020-11-16T22:32:19.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.178+0000 7f1e3a904700 1 -- start start 2020-11-16T22:32:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3415d9b0 con 0x7f1e3415d500 2020-11-16T22:32:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e340b31d0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340c8910 con 0x7f1e340b2900 2020-11-16T22:32:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41330/0 (socket says 172.21.15.181:41330) 2020-11-16T22:32:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 -- 172.21.15.181:0/1255878070 learned_addr learned my addr 172.21.15.181:0/1255878070 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 -- 172.21.15.181:0/1255878070 >> v1:172.21.15.181:6789/0 conn(0x7f1e340cf6b0 legacy=0x7f1e3415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 -- 172.21.15.181:0/1255878070 >> v1:172.21.15.175:6789/0 conn(0x7f1e3415d500 legacy=0x7f1e340c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 -- 172.21.15.181:0/1255878070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340c8130 con 0x7f1e340b2900 2020-11-16T22:32:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e38900700 1 -- 172.21.15.181:0/1255878070 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613490901 0 0) 0x7f1e340b31d0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e39902700 1 --2- 172.21.15.181:0/1255878070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1e3000ba50 tx=0x7f1e30006eb0).ready entity=mon.2 client_cookie=37b26a63e604a956 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e38900700 1 -- 172.21.15.181:0/1255878070 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30009a00 con 0x7f1e340b2900 2020-11-16T22:32:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e38900700 1 -- 172.21.15.181:0/1255878070 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30009b60 con 0x7f1e340b2900 2020-11-16T22:32:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e38900700 1 -- 172.21.15.181:0/1255878070 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011400 con 0x7f1e340b2900 2020-11-16T22:32:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- 172.21.15.181:0/1255878070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 msgr2=0x7f1e340b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 --2- 172.21.15.181:0/1255878070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1e3000ba50 tx=0x7f1e30006eb0).stop 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- 172.21.15.181:0/1255878070 shutdown_connections 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 --2- 172.21.15.181:0/1255878070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340b2900 0x7f1e340b2ce0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- 172.21.15.181:0/1255878070 >> 172.21.15.181:0/1255878070 conn(0x7f1e340d2dc0 msgr2=0x7f1e340ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- 172.21.15.181:0/1255878070 shutdown_connections 2020-11-16T22:32:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 -- 172.21.15.181:0/1255878070 wait complete. 2020-11-16T22:32:19.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.182+0000 7f1e3a904700 1 Processor -- start 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- start start 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e340b2900 0x7f1e340cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e3415d500 0x7f1e34162880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34155ca0 con 0x7f1e340b2900 2020-11-16T22:32:19.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34155f40 con 0x7f1e3415d500 2020-11-16T22:32:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340ccba0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:19.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a103700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e3415d500 0x7f1e34162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:19.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41338/0 (socket says 172.21.15.181:41338) 2020-11-16T22:32:19.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 -- 172.21.15.181:0/352129924 learned_addr learned my addr 172.21.15.181:0/352129924 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39902700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e340b2900 0x7f1e340cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 -- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e3415d500 msgr2=0x7f1e34162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e3415d500 0x7f1e34162880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 -- 172.21.15.181:0/352129924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e340b2900 msgr2=0x7f1e340cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e340b2900 0x7f1e340cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 -- 172.21.15.181:0/352129924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340cce40 con 0x7f1e340cf6b0 2020-11-16T22:32:19.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e39101700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1e2400c730 tx=0x7f1e24007a40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:19.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400e490 con 0x7f1e340cf6b0 2020-11-16T22:32:19.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2400e5f0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e340cd0e0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400e9b0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.186+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e34158430 con 0x7f1e340cf6b0 2020-11-16T22:32:19.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e34044f60 con 0x7f1e340cf6b0 2020-11-16T22:32:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1e24019030 con 0x7f1e340cf6b0 2020-11-16T22:32:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e21ffb700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 0x7f1e0c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e2406d2d0 con 0x7f1e340cf6b0 2020-11-16T22:32:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e39902700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 0x7f1e0c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.190+0000 7f1e39902700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 0x7f1e0c057520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f1e3000b1b0 tx=0x7f1e30015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.198+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1e24043530 con 0x7f1e340cf6b0 2020-11-16T22:32:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.406+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7f1e3405d550 con 0x7f1e340cf6b0 2020-11-16T22:32:19.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.842+0000 7f1e21ffb700 1 -- 172.21.15.181:0/352129924 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v268) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1e24017020 con 0x7f1e340cf6b0 2020-11-16T22:32:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 msgr2=0x7f1e0c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 0x7f1e0c057520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f1e3000b1b0 tx=0x7f1e30015000).stop 2020-11-16T22:32:20.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 msgr2=0x7f1e34162390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:20.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1e2400c730 tx=0x7f1e24007a40).stop 2020-11-16T22:32:20.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 shutdown_connections 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e0c055060 0x7f1e0c057520 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e3415d500 0x7f1e34162880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e340b2900 0x7f1e340cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 --2- 172.21.15.181:0/352129924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e340cf6b0 0x7f1e34162390 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 >> 172.21.15.181:0/352129924 conn(0x7f1e340d2dc0 msgr2=0x7f1e3415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 shutdown_connections 2020-11-16T22:32:20.040 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:19.846+0000 7f1e3a904700 1 -- 172.21.15.181:0/352129924 wait complete. 2020-11-16T22:32:20.041 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:32:20.041 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:32:20.042 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2020-11-16T22:32:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 Processor -- start 2020-11-16T22:32:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 -- start start 2020-11-16T22:32:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 0x7fa3bc0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc0b31d0 con 0x7fa3bc0cf6b0 2020-11-16T22:32:20.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c8910 con 0x7fa3bc15d500 2020-11-16T22:32:20.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3bb59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0b3460 con 0x7fa3bc0b2900 2020-11-16T22:32:20.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3ba59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.186+0000 7fa3ba59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33502/0 (socket says 172.21.15.181:33502) 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 -- 172.21.15.181:0/3712534522 learned_addr learned my addr 172.21.15.181:0/3712534522 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bad9d700 1 --2- 172.21.15.181:0/3712534522 >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 0x7fa3bc0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 -- 172.21.15.181:0/3712534522 >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 msgr2=0x7fa3bc0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 --2- 172.21.15.181:0/3712534522 >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 0x7fa3bc0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 -- 172.21.15.181:0/3712534522 >> v1:172.21.15.175:6789/0 conn(0x7fa3bc0cf6b0 legacy=0x7fa3bc15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:20.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 -- 172.21.15.181:0/3712534522 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0c8130 con 0x7fa3bc0b2900 2020-11-16T22:32:20.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bad9d700 1 --2- 172.21.15.181:0/3712534522 >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 0x7fa3bc0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:20.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3ba59c700 1 --2- 172.21.15.181:0/3712534522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fa3bc066500 tx=0x7fa3b0006eb0).ready entity=mon.1 client_cookie=ee2305dec2245438 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:20.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3b959a700 1 -- 172.21.15.181:0/3712534522 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00098d0 con 0x7fa3bc0b2900 2020-11-16T22:32:20.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3b959a700 1 -- 172.21.15.181:0/3712534522 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0009a30 con 0x7fa3bc0b2900 2020-11-16T22:32:20.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3b959a700 1 -- 172.21.15.181:0/3712534522 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0009db0 con 0x7fa3bc0b2900 2020-11-16T22:32:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 -- 172.21.15.181:0/3712534522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0b2900 msgr2=0x7fa3bc0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/3712534522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fa3bc066500 tx=0x7fa3b0006eb0).stop 2020-11-16T22:32:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 -- 172.21.15.181:0/3712534522 shutdown_connections 2020-11-16T22:32:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/3712534522 >> v2:172.21.15.175:3300/0 conn(0x7fa3bc15d500 0x7fa3bc0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/3712534522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0b2900 0x7fa3bc0b2ce0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 -- 172.21.15.181:0/3712534522 >> 172.21.15.181:0/3712534522 conn(0x7fa3bc0d2dc0 msgr2=0x7fa3bc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 -- 172.21.15.181:0/3712534522 shutdown_connections 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.190+0000 7fa3bb59e700 1 -- 172.21.15.181:0/3712534522 wait complete. 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 Processor -- start 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 -- start start 2020-11-16T22:32:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 0x7fa3bc147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 0x7fa3bc144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d04f0 con 0x7fa3bc15d500 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d0790 con 0x7fa3bc0cf6b0 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bb59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d0a30 con 0x7fa3bc0b2900 2020-11-16T22:32:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41350/0 (socket says 172.21.15.181:41350) 2020-11-16T22:32:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 -- 172.21.15.181:0/1250570435 learned_addr learned my addr 172.21.15.181:0/1250570435 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3b9d9b700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 0x7fa3bc147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bad9d700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 0x7fa3bc144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 -- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 msgr2=0x7fa3bc147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 0x7fa3bc147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 -- 172.21.15.181:0/1250570435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 msgr2=0x7fa3bc144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 0x7fa3bc144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3ba59c700 1 -- 172.21.15.181:0/1250570435 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0d0cd0 con 0x7fa3bc0b2900 2020-11-16T22:32:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3bad9d700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 0x7fa3bc144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.194+0000 7fa3b9d9b700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 0x7fa3bc147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3ba59c700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa3b000b1b0 tx=0x7fa3b0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0011e50 con 0x7fa3bc0b2900 2020-11-16T22:32:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b001e460 con 0x7fa3bc0b2900 2020-11-16T22:32:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc0d0f70 con 0x7fa3bc0b2900 2020-11-16T22:32:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b001e800 con 0x7fa3bc0b2900 2020-11-16T22:32:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc0d01f0 con 0x7fa3bc0b2900 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc044f60 con 0x7fa3bc0b2900 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa3b0025030 con 0x7fa3bc0b2900 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 0x7fa390057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa3b006f400 con 0x7fa3bc0b2900 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.198+0000 7fa3b9d9b700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 0x7fa390057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.202+0000 7fa3b9d9b700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 0x7fa390057520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fa3bc0b4d70 tx=0x7fa3ac009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.206+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa3b0045710 con 0x7fa3bc0b2900 2020-11-16T22:32:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.414+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7fa3bc143760 con 0x7fa3bc0b2900 2020-11-16T22:32:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.858+0000 7fa3aa7fc700 1 -- 172.21.15.181:0/1250570435 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v269) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa3b0023070 con 0x7fa3bc0b2900 2020-11-16T22:32:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 msgr2=0x7fa390057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 0x7fa390057520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fa3bc0b4d70 tx=0x7fa3ac009040).stop 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 msgr2=0x7fa3bc146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa3b000b1b0 tx=0x7fa3b0015000).stop 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 shutdown_connections 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa390055060 0x7fa390057520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3bc15d500 0x7fa3bc144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3bc0b2900 0x7fa3bc146da0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 --2- 172.21.15.181:0/1250570435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3bc0cf6b0 0x7fa3bc147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 >> 172.21.15.181:0/1250570435 conn(0x7fa3bc0d2dc0 msgr2=0x7fa3bc0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 shutdown_connections 2020-11-16T22:32:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:20.862+0000 7fa3bb59e700 1 -- 172.21.15.181:0/1250570435 wait complete. 2020-11-16T22:32:21.204 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:32:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 Processor -- start 2020-11-16T22:32:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 -- start start 2020-11-16T22:32:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4fb413e870 0x7f4fb413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb415d180 con 0x7f4fb415fdc0 2020-11-16T22:32:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb413f140 con 0x7f4fb4061810 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb3fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb415d810 con 0x7f4fb413e870 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4fb4061810 0x7f4fb415e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47824/0 (socket says 172.21.15.181:47824) 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb37fe700 1 -- 172.21.15.181:0/2039712806 learned_addr learned my addr 172.21.15.181:0/2039712806 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.358+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/2039712806 >> v2:172.21.15.175:3300/0 conn(0x7f4fb413e870 0x7f4fb413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425971273 0 0) 0x7f4fac001450 con 0x7f4fb4061810 2020-11-16T22:32:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f94004350 con 0x7f4fb4061810 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1735615456 0 0) 0x7f4f94004350 con 0x7f4fb4061810 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 >> v2:172.21.15.175:3300/0 conn(0x7f4fb413e870 msgr2=0x7f4fb413ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 --2- 172.21.15.181:0/2039712806 >> v2:172.21.15.175:3300/0 conn(0x7f4fb413e870 0x7f4fb413ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 >> v1:172.21.15.175:6789/0 conn(0x7f4fb415fdc0 legacy=0x7f4fb4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb415dab0 con 0x7f4fb4061810 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f4fac0019f0 con 0x7f4fb4061810 2020-11-16T22:32:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 >> v1:172.21.15.181:6789/0 conn(0x7f4fb4061810 legacy=0x7f4fb415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f94004840 0x7f4f94004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4f940055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f94005b30 0x7f4f94005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb415d810 con 0x7f4f94005b30 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94003380 con 0x7f4f94005150 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94003540 con 0x7f4f94004840 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4f940055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f94005b30 0x7f4f94005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 -- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f94004840 msgr2=0x7f4f94004c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f94004840 0x7f4f94004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:32:21.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 -- 172.21.15.181:0/2039712806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f94005b30 msgr2=0x7f4f94005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:21.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f94005b30 0x7f4f94005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 -- 172.21.15.181:0/2039712806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb415dab0 con 0x7f4f94005150 2020-11-16T22:32:21.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f94005b30 0x7f4f94005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:21.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb37fe700 1 -- 172.21.15.181:0/2039712806 reap_dead start 2020-11-16T22:32:21.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4f940055f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f4f9c001940 tx=0x7f4f9c007b50).ready entity=mon.1 client_cookie=1a20fe03ef85d59c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:21.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00c830 con 0x7f4f94005150 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00c990 con 0x7f4f94005150 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb27fc700 1 -- 172.21.15.181:0/2039712806 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00cd10 con 0x7f4f94005150 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 -- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 msgr2=0x7f4f940055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4f940055f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f4f9c001940 tx=0x7f4f9c007b50).stop 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 -- 172.21.15.181:0/2039712806 shutdown_connections 2020-11-16T22:32:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 --2- 172.21.15.181:0/2039712806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4f940055f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 -- 172.21.15.181:0/2039712806 >> 172.21.15.181:0/2039712806 conn(0x7f4fb40cf630 msgr2=0x7f4fb40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 -- 172.21.15.181:0/2039712806 shutdown_connections 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.362+0000 7f4fb3fff700 1 -- 172.21.15.181:0/2039712806 wait complete. 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 Processor -- start 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 -- start start 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb40b6f10 0x7f4fb4060530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:21.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4060a20 0x7f4fb415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:22.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40b5b20 con 0x7f4fb4060a20 2020-11-16T22:32:22.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40b5d50 con 0x7f4f94005150 2020-11-16T22:32:22.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40b5ff0 con 0x7f4fb40b6f10 2020-11-16T22:32:22.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33528/0 (socket says 172.21.15.181:33528) 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 -- 172.21.15.181:0/3021320693 learned_addr learned my addr 172.21.15.181:0/3021320693 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb40b6f10 0x7f4fb4060530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4060a20 0x7f4fb415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 -- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb40b6f10 msgr2=0x7f4fb4060530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb40b6f10 0x7f4fb4060530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 -- 172.21.15.181:0/3021320693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4060a20 msgr2=0x7f4fb415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:22.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4060a20 0x7f4fb415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.366+0000 7f4fb2ffd700 1 -- 172.21.15.181:0/3021320693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40b6290 con 0x7f4f94005150 2020-11-16T22:32:22.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f4fa400e880 tx=0x7f4fa4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4016070 con 0x7f4f94005150 2020-11-16T22:32:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4010590 con 0x7f4f94005150 2020-11-16T22:32:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fb3fff700 1 -- 172.21.15.181:0/3021320693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb40b6530 con 0x7f4f94005150 2020-11-16T22:32:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fb3fff700 1 -- 172.21.15.181:0/3021320693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb40b6a70 con 0x7f4f94005150 2020-11-16T22:32:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c004f90 con 0x7f4f94005150 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4010930 con 0x7f4f94005150 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.370+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4fa4010ce0 con 0x7f4f94005150 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.374+0000 7f4fab7fe700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 0x7f4f88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.374+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4fa406e5e0 con 0x7f4f94005150 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.374+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 0x7f4f88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.374+0000 7f4faa7fc700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 0x7f4f88057520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f4fb40ce0d0 tx=0x7f4f9c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.378+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4fa40448c0 con 0x7f4f94005150 2020-11-16T22:32:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.526+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f4f8c003b20 con 0x7f4f94005150 2020-11-16T22:32:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.538+0000 7f4fab7fe700 1 -- 172.21.15.181:0/3021320693 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fa4047ee0 con 0x7f4f94005150 2020-11-16T22:32:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 msgr2=0x7f4f88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 0x7f4f88057520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f4fb40ce0d0 tx=0x7f4f9c016000).stop 2020-11-16T22:32:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 msgr2=0x7f4fb416c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f4fa400e880 tx=0x7f4fa4006df0).stop 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 shutdown_connections 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f88055060 0x7f4f88057520 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb4060a20 0x7f4fb415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f94005150 0x7f4fb416c270 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 --2- 172.21.15.181:0/3021320693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb40b6f10 0x7f4fb4060530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 >> 172.21.15.181:0/3021320693 conn(0x7f4fb40cf630 msgr2=0x7f4fb40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 shutdown_connections 2020-11-16T22:32:22.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:21.542+0000 7f4fa8ff9700 1 -- 172.21.15.181:0/3021320693 wait complete. 2020-11-16T22:32:22.576 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:32:22.576 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:22.614 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:22.631 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:32:22.647 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:32:22.850 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:32:22.850 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:22.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 Processor -- start 2020-11-16T22:32:22.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 -- start start 2020-11-16T22:32:22.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 0x7efc08162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:22.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efc08162f20 0x7efc08163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:22.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc08163a40 0x7efc0815ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc081633a0 con 0x7efc08162f20 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc08161870 con 0x7efc08163a40 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc0d464700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc08161b10 con 0x7efc081428e0 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc08163a40 0x7efc0815ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc06ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 0x7efc08162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc08163a40 0x7efc0815ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33540/0 (socket says 172.21.15.181:33540) 2020-11-16T22:32:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.994+0000 7efc06ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 0x7efc08162a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41380/0 (socket says 172.21.15.181:41380) 2020-11-16T22:32:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 -- 172.21.15.181:0/2957875759 learned_addr learned my addr 172.21.15.181:0/2957875759 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc067fc700 1 --2- 172.21.15.181:0/2957875759 >> v2:172.21.15.175:3300/0 conn(0x7efc08162f20 0x7efc08163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 -- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 msgr2=0x7efc08162a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 --2- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 0x7efc08162a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 -- 172.21.15.181:0/2957875759 >> v2:172.21.15.175:3300/0 conn(0x7efc08162f20 msgr2=0x7efc08163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 --2- 172.21.15.181:0/2957875759 >> v2:172.21.15.175:3300/0 conn(0x7efc08162f20 0x7efc08163550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 -- 172.21.15.181:0/2957875759 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08161db0 con 0x7efc08163a40 2020-11-16T22:32:23.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc077fe700 1 --2- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc08163a40 0x7efc0815ff80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7efc0815a5a0 tx=0x7efc00009290).ready entity=mon.1 client_cookie=13f344df40909d39 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc05ffb700 1 -- 172.21.15.181:0/2957875759 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0000bba0 con 0x7efc08163a40 2020-11-16T22:32:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc05ffb700 1 -- 172.21.15.181:0/2957875759 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0000bd00 con 0x7efc08163a40 2020-11-16T22:32:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc05ffb700 1 -- 172.21.15.181:0/2957875759 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00011610 con 0x7efc08163a40 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 -- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc08163a40 msgr2=0x7efc0815ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 --2- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc08163a40 0x7efc0815ff80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7efc0815a5a0 tx=0x7efc00009290).stop 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 -- 172.21.15.181:0/2957875759 shutdown_connections 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 --2- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc08163a40 0x7efc0815ff80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 --2- 172.21.15.181:0/2957875759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc081428e0 0x7efc08162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 --2- 172.21.15.181:0/2957875759 >> v2:172.21.15.175:3300/0 conn(0x7efc08162f20 0x7efc08163550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 -- 172.21.15.181:0/2957875759 >> 172.21.15.181:0/2957875759 conn(0x7efc08060bc0 msgr2=0x7efc080d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 -- 172.21.15.181:0/2957875759 shutdown_connections 2020-11-16T22:32:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:22.998+0000 7efc0d464700 1 -- 172.21.15.181:0/2957875759 wait complete. 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 Processor -- start 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- start start 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 0x7efc08159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc08163a40 0x7efc08159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:23.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0813ed40 con 0x7efc08163a40 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08141250 con 0x7efc081428e0 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc081414f0 con 0x7efc08162f20 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc067fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 0x7efc08159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33542/0 (socket says 172.21.15.181:33542) 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc067fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 0x7efc08159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41386/0 (socket says 172.21.15.181:41386) 2020-11-16T22:32:23.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 -- 172.21.15.181:0/1246431535 learned_addr learned my addr 172.21.15.181:0/1246431535 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc077fe700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc08163a40 0x7efc08159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 -- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 msgr2=0x7efc08159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 0x7efc08159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 -- 172.21.15.181:0/1246431535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc08163a40 msgr2=0x7efc08159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc08163a40 0x7efc08159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 -- 172.21.15.181:0/1246431535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08141790 con 0x7efc081428e0 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc06ffd700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7efbf800ebd0 tx=0x7efbf8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800c8e0 con 0x7efc081428e0 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf800ca40 con 0x7efc081428e0 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0813ef70 con 0x7efc081428e0 2020-11-16T22:32:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800cdc0 con 0x7efc081428e0 2020-11-16T22:32:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.002+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc08140ef0 con 0x7efc081428e0 2020-11-16T22:32:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08069eb0 con 0x7efc081428e0 2020-11-16T22:32:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efbf8012600 con 0x7efc081428e0 2020-11-16T22:32:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efbeeffd700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 0x7efbe4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efbf806e150 con 0x7efc081428e0 2020-11-16T22:32:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efc067fc700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 0x7efbe4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.006+0000 7efc067fc700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 0x7efbe4057550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7efc081427f0 tx=0x7efbfc006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.014+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efbf8044240 con 0x7efc081428e0 2020-11-16T22:32:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc080d0170 con 0x7efc081428e0 2020-11-16T22:32:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efbeeffd700 1 -- 172.21.15.181:0/1246431535 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7efbf8047860 con 0x7efc081428e0 2020-11-16T22:32:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 msgr2=0x7efbe4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 0x7efbe4057550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7efc081427f0 tx=0x7efbfc006df0).stop 2020-11-16T22:32:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 msgr2=0x7efc08140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7efbf800ebd0 tx=0x7efbf8006df0).stop 2020-11-16T22:32:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 shutdown_connections 2020-11-16T22:32:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbe4055090 0x7efbe4057550 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc08163a40 0x7efc08159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc081428e0 0x7efc08140a00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 --2- 172.21.15.181:0/1246431535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc08162f20 0x7efc08159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 >> 172.21.15.181:0/1246431535 conn(0x7efc08060bc0 msgr2=0x7efc08160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:24.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 shutdown_connections 2020-11-16T22:32:24.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:23.214+0000 7efc0d464700 1 -- 172.21.15.181:0/1246431535 wait complete. 2020-11-16T22:32:24.011 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:24.011 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:24.011 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:24.214 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:24.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.350+0000 7f0cc1c06700 1 Processor -- start 2020-11-16T22:32:24.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- start start 2020-11-16T22:32:24.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0cbc061680 0x7f0cbc0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 0x7f0cbc15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cbc15cda0 con 0x7f0cbc0612a0 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc15d040 con 0x7f0cbc061680 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc15d2e0 con 0x7f0cbc15fa10 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 0x7f0cbc15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 0x7f0cbc15fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41396/0 (socket says 172.21.15.181:41396) 2020-11-16T22:32:24.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0cbc061680 0x7f0cbc0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbbfff700 1 -- 172.21.15.181:0/428266464 learned_addr learned my addr 172.21.15.181:0/428266464 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:24.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 -- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 msgr2=0x7f0cbc15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:24.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 --2- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 0x7f0cbc15fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 -- 172.21.15.181:0/428266464 >> v1:172.21.15.175:6789/0 conn(0x7f0cbc0612a0 legacy=0x7f0cbc0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 -- 172.21.15.181:0/428266464 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0cdd70 con 0x7f0cbc061680 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cbaffd700 1 --2- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc0cee60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0cac00b9a0 tx=0x7f0cac006df0).ready entity=mon.1 client_cookie=be413a38dfa8cfd2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cba7fc700 1 -- 172.21.15.181:0/428266464 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0097b0 con 0x7f0cbc061680 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cba7fc700 1 -- 172.21.15.181:0/428266464 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac009910 con 0x7f0cbc061680 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cba7fc700 1 -- 172.21.15.181:0/428266464 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac009c90 con 0x7f0cbc061680 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 msgr2=0x7f0cbc0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc0cee60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0cac00b9a0 tx=0x7f0cac006df0).stop 2020-11-16T22:32:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- 172.21.15.181:0/428266464 shutdown_connections 2020-11-16T22:32:24.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc15fa10 0x7f0cbc15fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/428266464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc0cee60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.354+0000 7f0cc1c06700 1 -- 172.21.15.181:0/428266464 >> 172.21.15.181:0/428266464 conn(0x7f0cbc0cf890 msgr2=0x7f0cbc0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- 172.21.15.181:0/428266464 shutdown_connections 2020-11-16T22:32:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- 172.21.15.181:0/428266464 wait complete. 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 Processor -- start 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- start start 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 0x7f0cbc064e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc16f820 con 0x7f0cbc15fa10 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc16fa80 con 0x7f0cbc061680 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cc1c06700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc16fd20 con 0x7f0cbc0612a0 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41398/0 (socket says 172.21.15.181:41398) 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 -- 172.21.15.181:0/3684222314 learned_addr learned my addr 172.21.15.181:0/3684222314 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbaffd700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbbfff700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 0x7f0cbc064e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 -- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 msgr2=0x7f0cbc142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 -- 172.21.15.181:0/3684222314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 msgr2=0x7f0cbc064e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 0x7f0cbc064e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbb7fe700 1 -- 172.21.15.181:0/3684222314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc16ffc0 con 0x7f0cbc0612a0 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.358+0000 7f0cbbfff700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 0x7f0cbc064e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cbb7fe700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f0cb000c6e0 tx=0x7f0cb0007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0007020 con 0x7f0cbc0612a0 2020-11-16T22:32:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0007180 con 0x7f0cbc0612a0 2020-11-16T22:32:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc170260 con 0x7f0cbc0612a0 2020-11-16T22:32:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000e560 con 0x7f0cbc0612a0 2020-11-16T22:32:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc157f50 con 0x7f0cbc0612a0 2020-11-16T22:32:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc158140 con 0x7f0cbc0612a0 2020-11-16T22:32:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0cb0010070 con 0x7f0cbc0612a0 2020-11-16T22:32:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 0x7f0c94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cbaffd700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 0x7f0c94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0cb006d0d0 con 0x7f0cbc0612a0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.362+0000 7f0cbaffd700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 0x7f0c94057520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f0cac006e30 tx=0x7f0cac015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.366+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0cb0043330 con 0x7f0cbc0612a0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.522+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cbc069eb0 con 0x7f0cbc0612a0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.522+0000 7f0c9b7fe700 1 -- 172.21.15.181:0/3684222314 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f0cb0024200 con 0x7f0cbc0612a0 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 msgr2=0x7f0c94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 0x7f0c94057520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f0cac006e30 tx=0x7f0cac015000).stop 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 msgr2=0x7f0cbc157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f0cb000c6e0 tx=0x7f0cb0007c50).stop 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 shutdown_connections 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c94055060 0x7f0c94057520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cbc15fa10 0x7f0cbc064e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cbc0612a0 0x7f0cbc157a60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 --2- 172.21.15.181:0/3684222314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cbc061680 0x7f0cbc142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:24.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 >> 172.21.15.181:0/3684222314 conn(0x7f0cbc0cf890 msgr2=0x7f0cbc15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:24.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 shutdown_connections 2020-11-16T22:32:24.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:24.526+0000 7f0cc1c06700 1 -- 172.21.15.181:0/3684222314 wait complete. 2020-11-16T22:32:24.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:24.765 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:54.520377+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:25.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:30:52.179372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:30:52.615461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:32:25.986 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:32:26.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.134+0000 7ff59d7d7700 1 Processor -- start 2020-11-16T22:32:26.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.134+0000 7ff59d7d7700 1 -- start start 2020-11-16T22:32:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff59815d140 0x7ff59815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980c85e0 con 0x7ff5980cf4a0 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59815ff70 con 0x7ff59815d140 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980c8840 con 0x7ff5980cf920 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41410/0 (socket says 172.21.15.181:41410) 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 -- 172.21.15.181:0/3936547938 learned_addr learned my addr 172.21.15.181:0/3936547938 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff597fff700 1 --2- 172.21.15.181:0/3936547938 >> v2:172.21.15.175:3300/0 conn(0x7ff59815d140 0x7ff59815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 -- 172.21.15.181:0/3936547938 >> v2:172.21.15.175:3300/0 conn(0x7ff59815d140 msgr2=0x7ff59815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 --2- 172.21.15.181:0/3936547938 >> v2:172.21.15.175:3300/0 conn(0x7ff59815d140 0x7ff59815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 -- 172.21.15.181:0/3936547938 >> v1:172.21.15.175:6789/0 conn(0x7ff5980cf4a0 legacy=0x7ff598064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 -- 172.21.15.181:0/3936547938 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980c8ae0 con 0x7ff5980cf920 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59cfd6700 1 --2- 172.21.15.181:0/3936547938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7ff590009d50 tx=0x7ff590009290).ready entity=mon.2 client_cookie=d5e4288a94b6036c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff596ffd700 1 -- 172.21.15.181:0/3936547938 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000bbc0 con 0x7ff5980cf920 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff596ffd700 1 -- 172.21.15.181:0/3936547938 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59000bd20 con 0x7ff5980cf920 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff596ffd700 1 -- 172.21.15.181:0/3936547938 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590010680 con 0x7ff5980cf920 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3936547938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 msgr2=0x7ff5980656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3936547938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7ff590009d50 tx=0x7ff590009290).stop 2020-11-16T22:32:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3936547938 shutdown_connections 2020-11-16T22:32:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3936547938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff5980656e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3936547938 >> v2:172.21.15.175:3300/0 conn(0x7ff59815d140 0x7ff59815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3936547938 >> 172.21.15.181:0/3936547938 conn(0x7ff5980d2dc0 msgr2=0x7ff5980ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3936547938 shutdown_connections 2020-11-16T22:32:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.138+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3936547938 wait complete. 2020-11-16T22:32:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.142+0000 7ff59d7d7700 1 Processor -- start 2020-11-16T22:32:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.142+0000 7ff59d7d7700 1 -- start start 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5980cf4a0 0x7ff598140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 0x7ff598144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59816b830 con 0x7ff59815d140 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598161880 con 0x7ff5980cf4a0 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff598140b20 con 0x7ff5980cf920 2020-11-16T22:32:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41414/0 (socket says 172.21.15.181:41414) 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff597fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5980cf4a0 0x7ff598140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 -- 172.21.15.181:0/3783582115 learned_addr learned my addr 172.21.15.181:0/3783582115 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59cfd6700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 0x7ff598144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 -- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5980cf4a0 msgr2=0x7ff598140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5980cf4a0 0x7ff598140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 -- 172.21.15.181:0/3783582115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 msgr2=0x7ff598144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 0x7ff598144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 -- 172.21.15.181:0/3783582115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff598140dc0 con 0x7ff5980cf920 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59cfd6700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 0x7ff598144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff5977fe700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7ff58c00c740 tx=0x7ff58c007bc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:26.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c007020 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c007180 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff598141060 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00e560 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.146+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5981431b0 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598140950 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff58c010070 con 0x7ff5980cf920 2020-11-16T22:32:26.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff577fff700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 0x7ff570057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:26.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff58c06d150 con 0x7ff5980cf920 2020-11-16T22:32:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff597fff700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 0x7ff570057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.150+0000 7ff597fff700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 0x7ff570057520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff588006fd0 tx=0x7ff588006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.158+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff58c0433b0 con 0x7ff5980cf920 2020-11-16T22:32:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff598044f60 con 0x7ff5980cf920 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff577fff700 1 -- 172.21.15.181:0/3783582115 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v64) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff58c0469d0 con 0x7ff5980cf920 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 msgr2=0x7ff570057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 0x7ff570057520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff588006fd0 tx=0x7ff588006df0).stop 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 msgr2=0x7ff59813dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7ff58c00c740 tx=0x7ff58c007bc0).stop 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 shutdown_connections 2020-11-16T22:32:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff570055060 0x7ff570057520 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff59815d140 0x7ff598144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5980cf4a0 0x7ff598140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 --2- 172.21.15.181:0/3783582115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5980cf920 0x7ff59813dc80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 >> 172.21.15.181:0/3783582115 conn(0x7ff5980d2dc0 msgr2=0x7ff5980d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 shutdown_connections 2020-11-16T22:32:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:26.314+0000 7ff59d7d7700 1 -- 172.21.15.181:0/3783582115 wait complete. 2020-11-16T22:32:27.151 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:32:27.151 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:32:27.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 Processor -- start 2020-11-16T22:32:27.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- start start 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe9a8161860 0x7fe9a8062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a80c8db0 con 0x7fe9a8161480 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a8165030 con 0x7fe9a8161860 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80c8330 con 0x7fe9a8062740 2020-11-16T22:32:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41424/0 (socket says 172.21.15.181:41424) 2020-11-16T22:32:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/3372042090 learned_addr learned my addr 172.21.15.181:0/3372042090 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9adbf8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe9a8161480 0x7fe9a8164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47882/0 (socket says 172.21.15.181:47882) 2020-11-16T22:32:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ad3f7700 1 --2- 172.21.15.181:0/3372042090 >> v2:172.21.15.175:3300/0 conn(0x7fe9a8161860 0x7fe9a8062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/3372042090 >> v2:172.21.15.175:3300/0 conn(0x7fe9a8161860 msgr2=0x7fe9a8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 --2- 172.21.15.181:0/3372042090 >> v2:172.21.15.175:3300/0 conn(0x7fe9a8161860 0x7fe9a8062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/3372042090 >> v1:172.21.15.181:6789/0 conn(0x7fe9a8161480 legacy=0x7fe9a8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/3372042090 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80c85d0 con 0x7fe9a8062740 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9ae3f9700 1 --2- 172.21.15.181:0/3372042090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fe9a0006fd0 tx=0x7fe9a0006eb0).ready entity=mon.2 client_cookie=8fe90be9da788ecd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9acbf6700 1 -- 172.21.15.181:0/3372042090 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0009810 con 0x7fe9a8062740 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9acbf6700 1 -- 172.21.15.181:0/3372042090 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a0009970 con 0x7fe9a8062740 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9acbf6700 1 -- 172.21.15.181:0/3372042090 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0009cf0 con 0x7fe9a8062740 2020-11-16T22:32:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- 172.21.15.181:0/3372042090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 msgr2=0x7fe9a80644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/3372042090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fe9a0006fd0 tx=0x7fe9a0006eb0).stop 2020-11-16T22:32:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- 172.21.15.181:0/3372042090 shutdown_connections 2020-11-16T22:32:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/3372042090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8062740 0x7fe9a80644e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/3372042090 >> v2:172.21.15.175:3300/0 conn(0x7fe9a8161860 0x7fe9a8062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- 172.21.15.181:0/3372042090 >> 172.21.15.181:0/3372042090 conn(0x7fe9a80d2dc0 msgr2=0x7fe9a80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:27.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- 172.21.15.181:0/3372042090 shutdown_connections 2020-11-16T22:32:27.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.306+0000 7fe9aebfa700 1 -- 172.21.15.181:0/3372042090 wait complete. 2020-11-16T22:32:27.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 Processor -- start 2020-11-16T22:32:27.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 -- start start 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a8062740 0x7fe9a8173a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 0x7fe9a8173f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8160540 con 0x7fe9a8062740 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80ccf10 con 0x7fe9a8161860 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.310+0000 7fe9aebfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80cd1b0 con 0x7fe9a8161480 2020-11-16T22:32:27.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:27.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ad3f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 0x7fe9a8173f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:27.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ad3f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 0x7fe9a8173f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41428/0 (socket says 172.21.15.181:41428) 2020-11-16T22:32:27.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ad3f7700 1 -- 172.21.15.181:0/1279445724 learned_addr learned my addr 172.21.15.181:0/1279445724 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:27.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33588/0 (socket says 172.21.15.181:33588) 2020-11-16T22:32:27.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9adbf8700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a8062740 0x7fe9a8173a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:27.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 msgr2=0x7fe9a8173f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:27.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 0x7fe9a8173f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/1279445724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a8062740 msgr2=0x7fe9a8173a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:27.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a8062740 0x7fe9a8173a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:27.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 -- 172.21.15.181:0/1279445724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80cd450 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9ae3f9700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fe9a0012000 tx=0x7fe9a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0009810 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a80cd6f0 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a80ccba0 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a0006b20 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a001d800 con 0x7fe9a8161860 2020-11-16T22:32:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a80cde00 con 0x7fe9a8161860 2020-11-16T22:32:27.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.314+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe9a0025030 con 0x7fe9a8161860 2020-11-16T22:32:27.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.318+0000 7fe999ffb700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 0x7fe980057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.318+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9a006f480 con 0x7fe9a8161860 2020-11-16T22:32:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.318+0000 7fe9adbf8700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 0x7fe980057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.318+0000 7fe9adbf8700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 0x7fe980057520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe9a8174700 tx=0x7fe9a4000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.322+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe9a0045790 con 0x7fe9a8161860 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.478+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9a8044f60 con 0x7fe9a8161860 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.478+0000 7fe999ffb700 1 -- 172.21.15.181:0/1279445724 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9a0023020 con 0x7fe9a8161860 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 msgr2=0x7fe980057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 0x7fe980057520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe9a8174700 tx=0x7fe9a4000b80).stop 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 msgr2=0x7fe9a80c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fe9a0012000 tx=0x7fe9a0015000).stop 2020-11-16T22:32:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 shutdown_connections 2020-11-16T22:32:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe980055060 0x7fe980057520 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a8161860 0x7fe9a80c7250 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a8062740 0x7fe9a8173a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 --2- 172.21.15.181:0/1279445724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9a8161480 0x7fe9a8173f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 >> 172.21.15.181:0/1279445724 conn(0x7fe9a80d2dc0 msgr2=0x7fe9a8149110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:28.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 shutdown_connections 2020-11-16T22:32:28.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:27.482+0000 7fe9aebfa700 1 -- 172.21.15.181:0/1279445724 wait complete. 2020-11-16T22:32:28.323 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:28.323 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:32:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.474+0000 7fbf46ec4700 1 Processor -- start 2020-11-16T22:32:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.474+0000 7fbf46ec4700 1 -- start start 2020-11-16T22:32:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf40145a30 0x7fbf40145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf401492c0 0x7fbf40080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf40061b50 con 0x7fbf401492c0 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf400618b0 con 0x7fbf40145a30 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf400cb930 con 0x7fbf400814c0 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf45ec2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf40145a30 0x7fbf40145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41438/0 (socket says 172.21.15.181:41438) 2020-11-16T22:32:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 -- 172.21.15.181:0/117669812 learned_addr learned my addr 172.21.15.181:0/117669812 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf456c1700 1 --2- 172.21.15.181:0/117669812 >> v2:172.21.15.175:3300/0 conn(0x7fbf401492c0 0x7fbf40080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 -- 172.21.15.181:0/117669812 >> v2:172.21.15.181:3300/0 conn(0x7fbf40145a30 msgr2=0x7fbf40145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:28.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 --2- 172.21.15.181:0/117669812 >> v2:172.21.15.181:3300/0 conn(0x7fbf40145a30 0x7fbf40145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 -- 172.21.15.181:0/117669812 >> v2:172.21.15.175:3300/0 conn(0x7fbf401492c0 msgr2=0x7fbf40080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 --2- 172.21.15.181:0/117669812 >> v2:172.21.15.175:3300/0 conn(0x7fbf401492c0 0x7fbf40080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 -- 172.21.15.181:0/117669812 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf400cbbd0 con 0x7fbf400814c0 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf466c3700 1 --2- 172.21.15.181:0/117669812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fbf400d43d0 tx=0x7fbf38009290).ready entity=mon.2 client_cookie=f8715dc6d3c5c180 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf44ec0700 1 -- 172.21.15.181:0/117669812 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800bbc0 con 0x7fbf400814c0 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf44ec0700 1 -- 172.21.15.181:0/117669812 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800bd20 con 0x7fbf400814c0 2020-11-16T22:32:28.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf44ec0700 1 -- 172.21.15.181:0/117669812 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38011610 con 0x7fbf400814c0 2020-11-16T22:32:28.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- 172.21.15.181:0/117669812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 msgr2=0x7fbf400cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:28.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/117669812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fbf400d43d0 tx=0x7fbf38009290).stop 2020-11-16T22:32:28.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- 172.21.15.181:0/117669812 shutdown_connections 2020-11-16T22:32:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/117669812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf400cc5b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/117669812 >> v2:172.21.15.181:3300/0 conn(0x7fbf40145a30 0x7fbf40145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/117669812 >> v2:172.21.15.175:3300/0 conn(0x7fbf401492c0 0x7fbf40080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- 172.21.15.181:0/117669812 >> 172.21.15.181:0/117669812 conn(0x7fbf400d68c0 msgr2=0x7fbf400d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- 172.21.15.181:0/117669812 shutdown_connections 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.478+0000 7fbf46ec4700 1 -- 172.21.15.181:0/117669812 wait complete. 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 Processor -- start 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- start start 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf40145a30 0x7fbf400d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf401492c0 0x7fbf400d2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4015ee40 con 0x7fbf40145a30 2020-11-16T22:32:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4015f0e0 con 0x7fbf401492c0 2020-11-16T22:32:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4015f380 con 0x7fbf400814c0 2020-11-16T22:32:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf466c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf401492c0 0x7fbf400d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41440/0 (socket says 172.21.15.181:41440) 2020-11-16T22:32:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 -- 172.21.15.181:0/3597587143 learned_addr learned my addr 172.21.15.181:0/3597587143 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf456c1700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf40145a30 0x7fbf400d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 -- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf401492c0 msgr2=0x7fbf400d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf401492c0 0x7fbf400d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 -- 172.21.15.181:0/3597587143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf40145a30 msgr2=0x7fbf400d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf40145a30 0x7fbf400d3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 -- 172.21.15.181:0/3597587143 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4015f620 con 0x7fbf400814c0 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf45ec2700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbf3c00eba0 tx=0x7fbf3c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00c8e0 con 0x7fbf400814c0 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c00ca40 con 0x7fbf400814c0 2020-11-16T22:32:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4015f8c0 con 0x7fbf400814c0 2020-11-16T22:32:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00cdc0 con 0x7fbf400814c0 2020-11-16T22:32:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.482+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf40160000 con 0x7fbf400814c0 2020-11-16T22:32:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf40044f60 con 0x7fbf400814c0 2020-11-16T22:32:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbf3c012600 con 0x7fbf400814c0 2020-11-16T22:32:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf2dffb700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 0x7fbf18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf3c06e1f0 con 0x7fbf400814c0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf456c1700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 0x7fbf18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.486+0000 7fbf456c1700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 0x7fbf18057520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fbf40081ca0 tx=0x7fbf30008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.494+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbf3c0442e0 con 0x7fbf400814c0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf400d6590 con 0x7fbf400814c0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf2dffb700 1 -- 172.21.15.181:0/3597587143 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbf3c047900 con 0x7fbf400814c0 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 msgr2=0x7fbf18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 0x7fbf18057520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fbf40081ca0 tx=0x7fbf30008040).stop 2020-11-16T22:32:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 msgr2=0x7fbf4015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbf3c00eba0 tx=0x7fbf3c006df0).stop 2020-11-16T22:32:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 shutdown_connections 2020-11-16T22:32:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf18055060 0x7fbf18057520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf401492c0 0x7fbf400d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf400814c0 0x7fbf4015e890 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 --2- 172.21.15.181:0/3597587143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf40145a30 0x7fbf400d3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 >> 172.21.15.181:0/3597587143 conn(0x7fbf400d68c0 msgr2=0x7fbf40145e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 shutdown_connections 2020-11-16T22:32:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:28.646+0000 7fbf46ec4700 1 -- 172.21.15.181:0/3597587143 wait complete. 2020-11-16T22:32:29.491 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:29.492 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:32:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 Processor -- start 2020-11-16T22:32:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 -- start start 2020-11-16T22:32:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff938161480 0x7ff938164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff938161860 0x7ff938062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380c8db0 con 0x7ff938164e10 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380c8330 con 0x7ff938161480 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff937fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380c85d0 con 0x7ff938161860 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff938161860 0x7ff938062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff938161860 0x7ff938062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33608/0 (socket says 172.21.15.181:33608) 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff92e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff938164e10 0x7ff938062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47910/0 (socket says 172.21.15.181:47910) 2020-11-16T22:32:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 -- 172.21.15.181:0/831434034 learned_addr learned my addr 172.21.15.181:0/831434034 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff936ffd700 1 --2- 172.21.15.181:0/831434034 >> v2:172.21.15.175:3300/0 conn(0x7ff938161480 0x7ff938164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 -- 172.21.15.181:0/831434034 >> v2:172.21.15.175:3300/0 conn(0x7ff938161480 msgr2=0x7ff938164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 --2- 172.21.15.181:0/831434034 >> v2:172.21.15.175:3300/0 conn(0x7ff938161480 0x7ff938164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 -- 172.21.15.181:0/831434034 >> v1:172.21.15.181:6789/0 conn(0x7ff938164e10 legacy=0x7ff938062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9367fc700 1 -- 172.21.15.181:0/831434034 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873175521 0 0) 0x7ff9380c8db0 con 0x7ff938164e10 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 -- 172.21.15.181:0/831434034 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380c8870 con 0x7ff938161860 2020-11-16T22:32:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9377fe700 1 --2- 172.21.15.181:0/831434034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161860 0x7ff938062960 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7ff9380c8260 tx=0x7ff930009290).ready entity=mon.1 client_cookie=235d4a8af61f620 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9367fc700 1 -- 172.21.15.181:0/831434034 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000bbc0 con 0x7ff938161860 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9367fc700 1 -- 172.21.15.181:0/831434034 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000bd20 con 0x7ff938161860 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.637+0000 7ff9367fc700 1 -- 172.21.15.181:0/831434034 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930011610 con 0x7ff938161860 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- 172.21.15.181:0/831434034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161860 msgr2=0x7ff938062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 --2- 172.21.15.181:0/831434034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161860 0x7ff938062960 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7ff9380c8260 tx=0x7ff930009290).stop 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- 172.21.15.181:0/831434034 shutdown_connections 2020-11-16T22:32:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 --2- 172.21.15.181:0/831434034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161860 0x7ff938062960 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 --2- 172.21.15.181:0/831434034 >> v2:172.21.15.175:3300/0 conn(0x7ff938161480 0x7ff938164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- 172.21.15.181:0/831434034 >> 172.21.15.181:0/831434034 conn(0x7ff9380d2dc0 msgr2=0x7ff9380ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- 172.21.15.181:0/831434034 shutdown_connections 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- 172.21.15.181:0/831434034 wait complete. 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 Processor -- start 2020-11-16T22:32:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 -- start start 2020-11-16T22:32:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.641+0000 7ff937fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938161860 0x7ff938130a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff93816b5d0 0x7ff9380c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938174990 con 0x7ff93816b5d0 2020-11-16T22:32:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938130f40 con 0x7ff938161480 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9381659c0 con 0x7ff938161860 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff92e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938161860 0x7ff938130a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33612/0 (socket says 172.21.15.181:33612) 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 -- 172.21.15.181:0/2838150953 learned_addr learned my addr 172.21.15.181:0/2838150953 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff9377fe700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff93816b5d0 0x7ff9380c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 -- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938161860 msgr2=0x7ff938130a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938161860 0x7ff938130a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 -- 172.21.15.181:0/2838150953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff93816b5d0 msgr2=0x7ff9380c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff93816b5d0 0x7ff9380c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 -- 172.21.15.181:0/2838150953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938165c60 con 0x7ff938161480 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff936ffd700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7ff92800eba0 tx=0x7ff928006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800c970 con 0x7ff938161480 2020-11-16T22:32:29.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92800cad0 con 0x7ff938161480 2020-11-16T22:32:29.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff938165f00 con 0x7ff938161480 2020-11-16T22:32:29.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800ce50 con 0x7ff938161480 2020-11-16T22:32:29.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff938166440 con 0x7ff938161480 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938174b00 con 0x7ff938161480 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.645+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff928012600 con 0x7ff938161480 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.649+0000 7ff92f7fe700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 0x7ff914057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.649+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff92806e1b0 con 0x7ff938161480 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.649+0000 7ff92e7fc700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 0x7ff914057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.649+0000 7ff92e7fc700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 0x7ff914057520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff938147c60 tx=0x7ff920007960).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.653+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff938174b00 con 0x7ff938161480 2020-11-16T22:32:29.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.809+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff938044f60 con 0x7ff938161480 2020-11-16T22:32:29.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff92f7fe700 1 -- 172.21.15.181:0/2838150953 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff9280443b0 con 0x7ff938161480 2020-11-16T22:32:29.802 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:29.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 msgr2=0x7ff914057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 0x7ff914057520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff938147c60 tx=0x7ff920007960).stop 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 msgr2=0x7ff938130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7ff92800eba0 tx=0x7ff928006df0).stop 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 shutdown_connections 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff914055060 0x7ff914057520 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff93816b5d0 0x7ff9380c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff938161480 0x7ff938130560 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 --2- 172.21.15.181:0/2838150953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938161860 0x7ff938130a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 >> 172.21.15.181:0/2838150953 conn(0x7ff9380d2dc0 msgr2=0x7ff9380ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:29.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 shutdown_connections 2020-11-16T22:32:29.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.813+0000 7ff937fff700 1 -- 172.21.15.181:0/2838150953 wait complete. 2020-11-16T22:32:29.824 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:29.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 Processor -- start 2020-11-16T22:32:29.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- start start 2020-11-16T22:32:29.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92280b3930 0x7f9228167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9228166bb0 0x7f9228167040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9228168390 con 0x7f9228167ac0 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280b5f00 con 0x7f9228166bb0 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280b2060 con 0x7f92280b3930 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922e507700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9228167ac0 0x7f92280b3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47924/0 (socket says 172.21.15.181:47924) 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922e507700 1 -- 172.21.15.181:0/1237618595 learned_addr learned my addr 172.21.15.181:0/1237618595 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 --2- 172.21.15.181:0/1237618595 >> v2:172.21.15.181:3300/0 conn(0x7f92280b3930 0x7f9228167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922ed08700 1 --2- 172.21.15.181:0/1237618595 >> v2:172.21.15.175:3300/0 conn(0x7f9228166bb0 0x7f9228167040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922d505700 1 -- 172.21.15.181:0/1237618595 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245158409 0 0) 0x7f9228168390 con 0x7f9228167ac0 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922d505700 1 -- 172.21.15.181:0/1237618595 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9210003240 con 0x7f9228167ac0 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 -- 172.21.15.181:0/1237618595 >> v2:172.21.15.175:3300/0 conn(0x7f9228166bb0 msgr2=0x7f9228167040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 --2- 172.21.15.181:0/1237618595 >> v2:172.21.15.175:3300/0 conn(0x7f9228166bb0 0x7f9228167040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 -- 172.21.15.181:0/1237618595 >> v1:172.21.15.181:6789/0 conn(0x7f9228167ac0 legacy=0x7f92280b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 -- 172.21.15.181:0/1237618595 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280b2300 con 0x7f92280b3930 2020-11-16T22:32:29.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922dd06700 1 --2- 172.21.15.181:0/1237618595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f9228167ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f92280660e0 tx=0x7f921c006df0).ready entity=mon.1 client_cookie=cc41ba332d4b4dbf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922d505700 1 -- 172.21.15.181:0/1237618595 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c009640 con 0x7f92280b3930 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922d505700 1 -- 172.21.15.181:0/1237618595 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0097a0 con 0x7f92280b3930 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f922d505700 1 -- 172.21.15.181:0/1237618595 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c009b20 con 0x7f92280b3930 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- 172.21.15.181:0/1237618595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 msgr2=0x7f9228167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 --2- 172.21.15.181:0/1237618595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f9228167ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f92280660e0 tx=0x7f921c006df0).stop 2020-11-16T22:32:29.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- 172.21.15.181:0/1237618595 shutdown_connections 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 --2- 172.21.15.181:0/1237618595 >> v2:172.21.15.175:3300/0 conn(0x7f9228166bb0 0x7f9228167040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 --2- 172.21.15.181:0/1237618595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f9228167ea0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.957+0000 7f923076b700 1 -- 172.21.15.181:0/1237618595 >> 172.21.15.181:0/1237618595 conn(0x7f9228060bc0 msgr2=0x7f92280d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- 172.21.15.181:0/1237618595 shutdown_connections 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- 172.21.15.181:0/1237618595 wait complete. 2020-11-16T22:32:29.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 Processor -- start 2020-11-16T22:32:29.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- start start 2020-11-16T22:32:29.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f92280d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228167ac0 0x7f92280b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228174710 con 0x7f9228167ac0 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92281741d0 con 0x7f92280b3930 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f923076b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9228174470 con 0x7f9228166bb0 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922e507700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f92280d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41470/0 (socket says 172.21.15.181:41470) 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 -- 172.21.15.181:0/921466855 learned_addr learned my addr 172.21.15.181:0/921466855 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922ed08700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228167ac0 0x7f92280b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 -- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 msgr2=0x7f92280d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f92280d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 -- 172.21.15.181:0/921466855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228167ac0 msgr2=0x7f92280b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228167ac0 0x7f92280b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:29.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.961+0000 7f922dd06700 1 -- 172.21.15.181:0/921466855 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280d12b0 con 0x7f9228166bb0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f922dd06700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f921c006e30 tx=0x7f921c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c011c40 con 0x7f9228166bb0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c011da0 con 0x7f9228166bb0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92280d1550 con 0x7f9228166bb0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c01e670 con 0x7f9228166bb0 2020-11-16T22:32:29.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9228065720 con 0x7f9228166bb0 2020-11-16T22:32:29.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f921c025030 con 0x7f9228166bb0 2020-11-16T22:32:29.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922805d2e0 con 0x7f9228166bb0 2020-11-16T22:32:29.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:29.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f922e507700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:29.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f921c06f280 con 0x7f9228166bb0 2020-11-16T22:32:29.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.965+0000 7f922e507700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f92280b1850 tx=0x7f9224009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:29.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:29.973+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f922805d2e0 con 0x7f9228166bb0 2020-11-16T22:32:30.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.173+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f922805d2e0 con 0x7f9228166bb0 2020-11-16T22:32:30.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f921a7fc700 1 -- 172.21.15.181:0/921466855 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f922805d2e0 con 0x7f9228166bb0 2020-11-16T22:32:30.167 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:30.167 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":73,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6165,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3995893893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3995893893},{"type":"v1","addr":"172.21.15.175:6833","nonce":3995893893}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6180,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1478076425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1478076425},{"type":"v1","addr":"172.21.15.175:6835","nonce":1478076425}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6189,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1833563248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1833563248},{"type":"v1","addr":"172.21.15.175:6837","nonce":1833563248}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6198,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3415483657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3415483657},{"type":"v1","addr":"172.21.15.175:6839","nonce":3415483657}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6840/471279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":471279035},{"type":"v1","addr":"172.21.15.181:6840","nonce":471279035}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59},{"gid":6280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3119822922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3119822922},{"type":"v1","addr":"172.21.15.181:6835","nonce":3119822922}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":59}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:30:53.516445+0000","modified":"2020-11-16T22:31:57.741211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6316,"mds_1":6313,"mds_2":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"d","rank":2,"incarnation":66,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/461666765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":461666765},{"type":"v1","addr":"172.21.15.181:6837","nonce":461666765}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6313":{"gid":6313,"name":"i","rank":1,"incarnation":64,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6841/2108826019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2108826019},{"type":"v1","addr":"172.21.15.175:6841","nonce":2108826019}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6316":{"gid":6316,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6841/3625693713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6839","nonce":3625693713},{"type":"v1","addr":"172.21.15.181:6841","nonce":3625693713}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:32:30.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 msgr2=0x7f9200057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f92280b1850 tx=0x7f9224009040).stop 2020-11-16T22:32:30.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 msgr2=0x7f9228065230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f921c006e30 tx=0x7f921c015000).stop 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 shutdown_connections 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228167ac0 0x7f92280b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280b3930 0x7f92280d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 --2- 172.21.15.181:0/921466855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228166bb0 0x7f9228065230 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 >> 172.21.15.181:0/921466855 conn(0x7f9228060bc0 msgr2=0x7f92280d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:30.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 shutdown_connections 2020-11-16T22:32:30.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.177+0000 7f923076b700 1 -- 172.21.15.181:0/921466855 wait complete. 2020-11-16T22:32:30.171 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 73 2020-11-16T22:32:30.187 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:30.265 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:30.272 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:30.274 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:32:30.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 Processor -- start 2020-11-16T22:32:30.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 -- start start 2020-11-16T22:32:30.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb8815e270 0x7fbb8815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb88063b80 0x7fbb880667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb88064050 con 0x7fbb880b6c20 2020-11-16T22:32:30.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb880642f0 con 0x7fbb8815e270 2020-11-16T22:32:30.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb8e57f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb88064590 con 0x7fbb88063b80 2020-11-16T22:32:30.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb88063b80 0x7fbb880667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb88063b80 0x7fbb880667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33636/0 (socket says 172.21.15.181:33636) 2020-11-16T22:32:30.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 -- 172.21.15.181:0/743041558 learned_addr learned my addr 172.21.15.181:0/743041558 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:30.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb87fff700 1 --2- 172.21.15.181:0/743041558 >> v2:172.21.15.175:3300/0 conn(0x7fbb8815e270 0x7fbb8815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 -- 172.21.15.181:0/743041558 >> v2:172.21.15.175:3300/0 conn(0x7fbb8815e270 msgr2=0x7fbb8815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:30.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 --2- 172.21.15.181:0/743041558 >> v2:172.21.15.175:3300/0 conn(0x7fbb8815e270 0x7fbb8815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 -- 172.21.15.181:0/743041558 >> v1:172.21.15.175:6789/0 conn(0x7fbb880b6c20 legacy=0x7fbb88066ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:30.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 -- 172.21.15.181:0/743041558 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88064830 con 0x7fbb88063b80 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb877fe700 1 --2- 172.21.15.181:0/743041558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb880667b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fbb880ba980 tx=0x7fbb74006eb0).ready entity=mon.1 client_cookie=88a4961738637f5a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb86ffd700 1 -- 172.21.15.181:0/743041558 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74009890 con 0x7fbb88063b80 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb86ffd700 1 -- 172.21.15.181:0/743041558 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740099f0 con 0x7fbb88063b80 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.317+0000 7fbb86ffd700 1 -- 172.21.15.181:0/743041558 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74009d70 con 0x7fbb88063b80 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- 172.21.15.181:0/743041558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 msgr2=0x7fbb880667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/743041558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb880667b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fbb880ba980 tx=0x7fbb74006eb0).stop 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- 172.21.15.181:0/743041558 shutdown_connections 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/743041558 >> v2:172.21.15.175:3300/0 conn(0x7fbb8815e270 0x7fbb8815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/743041558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb880667b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- 172.21.15.181:0/743041558 >> 172.21.15.181:0/743041558 conn(0x7fbb880d4790 msgr2=0x7fbb880d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- 172.21.15.181:0/743041558 shutdown_connections 2020-11-16T22:32:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- 172.21.15.181:0/743041558 wait complete. 2020-11-16T22:32:30.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 Processor -- start 2020-11-16T22:32:30.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- start start 2020-11-16T22:32:30.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb880b6c20 0x7fbb8814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 0x7fbb8814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815bf20 con 0x7fbb880b6c20 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815c1c0 con 0x7fbb88063b80 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8e57f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815c460 con 0x7fbb8815e270 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8cb1c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 0x7fbb8814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33640/0 (socket says 172.21.15.181:33640) 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 -- 172.21.15.181:0/699389951 learned_addr learned my addr 172.21.15.181:0/699389951 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb877fe700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb880b6c20 0x7fbb8814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 -- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 msgr2=0x7fbb8814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 0x7fbb8814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 -- 172.21.15.181:0/699389951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb880b6c20 msgr2=0x7fbb8814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb880b6c20 0x7fbb8814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb87fff700 1 -- 172.21.15.181:0/699389951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8815c700 con 0x7fbb88063b80 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.321+0000 7fbb8cb1c700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 0x7fbb8814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb87fff700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fbb8000ecc0 tx=0x7fbb80008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000c7d0 con 0x7fbb88063b80 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000c930 con 0x7fbb88063b80 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000ccb0 con 0x7fbb88063b80 2020-11-16T22:32:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8815c9a0 con 0x7fbb88063b80 2020-11-16T22:32:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8815bbc0 con 0x7fbb88063b80 2020-11-16T22:32:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8815f050 con 0x7fbb88063b80 2020-11-16T22:32:30.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb80013640 con 0x7fbb88063b80 2020-11-16T22:32:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 0x7fbb5c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb8006ddc0 con 0x7fbb88063b80 2020-11-16T22:32:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb877fe700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 0x7fbb5c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.325+0000 7fbb877fe700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 0x7fbb5c057520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fbb88063f80 tx=0x7fbb74015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.329+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbb80043fc0 con 0x7fbb88063b80 2020-11-16T22:32:30.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb88069eb0 con 0x7fbb88063b80 2020-11-16T22:32:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb7ffff700 1 -- 172.21.15.181:0/699389951 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fbb800475e0 con 0x7fbb88063b80 2020-11-16T22:32:30.468 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:30.468 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:30:54.520377+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:30:52.179372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:30:52.615461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:30.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 msgr2=0x7fbb5c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 0x7fbb5c057520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fbb88063f80 tx=0x7fbb74015000).stop 2020-11-16T22:32:30.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 msgr2=0x7fbb8815b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fbb8000ecc0 tx=0x7fbb80008040).stop 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 shutdown_connections 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb5c055060 0x7fbb5c057520 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb8815e270 0x7fbb8814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88063b80 0x7fbb8815b6d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 --2- 172.21.15.181:0/699389951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb880b6c20 0x7fbb8814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.477+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 >> 172.21.15.181:0/699389951 conn(0x7fbb880d4790 msgr2=0x7fbb881529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:30.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.481+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 shutdown_connections 2020-11-16T22:32:30.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.481+0000 7fbb8e57f700 1 -- 172.21.15.181:0/699389951 wait complete. 2020-11-16T22:32:30.483 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:32:30.483 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:32:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 Processor -- start 2020-11-16T22:32:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 -- start start 2020-11-16T22:32:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78e80615f0 0x7f78e815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80cccd0 con 0x7f78e815f1a0 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e80ccf70 con 0x7f78e80615f0 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.621+0000 7f78edeb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815e030 con 0x7f78e8156a90 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41490/0 (socket says 172.21.15.181:41490) 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 -- 172.21.15.181:0/370063409 learned_addr learned my addr 172.21.15.181:0/370063409 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:30.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78e7fff700 1 --2- 172.21.15.181:0/370063409 >> v2:172.21.15.175:3300/0 conn(0x7f78e80615f0 0x7f78e815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 -- 172.21.15.181:0/370063409 >> v2:172.21.15.175:3300/0 conn(0x7f78e80615f0 msgr2=0x7f78e815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:30.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 --2- 172.21.15.181:0/370063409 >> v2:172.21.15.175:3300/0 conn(0x7f78e80615f0 0x7f78e815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 -- 172.21.15.181:0/370063409 >> v1:172.21.15.175:6789/0 conn(0x7f78e815f1a0 legacy=0x7f78e8160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 -- 172.21.15.181:0/370063409 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e815e2d0 con 0x7f78e8156a90 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78eceb4700 1 --2- 172.21.15.181:0/370063409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f78d8006fd0 tx=0x7f78d8006eb0).ready entity=mon.2 client_cookie=c5d78c3f7d166448 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78e77fe700 1 -- 172.21.15.181:0/370063409 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8009810 con 0x7f78e8156a90 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78e77fe700 1 -- 172.21.15.181:0/370063409 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8009970 con 0x7f78e8156a90 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78e77fe700 1 -- 172.21.15.181:0/370063409 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8009cf0 con 0x7f78e8156a90 2020-11-16T22:32:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- 172.21.15.181:0/370063409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 msgr2=0x7f78e8061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 --2- 172.21.15.181:0/370063409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f78d8006fd0 tx=0x7f78d8006eb0).stop 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- 172.21.15.181:0/370063409 shutdown_connections 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 --2- 172.21.15.181:0/370063409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e8061100 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 --2- 172.21.15.181:0/370063409 >> v2:172.21.15.175:3300/0 conn(0x7f78e80615f0 0x7f78e815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- 172.21.15.181:0/370063409 >> 172.21.15.181:0/370063409 conn(0x7f78e80cf630 msgr2=0x7f78e80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- 172.21.15.181:0/370063409 shutdown_connections 2020-11-16T22:32:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- 172.21.15.181:0/370063409 wait complete. 2020-11-16T22:32:30.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 Processor -- start 2020-11-16T22:32:30.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 -- start start 2020-11-16T22:32:30.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.625+0000 7f78edeb6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e80615f0 0x7f78e80b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 0x7f78e80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80b63e0 con 0x7f78e80615f0 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8142720 con 0x7f78e815f1a0 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e816f920 con 0x7f78e8156a90 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41498/0 (socket says 172.21.15.181:41498) 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 -- 172.21.15.181:0/1887265999 learned_addr learned my addr 172.21.15.181:0/1887265999 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78ed6b5700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 0x7f78e80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78eceb4700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e80615f0 0x7f78e80b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 -- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 msgr2=0x7f78e80d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 0x7f78e80d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 -- 172.21.15.181:0/1887265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e80615f0 msgr2=0x7f78e80b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e80615f0 0x7f78e80b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 -- 172.21.15.181:0/1887265999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e816fbc0 con 0x7f78e8156a90 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78ed6b5700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 0x7f78e80d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e7fff700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f78d000ebd0 tx=0x7f78d0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000c8e0 con 0x7f78e8156a90 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000ca40 con 0x7f78e8156a90 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000cdc0 con 0x7f78e8156a90 2020-11-16T22:32:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e816fe60 con 0x7f78e8156a90 2020-11-16T22:32:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8156e70 con 0x7f78e8156a90 2020-11-16T22:32:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8044f60 con 0x7f78e8156a90 2020-11-16T22:32:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f78d001a030 con 0x7f78e8156a90 2020-11-16T22:32:30.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 0x7f78c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:30.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78d006d9e0 con 0x7f78e8156a90 2020-11-16T22:32:30.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78eceb4700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 0x7f78c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:30.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.629+0000 7f78eceb4700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 0x7f78c4057520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f78d800b1b0 tx=0x7f78d8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:30.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.637+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f78d0043cf0 con 0x7f78e8156a90 2020-11-16T22:32:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.837+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78e80cc6e0 con 0x7f78e8156a90 2020-11-16T22:32:30.883 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:32:30.897+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:32:30.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.909+0000 7f78e4ff9700 1 -- 172.21.15.181:0/1887265999 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f78d0047310 con 0x7f78e8156a90 2020-11-16T22:32:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 msgr2=0x7f78c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 0x7f78c4057520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f78d800b1b0 tx=0x7f78d8015000).stop 2020-11-16T22:32:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 msgr2=0x7f78e80cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f78d000ebd0 tx=0x7f78d0006df0).stop 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 shutdown_connections 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78c4055060 0x7f78c4057520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e815f1a0 0x7f78e80d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78e80615f0 0x7f78e80b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 --2- 172.21.15.181:0/1887265999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e8156a90 0x7f78e80cc1f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 >> 172.21.15.181:0/1887265999 conn(0x7f78e80cf630 msgr2=0x7f78e80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:30.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 shutdown_connections 2020-11-16T22:32:30.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:30.913+0000 7f78edeb6700 1 -- 172.21.15.181:0/1887265999 wait complete. 2020-11-16T22:32:30.906 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:32:30.908 INFO:tasks.ceph.mds.d.smithi181.stderr: -1201> 2020-11-16T22:31:50.489+0000 7fd27412f700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@4},l=6327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x559d77a39700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.908 INFO:tasks.ceph.mds.d.smithi181.stderr: -1179> 2020-11-16T22:31:50.489+0000 7fd27412f700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/AsLsXsFsx@2},l=6327 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x559d77aea000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.912 INFO:tasks.ceph.mds.i.smithi175.stderr: -1343> 2020-11-16T22:31:50.489+0000 7fd9433d0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v2 rc2020-11-16T22:31:14.259728+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@4},l=6327 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x562e1e55ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:31:14.259728+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.913 INFO:tasks.ceph.mds.i.smithi175.stderr: -1224> 2020-11-16T22:31:50.493+0000 7fd9493dc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1) n(v1 rc2020-11-16T22:31:14.259728+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6327=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x562e1e55f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:11.691728+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.691728+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:31:14.259728+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.913 INFO:tasks.ceph.mds.a.smithi181.stderr: -2176> 2020-11-16T22:31:50.489+0000 7f71957c3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1) n(v4 rc2020-11-16T22:31:14.259728+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6327=pAsLsXsFsx/pAsLsXsFsx@5},l=6327 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5650ff773b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v29>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:31:14.259728+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.914 INFO:tasks.ceph.mds.a.smithi181.stderr: -2078> 2020-11-16T22:31:50.493+0000 7f719b7cf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1) n(v3 rc2020-11-16T22:31:14.259728+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6327=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5650ff774100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v29>,<0x1/d1 v26>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:31:14.259728+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:31:11.687728+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:31:14.259728+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:32:30.922 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:32:30.929 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:32:30.933 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:32:30.937 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:32:31.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.053+0000 7f8c4892a700 1 Processor -- start 2020-11-16T22:32:31.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.053+0000 7f8c4892a700 1 -- start start 2020-11-16T22:32:31.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c440612a0 0x7f8c44061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:31.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c44061b70 con 0x7f8c4415fa10 2020-11-16T22:32:31.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4415cda0 con 0x7f8c440ce750 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4415d040 con 0x7f8c440612a0 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c440612a0 0x7f8c44061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c440612a0 0x7f8c44061680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33686/0 (socket says 172.21.15.181:33686) 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 -- 172.21.15.181:0/2593934585 learned_addr learned my addr 172.21.15.181:0/2593934585 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 -- 172.21.15.181:0/2593934585 >> v1:172.21.15.181:6789/0 conn(0x7f8c440ce750 legacy=0x7f8c440cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 -- 172.21.15.181:0/2593934585 >> v1:172.21.15.175:6789/0 conn(0x7f8c4415fa10 legacy=0x7f8c4415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 -- 172.21.15.181:0/2593934585 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4415d2e0 con 0x7f8c440612a0 2020-11-16T22:32:31.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4259c700 1 --2- 172.21.15.181:0/2593934585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c440612a0 0x7f8c44061680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f8c34009d50 tx=0x7f8c34009290).ready entity=mon.1 client_cookie=253245d27a438fcd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4159a700 1 -- 172.21.15.181:0/2593934585 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400bc40 con 0x7f8c440612a0 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4159a700 1 -- 172.21.15.181:0/2593934585 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3400bda0 con 0x7f8c440612a0 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4159a700 1 -- 172.21.15.181:0/2593934585 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34011610 con 0x7f8c440612a0 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- 172.21.15.181:0/2593934585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c440612a0 msgr2=0x7f8c44061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 --2- 172.21.15.181:0/2593934585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c440612a0 0x7f8c44061680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f8c34009d50 tx=0x7f8c34009290).stop 2020-11-16T22:32:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- 172.21.15.181:0/2593934585 shutdown_connections 2020-11-16T22:32:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 --2- 172.21.15.181:0/2593934585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c440612a0 0x7f8c44061680 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- 172.21.15.181:0/2593934585 >> 172.21.15.181:0/2593934585 conn(0x7f8c440cf890 msgr2=0x7f8c440cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- 172.21.15.181:0/2593934585 shutdown_connections 2020-11-16T22:32:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.057+0000 7f8c4892a700 1 -- 172.21.15.181:0/2593934585 wait complete. 2020-11-16T22:32:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.061+0000 7f8c4892a700 1 Processor -- start 2020-11-16T22:32:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.061+0000 7f8c4892a700 1 -- start start 2020-11-16T22:32:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c440612a0 0x7f8c440ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c440ce750 0x7f8c440cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440c9a90 con 0x7f8c440612a0 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c441375a0 con 0x7f8c4415fa10 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c44137060 con 0x7f8c440ce750 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c41d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c440ce750 0x7f8c440cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33696/0 (socket says 172.21.15.181:33696) 2020-11-16T22:32:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 -- 172.21.15.181:0/3767845198 learned_addr learned my addr 172.21.15.181:0/3767845198 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4259c700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c440612a0 0x7f8c440ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 -- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c440ce750 msgr2=0x7f8c440cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c440ce750 0x7f8c440cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 -- 172.21.15.181:0/3767845198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c440612a0 msgr2=0x7f8c440ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c440612a0 0x7f8c440ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 -- 172.21.15.181:0/3767845198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44137300 con 0x7f8c4415fa10 2020-11-16T22:32:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c42d9d700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f8c3c00c710 tx=0x7f8c3c007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e490 con 0x7f8c4415fa10 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00e5f0 con 0x7f8c4415fa10 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e9b0 con 0x7f8c4415fa10 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c44137c00 con 0x7f8c4415fa10 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44138140 con 0x7f8c4415fa10 2020-11-16T22:32:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c44138300 con 0x7f8c4415fa10 2020-11-16T22:32:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8c3c010030 con 0x7f8c4415fa10 2020-11-16T22:32:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.065+0000 7f8c327fc700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 0x7f8c18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.069+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f8c3c06dc40 con 0x7f8c4415fa10 2020-11-16T22:32:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.069+0000 7f8c4259c700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 0x7f8c18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.069+0000 7f8c4259c700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 0x7f8c18057520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f8c340092d0 tx=0x7f8c34016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:31.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.073+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8c3c043d40 con 0x7f8c4415fa10 2020-11-16T22:32:31.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.277+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c44069eb0 con 0x7f8c4415fa10 2020-11-16T22:32:31.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c327fc700 1 -- 172.21.15.181:0/3767845198 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c3c047360 con 0x7f8c4415fa10 2020-11-16T22:32:31.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 msgr2=0x7f8c18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 0x7f8c18057520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f8c340092d0 tx=0x7f8c34016000).stop 2020-11-16T22:32:31.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 msgr2=0x7f8c441385e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:31.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f8c3c00c710 tx=0x7f8c3c007b70).stop 2020-11-16T22:32:31.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 shutdown_connections 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8c18055060 0x7f8c18057520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4415fa10 0x7f8c441385e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c440612a0 0x7f8c440ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 --2- 172.21.15.181:0/3767845198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c440ce750 0x7f8c440cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.921+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 >> 172.21.15.181:0/3767845198 conn(0x7f8c440cf890 msgr2=0x7f8c4415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.925+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 shutdown_connections 2020-11-16T22:32:31.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:31.925+0000 7f8c4892a700 1 -- 172.21.15.181:0/3767845198 wait complete. 2020-11-16T22:32:31.929 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 Processor -- start 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 -- start start 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9bd0145e40 0x7f9bd0080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd00cb950 con 0x7f9bd0145a60 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd00cbbf0 con 0x7f9bd00814c0 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd7ff6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd00cbe90 con 0x7f9bd0145e40 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd6593700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9bd00814c0 0x7f9bd00d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48012/0 (socket says 172.21.15.181:48012) 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd6593700 1 -- 172.21.15.181:0/2752889385 learned_addr learned my addr 172.21.15.181:0/2752889385 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd5591700 1 --2- 172.21.15.181:0/2752889385 >> v2:172.21.15.175:3300/0 conn(0x7f9bd0145e40 0x7f9bd0080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd4d90700 1 -- 172.21.15.181:0/2752889385 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059804671 0 0) 0x7f9bd00cbbf0 con 0x7f9bd00814c0 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.117+0000 7f9bd4d90700 1 -- 172.21.15.181:0/2752889385 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb8003240 con 0x7f9bd00814c0 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd5591700 1 -- 172.21.15.181:0/2752889385 >> v1:172.21.15.181:6789/0 conn(0x7f9bd00814c0 legacy=0x7f9bd00d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd5591700 1 -- 172.21.15.181:0/2752889385 >> v1:172.21.15.175:6789/0 conn(0x7f9bd0145a60 legacy=0x7f9bd00615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd5591700 1 -- 172.21.15.181:0/2752889385 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00cc130 con 0x7f9bd0145e40 2020-11-16T22:32:32.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd5591700 1 --2- 172.21.15.181:0/2752889385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145e40 0x7f9bd0080fd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f9bc000b9a0 tx=0x7f9bc0006df0).ready entity=mon.0 client_cookie=64af6723dc7caa48 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:32.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd4d90700 1 -- 172.21.15.181:0/2752889385 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00097b0 con 0x7f9bd0145e40 2020-11-16T22:32:32.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd4d90700 1 -- 172.21.15.181:0/2752889385 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0009910 con 0x7f9bd0145e40 2020-11-16T22:32:32.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd4d90700 1 -- 172.21.15.181:0/2752889385 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0009c90 con 0x7f9bd0145e40 2020-11-16T22:32:32.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/2752889385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145e40 msgr2=0x7f9bd0080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/2752889385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145e40 0x7f9bd0080fd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f9bc000b9a0 tx=0x7f9bc0006df0).stop 2020-11-16T22:32:32.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/2752889385 shutdown_connections 2020-11-16T22:32:32.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/2752889385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145e40 0x7f9bd0080fd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/2752889385 >> 172.21.15.181:0/2752889385 conn(0x7f9bd00d68e0 msgr2=0x7f9bd00d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:32.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/2752889385 shutdown_connections 2020-11-16T22:32:32.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.121+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/2752889385 wait complete. 2020-11-16T22:32:32.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 Processor -- start 2020-11-16T22:32:32.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- start start 2020-11-16T22:32:32.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145a60 0x7f9bd0170510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bd0145e40 0x7f9bd00bc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00cefd0 con 0x7f9bd0145a60 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00cdc30 con 0x7f9bd00814c0 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0141c10 con 0x7f9bd0145e40 2020-11-16T22:32:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33710/0 (socket says 172.21.15.181:33710) 2020-11-16T22:32:32.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 -- 172.21.15.181:0/678583450 learned_addr learned my addr 172.21.15.181:0/678583450 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:32.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd6593700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bd0145e40 0x7f9bd00bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:32.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5591700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145a60 0x7f9bd0170510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:32.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 -- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bd0145e40 msgr2=0x7f9bd00bc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bd0145e40 0x7f9bd00bc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 -- 172.21.15.181:0/678583450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145a60 msgr2=0x7f9bd0170510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145a60 0x7f9bd0170510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 -- 172.21.15.181:0/678583450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00cde60 con 0x7f9bd00814c0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd5d92700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f9bcc00c6e0 tx=0x7f9bcc007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc007020 con 0x7f9bd00814c0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc007180 con 0x7f9bd00814c0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd00ce100 con 0x7f9bd00814c0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0064f0 con 0x7f9bd00814c0 2020-11-16T22:32:32.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.125+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd0170a00 con 0x7f9bd00814c0 2020-11-16T22:32:32.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0069eb0 con 0x7f9bd00814c0 2020-11-16T22:32:32.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9bcc010070 con 0x7f9bd00814c0 2020-11-16T22:32:32.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bbdffb700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 0x7f9ba8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:32.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f9bcc06d100 con 0x7f9bd00814c0 2020-11-16T22:32:32.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bd5591700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 0x7f9ba8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:32.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.129+0000 7f9bd5591700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 0x7f9ba8057520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f9bc000be60 tx=0x7f9bc0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.137+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9bcc043260 con 0x7f9bd00814c0 2020-11-16T22:32:32.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.289+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9bd005d590 con 0x7f9bd00814c0 2020-11-16T22:32:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.929+0000 7f9bbdffb700 1 -- 172.21.15.181:0/678583450 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9bcc046880 con 0x7f9bd00814c0 2020-11-16T22:32:32.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 msgr2=0x7f9ba8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 0x7f9ba8057520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f9bc000be60 tx=0x7f9bc0015000).stop 2020-11-16T22:32:32.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 msgr2=0x7f9bd00ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:32.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f9bcc00c6e0 tx=0x7f9bcc007c50).stop 2020-11-16T22:32:32.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 shutdown_connections 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ba8055060 0x7f9ba8057520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bd0145e40 0x7f9bd00bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bd00814c0 0x7f9bd00ceae0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 --2- 172.21.15.181:0/678583450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bd0145a60 0x7f9bd0170510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.933+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 >> 172.21.15.181:0/678583450 conn(0x7f9bd00d68e0 msgr2=0x7f9bd01422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:32.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.937+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 shutdown_connections 2020-11-16T22:32:32.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:32.937+0000 7f9bd7ff6700 1 -- 172.21.15.181:0/678583450 wait complete. 2020-11-16T22:32:32.928 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:32:33.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 Processor -- start 2020-11-16T22:32:33.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 -- start start 2020-11-16T22:32:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8098157e60 0x7f8098060670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980c82f0 con 0x7f80980c91a0 2020-11-16T22:32:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8098060b60 con 0x7f8098060290 2020-11-16T22:32:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809d530700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80980ce440 con 0x7f8098157e60 2020-11-16T22:32:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f8097fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8098060290 0x7f80980cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48026/0 (socket says 172.21.15.181:48026) 2020-11-16T22:32:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f8097fff700 1 -- 172.21.15.181:0/4293029190 learned_addr learned my addr 172.21.15.181:0/4293029190 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.073+0000 7f809cd2f700 1 --2- 172.21.15.181:0/4293029190 >> v2:172.21.15.181:3300/0 conn(0x7f8098157e60 0x7f8098060670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050962221 0 0) 0x7f8098060b60 con 0x7f8098060290 2020-11-16T22:32:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8078003240 con 0x7f8098060290 2020-11-16T22:32:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433293940 0 0) 0x7f80980c82f0 con 0x7f80980c91a0 2020-11-16T22:32:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8098060b60 con 0x7f80980c91a0 2020-11-16T22:32:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809cd2f700 1 -- 172.21.15.181:0/4293029190 >> v1:172.21.15.181:6789/0 conn(0x7f8098060290 legacy=0x7f80980cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809cd2f700 1 -- 172.21.15.181:0/4293029190 >> v1:172.21.15.175:6789/0 conn(0x7f80980c91a0 legacy=0x7f8098157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809cd2f700 1 -- 172.21.15.181:0/4293029190 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809815ea70 con 0x7f8098157e60 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809cd2f700 1 --2- 172.21.15.181:0/4293029190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098060670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f8090009c90 tx=0x7f80900091d0).ready entity=mon.1 client_cookie=b66698cb39db159d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000b9c0 con 0x7f8098157e60 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000bb20 con 0x7f8098157e60 2020-11-16T22:32:33.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f80977fe700 1 -- 172.21.15.181:0/4293029190 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000bea0 con 0x7f8098157e60 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 -- 172.21.15.181:0/4293029190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 msgr2=0x7f8098060670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 --2- 172.21.15.181:0/4293029190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098060670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f8090009c90 tx=0x7f80900091d0).stop 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 -- 172.21.15.181:0/4293029190 shutdown_connections 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 --2- 172.21.15.181:0/4293029190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098060670 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 -- 172.21.15.181:0/4293029190 >> 172.21.15.181:0/4293029190 conn(0x7f8098061c90 msgr2=0x7f80980d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 -- 172.21.15.181:0/4293029190 shutdown_connections 2020-11-16T22:32:33.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.077+0000 7f809d530700 1 -- 172.21.15.181:0/4293029190 wait complete. 2020-11-16T22:32:33.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 Processor -- start 2020-11-16T22:32:33.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 -- start start 2020-11-16T22:32:33.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8098060290 0x7f80981726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 0x7f80980b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809815b790 con 0x7f8098060290 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098172c70 con 0x7f8098157e60 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809d530700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8098172f10 con 0x7f80980c91a0 2020-11-16T22:32:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 0x7f80980b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f809cd2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 0x7f80980b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41572/0 (socket says 172.21.15.181:41572) 2020-11-16T22:32:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f808f7fe700 1 -- 172.21.15.181:0/3639112842 learned_addr learned my addr 172.21.15.181:0/3639112842 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.081+0000 7f8097fff700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8098060290 0x7f80981726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 -- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 msgr2=0x7f80980b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 0x7f80980b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 -- 172.21.15.181:0/3639112842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8098060290 msgr2=0x7f80981726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8098060290 0x7f80981726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 -- 172.21.15.181:0/3639112842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80981731b0 con 0x7f8098157e60 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809cd2f700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098154400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f8090013000 tx=0x7f8090016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809001c6c0 con 0x7f8098157e60 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809001c820 con 0x7f8098157e60 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809d530700 1 -- 172.21.15.181:0/3639112842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8098173450 con 0x7f8098157e60 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809001cbc0 con 0x7f8098157e60 2020-11-16T22:32:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f809d530700 1 -- 172.21.15.181:0/3639112842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80980b5dd0 con 0x7f8098157e60 2020-11-16T22:32:33.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8068004f90 con 0x7f8098157e60 2020-11-16T22:32:33.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8090017070 con 0x7f8098157e60 2020-11-16T22:32:33.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 0x7f8074057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:33.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.085+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f8090070380 con 0x7f8098157e60 2020-11-16T22:32:33.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.089+0000 7f8097fff700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 0x7f8074057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:33.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.089+0000 7f8097fff700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 0x7f8074057550 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f80880055a0 tx=0x7f8088009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:33.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.093+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8090046690 con 0x7f8098157e60 2020-11-16T22:32:33.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.245+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8068003b30 con 0x7f8098157e60 2020-11-16T22:32:33.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f8094ff9700 1 -- 172.21.15.181:0/3639112842 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8090015020 con 0x7f8098157e60 2020-11-16T22:32:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 msgr2=0x7f8074057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 0x7f8074057550 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f80880055a0 tx=0x7f8088009040).stop 2020-11-16T22:32:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 msgr2=0x7f8098154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098154400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f8090013000 tx=0x7f8090016000).stop 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 shutdown_connections 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8074055090 0x7f8074057550 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8098157e60 0x7f8098154400 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8098060290 0x7f80981726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 --2- 172.21.15.181:0/3639112842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80980c91a0 0x7f80980b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.949+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 >> 172.21.15.181:0/3639112842 conn(0x7f8098061c90 msgr2=0x7f80980d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:33.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.953+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 shutdown_connections 2020-11-16T22:32:33.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:33.953+0000 7f808dffb700 1 -- 172.21.15.181:0/3639112842 wait complete. 2020-11-16T22:32:33.940 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:32:33.955 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:32:33.955 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:32:33.956 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:32:33.956 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.956 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:32:33.956 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:32:33.957 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:32:33.957 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.957 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:32:33.957 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:32:33.957 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:32:33.958 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.958 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:32:33.958 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:32:33.958 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:32:33.959 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.959 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:32:33.959 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:32:33.959 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:32:33.959 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.960 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:32:33.960 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:32:33.960 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:32:33.960 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.961 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:32:33.961 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:32:33.961 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:32:33.961 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.961 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:32:33.962 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:32:33.962 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:32:33.962 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.962 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:32:33.963 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:32:33.963 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:32:33.963 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:32:33.973 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:32:33.973 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:32:33.974 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:32:33.974 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:32:33.975 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:32:33.975 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:32:33.976 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:32:33.977 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:32:33.978 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:32:33.978 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:32:33.979 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:32:33.979 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:32:33.980 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:32:33.980 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:32:33.984 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:32:33.985 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:32:33.986 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:32:33.986 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:32:33.987 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:32:33.987 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:33.988 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:33.988 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:33.989 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:32:33.989 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:32:33.989 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:33.990 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:34.020 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:32:34.020 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:32:34.021 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:32:34.021 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:34.022 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:34.022 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:34.023 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:32:34.023 DEBUG:teuthology.parallel:result is None 2020-11-16T22:32:34.024 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:32:34.045 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:32:34.062 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:32:34.064 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:32:34.066 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:32:34.073 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:32:34.104 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:32:34.107 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:32:34.120 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:32:34.121 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:32:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.189+0000 7febe4832700 1 Processor -- start 2020-11-16T22:32:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.189+0000 7febe4832700 1 -- start start 2020-11-16T22:32:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febe0063ec0 0x7febe0064370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe0062210 con 0x7febe00637e0 2020-11-16T22:32:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe0062620 con 0x7febe0063ec0 2020-11-16T22:32:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00628c0 con 0x7febe0061940 2020-11-16T22:32:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41626/0 (socket says 172.21.15.181:41626) 2020-11-16T22:32:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 -- 172.21.15.181:0/406902821 learned_addr learned my addr 172.21.15.181:0/406902821 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febdf59e700 1 --2- 172.21.15.181:0/406902821 >> v2:172.21.15.181:3300/0 conn(0x7febe0063ec0 0x7febe0064370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 -- 172.21.15.181:0/406902821 >> v2:172.21.15.181:3300/0 conn(0x7febe0063ec0 msgr2=0x7febe0064370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 --2- 172.21.15.181:0/406902821 >> v2:172.21.15.181:3300/0 conn(0x7febe0063ec0 0x7febe0064370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 -- 172.21.15.181:0/406902821 >> v1:172.21.15.181:6789/0 conn(0x7febe00637e0 legacy=0x7febe0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 -- 172.21.15.181:0/406902821 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0062b60 con 0x7febe0061940 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febded9d700 1 --2- 172.21.15.181:0/406902821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7febd8009d50 tx=0x7febd8009290).ready entity=mon.2 client_cookie=1a7198dbc60f205f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febddd9b700 1 -- 172.21.15.181:0/406902821 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800bbc0 con 0x7febe0061940 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febddd9b700 1 -- 172.21.15.181:0/406902821 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800bd20 con 0x7febe0061940 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febddd9b700 1 -- 172.21.15.181:0/406902821 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8011610 con 0x7febe0061940 2020-11-16T22:32:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- 172.21.15.181:0/406902821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 msgr2=0x7febe0061d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:34.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 --2- 172.21.15.181:0/406902821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7febd8009d50 tx=0x7febd8009290).stop 2020-11-16T22:32:34.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- 172.21.15.181:0/406902821 shutdown_connections 2020-11-16T22:32:34.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 --2- 172.21.15.181:0/406902821 >> v2:172.21.15.181:3300/0 conn(0x7febe0063ec0 0x7febe0064370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:34.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 --2- 172.21.15.181:0/406902821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe0061940 0x7febe0061d20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:34.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- 172.21.15.181:0/406902821 >> 172.21.15.181:0/406902821 conn(0x7febe00d22f0 msgr2=0x7febe00d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:34.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- 172.21.15.181:0/406902821 shutdown_connections 2020-11-16T22:32:34.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- 172.21.15.181:0/406902821 wait complete. 2020-11-16T22:32:34.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 Processor -- start 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.193+0000 7febe4832700 1 -- start start 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe00637e0 0x7febe013d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe0063ec0 0x7febe0172850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01731b0 con 0x7febe0063ec0 2020-11-16T22:32:34.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0173450 con 0x7febe0061940 2020-11-16T22:32:34.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe01736f0 con 0x7febe00637e0 2020-11-16T22:32:34.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febde59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe00637e0 0x7febe013d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33790/0 (socket says 172.21.15.181:33790) 2020-11-16T22:32:34.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 -- 172.21.15.181:0/1926200427 learned_addr learned my addr 172.21.15.181:0/1926200427 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:34.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febdf59e700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe0063ec0 0x7febe0172850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 -- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe00637e0 msgr2=0x7febe013d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:34.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe00637e0 0x7febe013d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 -- 172.21.15.181:0/1926200427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe0063ec0 msgr2=0x7febe0172850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe0063ec0 0x7febe0172850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 -- 172.21.15.181:0/1926200427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe013c660 con 0x7febe0061940 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febded9d700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7febe0062a00 tx=0x7febd8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd801b5e0 con 0x7febe0061940 2020-11-16T22:32:34.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd801b740 con 0x7febe0061940 2020-11-16T22:32:34.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd801bac0 con 0x7febe0061940 2020-11-16T22:32:34.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe013c900 con 0x7febe0061940 2020-11-16T22:32:34.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.197+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe0170c40 con 0x7febe0061940 2020-11-16T22:32:34.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7febd8017070 con 0x7febe0061940 2020-11-16T22:32:34.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe0063520 con 0x7febe0061940 2020-11-16T22:32:34.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febd6ffd700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 0x7febbc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:34.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5511+0+0 (secure 0 0 0) 0x7febd806fb60 con 0x7febe0061940 2020-11-16T22:32:34.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febde59c700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 0x7febbc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:34.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.201+0000 7febde59c700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 0x7febbc057520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7febe00d3170 tx=0x7febc8007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:34.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.205+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7febe0063520 con 0x7febe0061940 2020-11-16T22:32:34.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:34.369+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7febe013cda0 con 0x7febe0061940 2020-11-16T22:32:35.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.181+0000 7febd6ffd700 1 -- 172.21.15.181:0/1926200427 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7febe013cda0 con 0x7febe0061940 2020-11-16T22:32:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 msgr2=0x7febbc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 0x7febbc057520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7febe00d3170 tx=0x7febc8007b90).stop 2020-11-16T22:32:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 msgr2=0x7febe0170750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7febe0062a00 tx=0x7febd8016000).stop 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 shutdown_connections 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7febbc055060 0x7febbc057520 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe0063ec0 0x7febe0172850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febe0061940 0x7febe0170750 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 --2- 172.21.15.181:0/1926200427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febe00637e0 0x7febe013d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 >> 172.21.15.181:0/1926200427 conn(0x7febe00d22f0 msgr2=0x7febe0163a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:35.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 shutdown_connections 2020-11-16T22:32:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.185+0000 7febe4832700 1 -- 172.21.15.181:0/1926200427 wait complete. 2020-11-16T22:32:35.174 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:32:35.189 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:32:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 Processor -- start 2020-11-16T22:32:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 -- start start 2020-11-16T22:32:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39300cf920 0x7f39300656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300c85e0 con 0x7f39300cf4a0 2020-11-16T22:32:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393015ff70 con 0x7f39300cf920 2020-11-16T22:32:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3937025700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300c8840 con 0x7f393015d140 2020-11-16T22:32:35.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3936023700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3936824700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39300cf920 0x7f39300656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3935822700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f39300cf4a0 0x7f3930064fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48110/0 (socket says 172.21.15.181:48110) 2020-11-16T22:32:35.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.325+0000 7f3935822700 1 -- 172.21.15.181:0/851404295 learned_addr learned my addr 172.21.15.181:0/851404295 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:35.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3936023700 1 -- 172.21.15.181:0/851404295 >> v2:172.21.15.181:3300/0 conn(0x7f39300cf920 msgr2=0x7f39300656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:35.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3936023700 1 --2- 172.21.15.181:0/851404295 >> v2:172.21.15.181:3300/0 conn(0x7f39300cf920 0x7f39300656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3936023700 1 -- 172.21.15.181:0/851404295 >> v1:172.21.15.181:6789/0 conn(0x7f39300cf4a0 legacy=0x7f3930064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3936023700 1 -- 172.21.15.181:0/851404295 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300c8ae0 con 0x7f393015d140 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3936023700 1 --2- 172.21.15.181:0/851404295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393015d520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f392c00b9a0 tx=0x7f392c006df0).ready entity=mon.2 client_cookie=e745a0607b357cc2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3935021700 1 -- 172.21.15.181:0/851404295 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0097b0 con 0x7f393015d140 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3935021700 1 -- 172.21.15.181:0/851404295 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c009910 con 0x7f393015d140 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3935021700 1 -- 172.21.15.181:0/851404295 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c009c90 con 0x7f393015d140 2020-11-16T22:32:35.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- 172.21.15.181:0/851404295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 msgr2=0x7f393015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 --2- 172.21.15.181:0/851404295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393015d520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f392c00b9a0 tx=0x7f392c006df0).stop 2020-11-16T22:32:35.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- 172.21.15.181:0/851404295 shutdown_connections 2020-11-16T22:32:35.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 --2- 172.21.15.181:0/851404295 >> v2:172.21.15.181:3300/0 conn(0x7f39300cf920 0x7f39300656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 --2- 172.21.15.181:0/851404295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393015d520 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- 172.21.15.181:0/851404295 >> 172.21.15.181:0/851404295 conn(0x7f39300d2dc0 msgr2=0x7f39300ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:35.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- 172.21.15.181:0/851404295 shutdown_connections 2020-11-16T22:32:35.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- 172.21.15.181:0/851404295 wait complete. 2020-11-16T22:32:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 Processor -- start 2020-11-16T22:32:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.329+0000 7f3937025700 1 -- start start 2020-11-16T22:32:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 0x7f393015f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393013c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015e540 con 0x7f39300cf920 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015e770 con 0x7f39300cf4a0 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393015ea10 con 0x7f393015d140 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936824700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393013c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33812/0 (socket says 172.21.15.181:33812) 2020-11-16T22:32:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 -- 172.21.15.181:0/2331430439 learned_addr learned my addr 172.21.15.181:0/2331430439 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3935822700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 0x7f393015f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 -- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 msgr2=0x7f393013c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393013c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 -- 172.21.15.181:0/2331430439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 msgr2=0x7f393015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 0x7f393015f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 -- 172.21.15.181:0/2331430439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393015ecb0 con 0x7f39300cf4a0 2020-11-16T22:32:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3935822700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 0x7f393015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936824700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393013c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3936023700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f392c00c010 tx=0x7f392c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c010dc0 con 0x7f39300cf4a0 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c018400 con 0x7f39300cf4a0 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393015ef50 con 0x7f39300cf4a0 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0187a0 con 0x7f39300cf4a0 2020-11-16T22:32:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393016f180 con 0x7f39300cf4a0 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f392c02b070 con 0x7f39300cf4a0 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393016f370 con 0x7f39300cf4a0 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 0x7f3908057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f392c06ef80 con 0x7f39300cf4a0 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.333+0000 7f3935822700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 0x7f3908057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.337+0000 7f3935822700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 0x7f3908057520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f39300b1850 tx=0x7f3920009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.341+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f392c045510 con 0x7f39300cf4a0 2020-11-16T22:32:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.541+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3930044f60 con 0x7f39300cf4a0 2020-11-16T22:32:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.541+0000 7f391e7fc700 1 -- 172.21.15.181:0/2331430439 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5978 (secure 0 0 0) 0x7f392c006980 con 0x7f39300cf4a0 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:32:35.531 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:32:35.534 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.536 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.537 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.538 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.540 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:32:35.541 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.543 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:32:35.544 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.545 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.547 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.548 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:32:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.552 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:32:35.553 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.554 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:32:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.557 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.558 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:32:35.560 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:32:35.561 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.561 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:35.561 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:32:35.561 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:32:35.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 msgr2=0x7f3908057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 0x7f3908057520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f39300b1850 tx=0x7f3920009040).stop 2020-11-16T22:32:35.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 msgr2=0x7f39300ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f392c00c010 tx=0x7f392c017000).stop 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 shutdown_connections 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3908055060 0x7f3908057520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f393015d140 0x7f393013c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39300cf4a0 0x7f39300ccba0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 --2- 172.21.15.181:0/2331430439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39300cf920 0x7f393015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 >> 172.21.15.181:0/2331430439 conn(0x7f39300d2dc0 msgr2=0x7f39300d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 shutdown_connections 2020-11-16T22:32:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.545+0000 7f3937025700 1 -- 172.21.15.181:0/2331430439 wait complete. 2020-11-16T22:32:35.565 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 Processor -- start 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- start start 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 0x7f466c066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c064050 con 0x7f466c063b80 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c0642f0 con 0x7f466c0b6c20 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c064590 con 0x7f466c15e270 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41662/0 (socket says 172.21.15.181:41662) 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 -- 172.21.15.181:0/953325366 learned_addr learned my addr 172.21.15.181:0/953325366 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4672346700 1 --2- 172.21.15.181:0/953325366 >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 0x7f466c066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 -- 172.21.15.181:0/953325366 >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 msgr2=0x7f466c066ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 --2- 172.21.15.181:0/953325366 >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 0x7f466c066ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 -- 172.21.15.181:0/953325366 >> v1:172.21.15.175:6789/0 conn(0x7f466c063b80 legacy=0x7f466c0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 -- 172.21.15.181:0/953325366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c064830 con 0x7f466c15e270 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4672346700 1 --2- 172.21.15.181:0/953325366 >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 0x7f466c066ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4671b45700 1 --2- 172.21.15.181:0/953325366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f4660006fd0 tx=0x7f4660006eb0).ready entity=mon.2 client_cookie=13bad5c04f63a109 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4670b43700 1 -- 172.21.15.181:0/953325366 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46600098d0 con 0x7f466c15e270 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4670b43700 1 -- 172.21.15.181:0/953325366 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660009a30 con 0x7f466c15e270 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4670b43700 1 -- 172.21.15.181:0/953325366 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660009db0 con 0x7f466c15e270 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- 172.21.15.181:0/953325366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 msgr2=0x7f466c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 --2- 172.21.15.181:0/953325366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f4660006fd0 tx=0x7f4660006eb0).stop 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- 172.21.15.181:0/953325366 shutdown_connections 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 --2- 172.21.15.181:0/953325366 >> v2:172.21.15.175:3300/0 conn(0x7f466c0b6c20 0x7f466c066ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 --2- 172.21.15.181:0/953325366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c15e270 0x7f466c15e650 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- 172.21.15.181:0/953325366 >> 172.21.15.181:0/953325366 conn(0x7f466c0d4790 msgr2=0x7f466c0d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- 172.21.15.181:0/953325366 shutdown_connections 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.701+0000 7f4673da9700 1 -- 172.21.15.181:0/953325366 wait complete. 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.705+0000 7f4673da9700 1 Processor -- start 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.705+0000 7f4673da9700 1 -- start start 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466c0b6c20 0x7f466c0b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 0x7f466c0ba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c158230 con 0x7f466c15e270 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c158460 con 0x7f466c0b6c20 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c158700 con 0x7f466c063b80 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41668/0 (socket says 172.21.15.181:41668) 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 -- 172.21.15.181:0/142403955 learned_addr learned my addr 172.21.15.181:0/142403955 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671344700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466c0b6c20 0x7f466c0b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4672346700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 0x7f466c0ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 -- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466c0b6c20 msgr2=0x7f466c0b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466c0b6c20 0x7f466c0b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 -- 172.21.15.181:0/142403955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 msgr2=0x7f466c0ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 0x7f466c0ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 -- 172.21.15.181:0/142403955 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c1589a0 con 0x7f466c063b80 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4672346700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 0x7f466c0ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671b45700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f466000b1b0 tx=0x7f4660015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660011d50 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466001e460 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c158c40 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466001e800 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c063190 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c069eb0 con 0x7f466c063b80 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4660025030 con 0x7f466c063b80 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 0x7f4644057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f466006f260 con 0x7f466c063b80 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671344700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 0x7f4644057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.709+0000 7f4671344700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 0x7f4644057520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f466c067d50 tx=0x7f4668007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.717+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f46600458a0 con 0x7f466c063b80 2020-11-16T22:32:35.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f466c063410 con 0x7f466c063b80 2020-11-16T22:32:35.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f465dffb700 1 -- 172.21.15.181:0/142403955 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13724 (secure 0 0 0) 0x7f46600305b0 con 0x7f466c063b80 2020-11-16T22:32:35.860 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:35.860 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":75,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:35.175807+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 msgr2=0x7f4644057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 0x7f4644057520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f466c067d50 tx=0x7f4668007b00).stop 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 msgr2=0x7f466c062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f466000b1b0 tx=0x7f4660015000).stop 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 shutdown_connections 2020-11-16T22:32:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4644055060 0x7f4644057520 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f466c15e270 0x7f466c0ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f466c063b80 0x7f466c062ca0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 --2- 172.21.15.181:0/142403955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466c0b6c20 0x7f466c0b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 >> 172.21.15.181:0/142403955 conn(0x7f466c0d4790 msgr2=0x7f466c15c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 shutdown_connections 2020-11-16T22:32:35.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:35.873+0000 7f4673da9700 1 -- 172.21.15.181:0/142403955 wait complete. 2020-11-16T22:32:35.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:32:35.876 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:32:35.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.009+0000 7faa18894700 1 Processor -- start 2020-11-16T22:32:35.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.009+0000 7faa18894700 1 -- start start 2020-11-16T22:32:35.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faa1415de80 0x7faa140b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:35.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1415e2e0 con 0x7faa140b2190 2020-11-16T22:32:35.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140b2a60 con 0x7faa1415ff80 2020-11-16T22:32:36.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa140ce460 con 0x7faa1415de80 2020-11-16T22:32:36.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1359e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faa1415ff80 0x7faa14160410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48142/0 (socket says 172.21.15.181:48142) 2020-11-16T22:32:36.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1359e700 1 -- 172.21.15.181:0/2785163441 learned_addr learned my addr 172.21.15.181:0/2785163441 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:36.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 --2- 172.21.15.181:0/2785163441 >> v2:172.21.15.175:3300/0 conn(0x7faa1415de80 0x7faa140b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963814690 0 0) 0x7faa140b2a60 con 0x7faa1415ff80 2020-11-16T22:32:36.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa00003240 con 0x7faa1415ff80 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695419066 0 0) 0x7faa1415e2e0 con 0x7faa140b2190 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa140b2a60 con 0x7faa140b2190 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1648927462 0 0) 0x7faa00003240 con 0x7faa1415ff80 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 >> v2:172.21.15.175:3300/0 conn(0x7faa1415de80 msgr2=0x7faa140b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 --2- 172.21.15.181:0/2785163441 >> v2:172.21.15.175:3300/0 conn(0x7faa1415de80 0x7faa140b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 >> v1:172.21.15.175:6789/0 conn(0x7faa140b2190 legacy=0x7faa140cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140ce700 con 0x7faa1415ff80 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7faa0c001860 con 0x7faa1415ff80 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 >> v1:172.21.15.181:6789/0 conn(0x7faa1415ff80 legacy=0x7faa14160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00004920 0x7faa00004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 0x7faa00003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003df0 0x7faa0000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140ce460 con 0x7faa00003df0 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00004350 con 0x7faa00004920 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0000bca0 con 0x7faa00003480 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 0x7faa00003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1359e700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003df0 0x7faa0000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa12d9d700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00004920 0x7faa00004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 -- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00004920 msgr2=0x7faa00004d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00004920 0x7faa00004d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 -- 172.21.15.181:0/2785163441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003df0 msgr2=0x7faa0000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003df0 0x7faa0000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 -- 172.21.15.181:0/2785163441 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00003240 con 0x7faa00003480 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1359e700 1 -- 172.21.15.181:0/2785163441 reap_dead start 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa12d9d700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00004920 0x7faa00004d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa1259c700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 0x7faa00003900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7faa0800bc00 tx=0x7faa08006df0).ready entity=mon.2 client_cookie=489397a0814246f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800b300 con 0x7faa00003480 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800ea90 con 0x7faa00003480 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa11d9b700 1 -- 172.21.15.181:0/2785163441 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800ec00 con 0x7faa00003480 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 msgr2=0x7faa00003900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 0x7faa00003900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7faa0800bc00 tx=0x7faa08006df0).stop 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- 172.21.15.181:0/2785163441 shutdown_connections 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 --2- 172.21.15.181:0/2785163441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa00003480 0x7faa00003900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- 172.21.15.181:0/2785163441 >> 172.21.15.181:0/2785163441 conn(0x7faa140cfe90 msgr2=0x7faa140d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- 172.21.15.181:0/2785163441 shutdown_connections 2020-11-16T22:32:36.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.013+0000 7faa18894700 1 -- 172.21.15.181:0/2785163441 wait complete. 2020-11-16T22:32:36.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 Processor -- start 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- start start 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003480 0x7faa1415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1416ade0 0x7faa1415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416b320 con 0x7faa00003480 2020-11-16T22:32:36.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416b5c0 con 0x7faa140b8860 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416b860 con 0x7faa1416ade0 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1416ade0 0x7faa1415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33850/0 (socket says 172.21.15.181:33850) 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 -- 172.21.15.181:0/2009650527 learned_addr learned my addr 172.21.15.181:0/2009650527 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:36.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa12d9d700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003480 0x7faa1415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 -- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1416ade0 msgr2=0x7faa1415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1416ade0 0x7faa1415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 -- 172.21.15.181:0/2009650527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003480 msgr2=0x7faa1415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003480 0x7faa1415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 -- 172.21.15.181:0/2009650527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1416bb00 con 0x7faa140b8860 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa1359e700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7faa14160360 tx=0x7faa1415a510).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0077f0 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00fbd0 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa140b4e00 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c012350 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.017+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa140b5790 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa14044f60 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faa0c01b030 con 0x7faa140b8860 2020-11-16T22:32:36.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7fa9faffd700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 0x7fa9f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7faa0c06d410 con 0x7faa140b8860 2020-11-16T22:32:36.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7faa12d9d700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 0x7fa9f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.021+0000 7faa12d9d700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 0x7fa9f0057520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7faa14045400 tx=0x7faa04009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.029+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faa0c0439a0 con 0x7faa140b8860 2020-11-16T22:32:36.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faa140b5ed0 con 0x7faa140b8860 2020-11-16T22:32:36.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7fa9faffd700 1 -- 172.21.15.181:0/2009650527 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v75) v1 ==== 110+0+10 (secure 0 0 0) 0x7faa0c019030 con 0x7faa140b8860 2020-11-16T22:32:36.171 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:32:36.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 msgr2=0x7fa9f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 0x7fa9f0057520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7faa14045400 tx=0x7faa04009040).stop 2020-11-16T22:32:36.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 msgr2=0x7faa140b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7faa14160360 tx=0x7faa1415a510).stop 2020-11-16T22:32:36.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 shutdown_connections 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa9f0055060 0x7fa9f0057520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa140b8860 0x7faa140b52a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa00003480 0x7faa1415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 --2- 172.21.15.181:0/2009650527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1416ade0 0x7faa1415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 >> 172.21.15.181:0/2009650527 conn(0x7faa140cfe90 msgr2=0x7faa14145360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 shutdown_connections 2020-11-16T22:32:36.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.185+0000 7faa18894700 1 -- 172.21.15.181:0/2009650527 wait complete. 2020-11-16T22:32:36.186 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:32:36.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 Processor -- start 2020-11-16T22:32:36.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 -- start start 2020-11-16T22:32:36.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 0x7f40e8145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 0x7f40e81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e80d2320 0x7f40e80d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e8081f20 con 0x7f40e8145770 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e8060470 con 0x7f40e80d2320 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40edfce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8060710 con 0x7f40e8160450 2020-11-16T22:32:36.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e80d2320 0x7f40e80d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 0x7f40e81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e80d2320 0x7f40e80d27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33856/0 (socket says 172.21.15.181:33856) 2020-11-16T22:32:36.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40e7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 0x7f40e81608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41698/0 (socket says 172.21.15.181:41698) 2020-11-16T22:32:36.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 -- 172.21.15.181:0/1550754434 learned_addr learned my addr 172.21.15.181:0/1550754434 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:36.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ecfcc700 1 --2- 172.21.15.181:0/1550754434 >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 0x7f40e8145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 -- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 msgr2=0x7f40e81608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 0x7f40e81608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 -- 172.21.15.181:0/1550754434 >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 msgr2=0x7f40e8145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/1550754434 >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 0x7f40e8145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 -- 172.21.15.181:0/1550754434 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80609b0 con 0x7f40e80d2320 2020-11-16T22:32:36.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ecfcc700 1 --2- 172.21.15.181:0/1550754434 >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 0x7f40e8145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.325+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e80d2320 0x7f40e80d27b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f40e8083910 tx=0x7f40e0006eb0).ready entity=mon.1 client_cookie=d0a89faf9a785dc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40e77fe700 1 -- 172.21.15.181:0/1550754434 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0009810 con 0x7f40e80d2320 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40e77fe700 1 -- 172.21.15.181:0/1550754434 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0009970 con 0x7f40e80d2320 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40e77fe700 1 -- 172.21.15.181:0/1550754434 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0009cf0 con 0x7f40e80d2320 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e80d2320 msgr2=0x7f40e80d27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 --2- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e80d2320 0x7f40e80d27b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f40e8083910 tx=0x7f40e0006eb0).stop 2020-11-16T22:32:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- 172.21.15.181:0/1550754434 shutdown_connections 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 --2- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e80d2320 0x7f40e80d27b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 --2- 172.21.15.181:0/1550754434 >> v2:172.21.15.175:3300/0 conn(0x7f40e8145770 0x7f40e8145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 --2- 172.21.15.181:0/1550754434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e8160450 0x7f40e81608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- 172.21.15.181:0/1550754434 >> 172.21.15.181:0/1550754434 conn(0x7f40e80d6760 msgr2=0x7f40e80d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- 172.21.15.181:0/1550754434 shutdown_connections 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- 172.21.15.181:0/1550754434 wait complete. 2020-11-16T22:32:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 Processor -- start 2020-11-16T22:32:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.329+0000 7f40edfce700 1 -- start start 2020-11-16T22:32:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 0x7f40e8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e8145770 0x7f40e8162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8166a00 con 0x7f40e8145770 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8166ca0 con 0x7f40e8160450 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8166f40 con 0x7f40e80d2320 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ecfcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 0x7f40e8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33862/0 (socket says 172.21.15.181:33862) 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 -- 172.21.15.181:0/2511421751 learned_addr learned my addr 172.21.15.181:0/2511421751 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e7fff700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e8145770 0x7f40e8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 -- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 msgr2=0x7f40e8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 0x7f40e8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 -- 172.21.15.181:0/2511421751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e8145770 msgr2=0x7f40e8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e8145770 0x7f40e8162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 -- 172.21.15.181:0/2511421751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e81671e0 con 0x7f40e8160450 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ecfcc700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 0x7f40e8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ed7cd700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f40e8083940 tx=0x7f40e0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0002f10 con 0x7f40e8160450 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0006890 con 0x7f40e8160450 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e8167480 con 0x7f40e8160450 2020-11-16T22:32:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e001d5d0 con 0x7f40e8160450 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e8166650 con 0x7f40e8160450 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e8044f60 con 0x7f40e8160450 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f40e0025030 con 0x7f40e8160450 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 0x7f40c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f40e006f2a0 con 0x7f40e8160450 2020-11-16T22:32:36.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ecfcc700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 0x7f40c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.333+0000 7f40ecfcc700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 0x7f40c0057520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f40dc00bb20 tx=0x7f40dc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.341+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f40e0045830 con 0x7f40e8160450 2020-11-16T22:32:36.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.497+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f40e81667e0 con 0x7f40e8160450 2020-11-16T22:32:36.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.497+0000 7f40e4ff9700 1 -- 172.21.15.181:0/2511421751 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f40e0023020 con 0x7f40e8160450 2020-11-16T22:32:36.485 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:36.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 msgr2=0x7f40c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 0x7f40c0057520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f40dc00bb20 tx=0x7f40dc008040).stop 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 msgr2=0x7f40e8162930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f40e8083940 tx=0x7f40e0015000).stop 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 shutdown_connections 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f40c0055060 0x7f40c0057520 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8160450 0x7f40e8162930 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40e80d2320 0x7f40e8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 --2- 172.21.15.181:0/2511421751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e8145770 0x7f40e8162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 >> 172.21.15.181:0/2511421751 conn(0x7f40e80d6760 msgr2=0x7f40e80d5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 shutdown_connections 2020-11-16T22:32:36.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.501+0000 7f40edfce700 1 -- 172.21.15.181:0/2511421751 wait complete. 2020-11-16T22:32:36.507 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:32:36.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 Processor -- start 2020-11-16T22:32:36.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- start start 2020-11-16T22:32:36.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5afc167ac0 0x7f5afc167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5afc168390 con 0x7f5afc1672c0 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5afc0b2060 con 0x7f5afc0b3220 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc0b5f00 con 0x7f5afc167ac0 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5afc167ac0 0x7f5afc167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afa59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5afc0b3220 0x7f5afc166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48172/0 (socket says 172.21.15.181:48172) 2020-11-16T22:32:36.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5afc167ac0 0x7f5afc167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33866/0 (socket says 172.21.15.181:33866) 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afa59c700 1 -- 172.21.15.181:0/3642336924 learned_addr learned my addr 172.21.15.181:0/3642336924 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 -- 172.21.15.181:0/3642336924 >> v1:172.21.15.181:6789/0 conn(0x7f5afc0b3220 legacy=0x7f5afc166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 -- 172.21.15.181:0/3642336924 >> v1:172.21.15.175:6789/0 conn(0x7f5afc1672c0 legacy=0x7f5afc161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 -- 172.21.15.181:0/3642336924 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0b2300 con 0x7f5afc167ac0 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5afad9d700 1 --2- 172.21.15.181:0/3642336924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc167ea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5aec009c90 tx=0x7f5aec0091d0).ready entity=mon.1 client_cookie=64ef9a23492426e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5af9d9b700 1 -- 172.21.15.181:0/3642336924 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00bbc0 con 0x7f5afc167ac0 2020-11-16T22:32:36.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5af9d9b700 1 -- 172.21.15.181:0/3642336924 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00bd20 con 0x7f5afc167ac0 2020-11-16T22:32:36.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5af9d9b700 1 -- 172.21.15.181:0/3642336924 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec011610 con 0x7f5afc167ac0 2020-11-16T22:32:36.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- 172.21.15.181:0/3642336924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 msgr2=0x7f5afc167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3642336924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc167ea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5aec009c90 tx=0x7f5aec0091d0).stop 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- 172.21.15.181:0/3642336924 shutdown_connections 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3642336924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc167ea0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- 172.21.15.181:0/3642336924 >> 172.21.15.181:0/3642336924 conn(0x7f5afc060bc0 msgr2=0x7f5afc0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- 172.21.15.181:0/3642336924 shutdown_connections 2020-11-16T22:32:36.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.641+0000 7f5b0124d700 1 -- 172.21.15.181:0/3642336924 wait complete. 2020-11-16T22:32:36.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.645+0000 7f5b0124d700 1 Processor -- start 2020-11-16T22:32:36.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.645+0000 7f5b0124d700 1 -- start start 2020-11-16T22:32:36.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 0x7f5afc064590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5afc1672c0 0x7f5afc14bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc064be0 con 0x7f5afc1672c0 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc064e80 con 0x7f5afc167ac0 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc065120 con 0x7f5afc0b3220 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 0x7f5afc064590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33876/0 (socket says 172.21.15.181:33876) 2020-11-16T22:32:36.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 0x7f5afc064590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41714/0 (socket says 172.21.15.181:41714) 2020-11-16T22:32:36.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 -- 172.21.15.181:0/3581587029 learned_addr learned my addr 172.21.15.181:0/3581587029 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:36.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afa59c700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5afc1672c0 0x7f5afc14bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 -- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 msgr2=0x7f5afc064590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 0x7f5afc064590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 -- 172.21.15.181:0/3581587029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5afc1672c0 msgr2=0x7f5afc14bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5afc1672c0 0x7f5afc14bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 -- 172.21.15.181:0/3581587029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0653c0 con 0x7f5afc167ac0 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afb59e700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f5af400c710 tx=0x7f5af4007af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4013410 con 0x7f5afc167ac0 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af4013570 con 0x7f5afc167ac0 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4013910 con 0x7f5afc167ac0 2020-11-16T22:32:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc065660 con 0x7f5afc167ac0 2020-11-16T22:32:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc177cf0 con 0x7f5afc167ac0 2020-11-16T22:32:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc0618b0 con 0x7f5afc167ac0 2020-11-16T22:32:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5af4010030 con 0x7f5afc167ac0 2020-11-16T22:32:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 0x7f5ad8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5af406d580 con 0x7f5afc167ac0 2020-11-16T22:32:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.649+0000 7f5afad9d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 0x7f5ad8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.653+0000 7f5afad9d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 0x7f5ad8057520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f5aec009fd0 tx=0x7f5aec016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.653+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5afc0618b0 con 0x7f5afc167ac0 2020-11-16T22:32:36.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5afc0618b0 con 0x7f5afc167ac0 2020-11-16T22:32:36.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5af2ffd700 1 -- 172.21.15.181:0/3581587029 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5afc0618b0 con 0x7f5afc167ac0 2020-11-16T22:32:36.796 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:32:36.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:32:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:32:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:32:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:32:36.798 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:32:36.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 msgr2=0x7f5ad8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 0x7f5ad8057520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f5aec009fd0 tx=0x7f5aec016000).stop 2020-11-16T22:32:36.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 msgr2=0x7f5afc14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.809+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f5af400c710 tx=0x7f5af4007af0).stop 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 shutdown_connections 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ad8055060 0x7f5ad8057520 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5afc167ac0 0x7f5afc14c3f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5afc0b3220 0x7f5afc064590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 --2- 172.21.15.181:0/3581587029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5afc1672c0 0x7f5afc14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 >> 172.21.15.181:0/3581587029 conn(0x7f5afc060bc0 msgr2=0x7f5afc0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 shutdown_connections 2020-11-16T22:32:36.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.813+0000 7f5b0124d700 1 -- 172.21.15.181:0/3581587029 wait complete. 2020-11-16T22:32:36.815 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:32:36.816 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:32:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 Processor -- start 2020-11-16T22:32:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 -- start start 2020-11-16T22:32:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 0x7efec4145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 0x7efec40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec4061b50 con 0x7efec4145a30 2020-11-16T22:32:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec40618b0 con 0x7efec40816e0 2020-11-16T22:32:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec9599700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40cb930 con 0x7efec41492c0 2020-11-16T22:32:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efec4145a30 0x7efec4080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48184/0 (socket says 172.21.15.181:48184) 2020-11-16T22:32:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec3fff700 1 -- 172.21.15.181:0/3827851512 learned_addr learned my addr 172.21.15.181:0/3827851512 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec8d98700 1 --2- 172.21.15.181:0/3827851512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 0x7efec40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec37fe700 1 --2- 172.21.15.181:0/3827851512 >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 0x7efec4145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.957+0000 7efec2ffd700 1 -- 172.21.15.181:0/3827851512 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364814259 0 0) 0x7efec4061b50 con 0x7efec4145a30 2020-11-16T22:32:36.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec2ffd700 1 -- 172.21.15.181:0/3827851512 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb0003240 con 0x7efec4145a30 2020-11-16T22:32:36.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec8d98700 1 -- 172.21.15.181:0/3827851512 >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 msgr2=0x7efec4145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:36.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec8d98700 1 --2- 172.21.15.181:0/3827851512 >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 0x7efec4145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec8d98700 1 -- 172.21.15.181:0/3827851512 >> v1:172.21.15.181:6789/0 conn(0x7efec4145a30 legacy=0x7efec4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec8d98700 1 -- 172.21.15.181:0/3827851512 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40cbbd0 con 0x7efec41492c0 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec37fe700 1 --2- 172.21.15.181:0/3827851512 >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 0x7efec4145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec8d98700 1 --2- 172.21.15.181:0/3827851512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 0x7efec40cc5b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7efebc009c90 tx=0x7efebc0091d0).ready entity=mon.2 client_cookie=55880095519e4bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec2ffd700 1 -- 172.21.15.181:0/3827851512 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00b9c0 con 0x7efec41492c0 2020-11-16T22:32:36.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec2ffd700 1 -- 172.21.15.181:0/3827851512 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00bb20 con 0x7efec41492c0 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec2ffd700 1 -- 172.21.15.181:0/3827851512 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00bea0 con 0x7efec41492c0 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- 172.21.15.181:0/3827851512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 msgr2=0x7efec40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 --2- 172.21.15.181:0/3827851512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 0x7efec40cc5b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7efebc009c90 tx=0x7efebc0091d0).stop 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- 172.21.15.181:0/3827851512 shutdown_connections 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 --2- 172.21.15.181:0/3827851512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec41492c0 0x7efec40cc5b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 --2- 172.21.15.181:0/3827851512 >> v2:172.21.15.181:3300/0 conn(0x7efec40816e0 0x7efec4145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- 172.21.15.181:0/3827851512 >> 172.21.15.181:0/3827851512 conn(0x7efec40d68c0 msgr2=0x7efec40d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:36.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- 172.21.15.181:0/3827851512 shutdown_connections 2020-11-16T22:32:36.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- 172.21.15.181:0/3827851512 wait complete. 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 Processor -- start 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.961+0000 7efec9599700 1 -- start start 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec4145a30 0x7efec40d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec41492c0 0x7efec40d4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec40ce420 con 0x7efec4145a30 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec40ce610 con 0x7efec41492c0 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40d2e30 con 0x7efec40816e0 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41728/0 (socket says 172.21.15.181:41728) 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 -- 172.21.15.181:0/989251056 learned_addr learned my addr 172.21.15.181:0/989251056 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec37fe700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec4145a30 0x7efec40d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 -- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec41492c0 msgr2=0x7efec40d4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec41492c0 0x7efec40d4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 -- 172.21.15.181:0/989251056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec4145a30 msgr2=0x7efec40d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec4145a30 0x7efec40d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 -- 172.21.15.181:0/989251056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40d30d0 con 0x7efec40816e0 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec3fff700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7efeb400c6c0 tx=0x7efeb4007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb40072e0 con 0x7efec40816e0 2020-11-16T22:32:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb4007440 con 0x7efec40816e0 2020-11-16T22:32:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec40d3370 con 0x7efec40816e0 2020-11-16T22:32:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4005840 con 0x7efec40816e0 2020-11-16T22:32:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.965+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec40d38b0 con 0x7efec40816e0 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efeb4010070 con 0x7efec40816e0 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efea3fff700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 0x7efe9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5253+0+0 (secure 0 0 0) 0x7efeb406cf80 con 0x7efec40816e0 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec40d5390 con 0x7efec40816e0 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efec37fe700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 0x7efe9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.969+0000 7efec37fe700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 0x7efe9c057520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7efec416f0a0 tx=0x7efeb8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:36.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:36.973+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efec40d5390 con 0x7efec40816e0 2020-11-16T22:32:37.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.125+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7efec40d5390 con 0x7efec40816e0 2020-11-16T22:32:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.993+0000 7efea3fff700 1 -- 172.21.15.181:0/989251056 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v76) v1 ==== 133+0+0 (secure 0 0 0) 0x7efec40d5390 con 0x7efec40816e0 2020-11-16T22:32:37.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 msgr2=0x7efe9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 0x7efe9c057520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7efec416f0a0 tx=0x7efeb8006df0).stop 2020-11-16T22:32:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 msgr2=0x7efec40d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7efeb400c6c0 tx=0x7efeb4007ac0).stop 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 shutdown_connections 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efe9c055060 0x7efe9c057520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec41492c0 0x7efec40d4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec40816e0 0x7efec40d1260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 --2- 172.21.15.181:0/989251056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec4145a30 0x7efec40d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 >> 172.21.15.181:0/989251056 conn(0x7efec40d68c0 msgr2=0x7efec4148040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 shutdown_connections 2020-11-16T22:32:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:37.997+0000 7efec9599700 1 -- 172.21.15.181:0/989251056 wait complete. 2020-11-16T22:32:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:32:37.998 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:32:38.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 Processor -- start 2020-11-16T22:32:38.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 -- start start 2020-11-16T22:32:38.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f987c066c40 0x7f987c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987c162500 con 0x7f987c0cac00 2020-11-16T22:32:38.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987c1627a0 con 0x7f987c0cafe0 2020-11-16T22:32:38.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98833ba700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987c160e90 con 0x7f987c066c40 2020-11-16T22:32:38.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9881bb7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f987c0cafe0 0x7f987c066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48200/0 (socket says 172.21.15.181:48200) 2020-11-16T22:32:38.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9881bb7700 1 -- 172.21.15.181:0/2734960898 learned_addr learned my addr 172.21.15.181:0/2734960898 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9882bb9700 1 --2- 172.21.15.181:0/2734960898 >> v2:172.21.15.175:3300/0 conn(0x7f987c066c40 0x7f987c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933572716 0 0) 0x7f987c1627a0 con 0x7f987c0cafe0 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9864003240 con 0x7f987c0cafe0 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 308775074 0 0) 0x7f9864003240 con 0x7f987c0cafe0 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 >> v2:172.21.15.175:3300/0 conn(0x7f987c066c40 msgr2=0x7f987c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 --2- 172.21.15.181:0/2734960898 >> v2:172.21.15.175:3300/0 conn(0x7f987c066c40 0x7f987c0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 >> v1:172.21.15.175:6789/0 conn(0x7f987c0cac00 legacy=0x7f987c160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c162a40 con 0x7f987c0cafe0 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f98780014e0 con 0x7f987c0cafe0 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 >> v1:172.21.15.181:6789/0 conn(0x7f987c0cafe0 legacy=0x7f987c066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 0x7f9864003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98640044e0 0x7f9864004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864004e50 0x7f986400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c160e90 con 0x7f9864004e50 2020-11-16T22:32:38.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9864004350 con 0x7f9864003730 2020-11-16T22:32:38.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986400bc50 con 0x7f98640044e0 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9882bb9700 1 --2- 172.21.15.181:0/2734960898 >> v2:172.21.15.175:3300/0 conn(0x7f987c066c40 0x7f987c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f98823b8700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 0x7f9864003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9881bb7700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98640044e0 0x7f9864004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.129+0000 7f9882bb9700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864004e50 0x7f986400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 -- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98640044e0 msgr2=0x7f9864004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98640044e0 0x7f9864004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 -- 172.21.15.181:0/2734960898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864004e50 msgr2=0x7f986400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864004e50 0x7f986400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 -- 172.21.15.181:0/2734960898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864003240 con 0x7f9864003730 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f9882bb9700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864004e50 0x7f986400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f9882bb9700 1 -- 172.21.15.181:0/2734960898 reap_dead start 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 0x7f9864003b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9870001910 tx=0x7f9870007b90).ready entity=mon.1 client_cookie=364e5c634bee5a44 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870007020 con 0x7f9864003730 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9870007180 con 0x7f9864003730 2020-11-16T22:32:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98813b6700 1 -- 172.21.15.181:0/2734960898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000c830 con 0x7f9864003730 2020-11-16T22:32:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 msgr2=0x7f9864003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 0x7f9864003b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9870001910 tx=0x7f9870007b90).stop 2020-11-16T22:32:38.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- 172.21.15.181:0/2734960898 shutdown_connections 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 --2- 172.21.15.181:0/2734960898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9864003730 0x7f9864003b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- 172.21.15.181:0/2734960898 >> 172.21.15.181:0/2734960898 conn(0x7f987c0d4590 msgr2=0x7f987c0d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- 172.21.15.181:0/2734960898 shutdown_connections 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- 172.21.15.181:0/2734960898 wait complete. 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 Processor -- start 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- start start 2020-11-16T22:32:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f987c16fec0 0x7f987c0613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f987c061890 0x7f987c170710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0cb8e0 con 0x7f987c061890 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0cbb10 con 0x7f987c16fec0 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98833ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0cbdb0 con 0x7f9864003730 2020-11-16T22:32:38.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f9881bb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f987c16fec0 0x7f987c0613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41748/0 (socket says 172.21.15.181:41748) 2020-11-16T22:32:38.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f98823b8700 1 -- 172.21.15.181:0/3080398330 learned_addr learned my addr 172.21.15.181:0/3080398330 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:38.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.133+0000 7f9882bb9700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f987c061890 0x7f987c170710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 -- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f987c16fec0 msgr2=0x7f987c0613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f987c16fec0 0x7f987c0613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 -- 172.21.15.181:0/3080398330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f987c061890 msgr2=0x7f987c170710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f987c061890 0x7f987c170710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 -- 172.21.15.181:0/3080398330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c0cc050 con 0x7f9864003730 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98823b8700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9870000dd0 tx=0x7f9870015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870007560 con 0x7f9864003730 2020-11-16T22:32:38.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98700137c0 con 0x7f9864003730 2020-11-16T22:32:38.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c0cc2f0 con 0x7f9864003730 2020-11-16T22:32:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870013b80 con 0x7f9864003730 2020-11-16T22:32:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c0cca30 con 0x7f9864003730 2020-11-16T22:32:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c162c20 con 0x7f9864003730 2020-11-16T22:32:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9870010030 con 0x7f9864003730 2020-11-16T22:32:38.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 0x7f9854057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f987006ed10 con 0x7f9864003730 2020-11-16T22:32:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f9881bb7700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 0x7f9854057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.137+0000 7f9881bb7700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 0x7f9854057520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f987c13f700 tx=0x7f987c063290).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.145+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9870045140 con 0x7f9864003730 2020-11-16T22:32:38.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:38.297+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f987c061c70 con 0x7f9864003730 2020-11-16T22:32:39.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.117+0000 7f986e7fc700 1 -- 172.21.15.181:0/3080398330 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v79) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9870048760 con 0x7f9864003730 2020-11-16T22:32:39.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 msgr2=0x7f9854057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 0x7f9854057520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f987c13f700 tx=0x7f987c063290).stop 2020-11-16T22:32:39.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 msgr2=0x7f987c16f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9870000dd0 tx=0x7f9870015000).stop 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 shutdown_connections 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9854055060 0x7f9854057520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f987c061890 0x7f987c170710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9864003730 0x7f987c16f9d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 --2- 172.21.15.181:0/3080398330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f987c16fec0 0x7f987c0613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 >> 172.21.15.181:0/3080398330 conn(0x7f987c0d4590 msgr2=0x7f987c13ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:39.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 shutdown_connections 2020-11-16T22:32:39.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.121+0000 7f98833ba700 1 -- 172.21.15.181:0/3080398330 wait complete. 2020-11-16T22:32:39.110 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:32:39.125 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:32:39.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 Processor -- start 2020-11-16T22:32:39.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 -- start start 2020-11-16T22:32:39.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa18015c440 0x7fa18015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800b4800 con 0x7fa1800b50d0 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18015d0f0 con 0x7fa18015c440 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa184ca5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18015d390 con 0x7fa1800b54b0 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41758/0 (socket says 172.21.15.181:41758) 2020-11-16T22:32:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 -- 172.21.15.181:0/1268021780 learned_addr learned my addr 172.21.15.181:0/1268021780 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17ed9d700 1 --2- 172.21.15.181:0/1268021780 >> v2:172.21.15.175:3300/0 conn(0x7fa18015c440 0x7fa18015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 -- 172.21.15.181:0/1268021780 >> v2:172.21.15.175:3300/0 conn(0x7fa18015c440 msgr2=0x7fa18015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 --2- 172.21.15.181:0/1268021780 >> v2:172.21.15.175:3300/0 conn(0x7fa18015c440 0x7fa18015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 -- 172.21.15.181:0/1268021780 >> v1:172.21.15.175:6789/0 conn(0x7fa1800b50d0 legacy=0x7fa1800ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.265+0000 7fa17dd9b700 1 -- 172.21.15.181:0/1268021780 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1800cd950 con 0x7fa1800b54b0 2020-11-16T22:32:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa17dd9b700 1 --2- 172.21.15.181:0/1268021780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa168006fd0 tx=0x7fa168006eb0).ready entity=mon.2 client_cookie=e56c215ca657b7b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa17d59a700 1 -- 172.21.15.181:0/1268021780 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168009810 con 0x7fa1800b54b0 2020-11-16T22:32:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa17d59a700 1 -- 172.21.15.181:0/1268021780 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168009970 con 0x7fa1800b54b0 2020-11-16T22:32:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa17d59a700 1 -- 172.21.15.181:0/1268021780 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168009cf0 con 0x7fa1800b54b0 2020-11-16T22:32:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- 172.21.15.181:0/1268021780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 msgr2=0x7fa1800b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- 172.21.15.181:0/1268021780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa168006fd0 tx=0x7fa168006eb0).stop 2020-11-16T22:32:39.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- 172.21.15.181:0/1268021780 shutdown_connections 2020-11-16T22:32:39.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- 172.21.15.181:0/1268021780 >> v2:172.21.15.175:3300/0 conn(0x7fa18015c440 0x7fa18015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- 172.21.15.181:0/1268021780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa1800b4310 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- 172.21.15.181:0/1268021780 >> 172.21.15.181:0/1268021780 conn(0x7fa1800cf770 msgr2=0x7fa1800cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:39.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- 172.21.15.181:0/1268021780 shutdown_connections 2020-11-16T22:32:39.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- 172.21.15.181:0/1268021780 wait complete. 2020-11-16T22:32:39.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 Processor -- start 2020-11-16T22:32:39.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- start start 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 0x7fa18016f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18015c440 0x7fa1801650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801700f0 con 0x7fa18015c440 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa180170390 con 0x7fa1800b50d0 2020-11-16T22:32:39.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa184ca5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800b6220 con 0x7fa1800b54b0 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.269+0000 7fa17e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 0x7fa18016f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41762/0 (socket says 172.21.15.181:41762) 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17dd9b700 1 -- 172.21.15.181:0/629512741 learned_addr learned my addr 172.21.15.181:0/629512741 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17ed9d700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18015c440 0x7fa1801650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 -- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 msgr2=0x7fa18016fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 -- 172.21.15.181:0/629512741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18015c440 msgr2=0x7fa1801650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18015c440 0x7fa1801650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 -- 172.21.15.181:0/629512741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1800b64c0 con 0x7fa1800b50d0 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17dd9b700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa17e59c700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 0x7fa18016f640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fa17000c710 tx=0x7fa170007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:39.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170007020 con 0x7fa1800b50d0 2020-11-16T22:32:39.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170007180 con 0x7fa1800b50d0 2020-11-16T22:32:39.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1800b6760 con 0x7fa1800b50d0 2020-11-16T22:32:39.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700064f0 con 0x7fa1800b50d0 2020-11-16T22:32:39.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801655b0 con 0x7fa1800b50d0 2020-11-16T22:32:39.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1800b6060 con 0x7fa1800b50d0 2020-11-16T22:32:39.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.273+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa170010030 con 0x7fa1800b50d0 2020-11-16T22:32:39.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.277+0000 7fa1767fc700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 0x7fa15c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:39.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.277+0000 7fa17dd9b700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 0x7fa15c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:39.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.277+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa17006d250 con 0x7fa1800b50d0 2020-11-16T22:32:39.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.277+0000 7fa17dd9b700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 0x7fa15c057520 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa18015ba10 tx=0x7fa168015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:39.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.281+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa170043410 con 0x7fa1800b50d0 2020-11-16T22:32:39.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.485+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa1800b4800 con 0x7fa1800b50d0 2020-11-16T22:32:39.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.497+0000 7fa1767fc700 1 -- 172.21.15.181:0/629512741 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v78) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa1800b4800 con 0x7fa1800b50d0 2020-11-16T22:32:39.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.505+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 msgr2=0x7fa15c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.505+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 0x7fa15c057520 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa18015ba10 tx=0x7fa168015000).stop 2020-11-16T22:32:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 msgr2=0x7fa18016f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 0x7fa18016f640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fa17000c710 tx=0x7fa170007b90).stop 2020-11-16T22:32:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 shutdown_connections 2020-11-16T22:32:39.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa15c055060 0x7fa15c057520 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa18015c440 0x7fa1801650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1800b50d0 0x7fa18016f640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 --2- 172.21.15.181:0/629512741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1800b54b0 0x7fa18016fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 >> 172.21.15.181:0/629512741 conn(0x7fa1800cf770 msgr2=0x7fa1800ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 shutdown_connections 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:39.509+0000 7fa184ca5700 1 -- 172.21.15.181:0/629512741 wait complete. 2020-11-16T22:32:39.871 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 11 and data pool 12 2020-11-16T22:32:39.872 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:39.946 INFO:tasks.ceph.mds.a.smithi181.stderr:2020-11-16T22:32:39.957+0000 7fc697e19700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:32:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.017+0000 7f75df5ca700 1 Processor -- start 2020-11-16T22:32:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.017+0000 7f75df5ca700 1 -- start start 2020-11-16T22:32:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75d80c9640 0x7f75d8064970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d8164580 con 0x7f75d80612a0 2020-11-16T22:32:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d8164360 con 0x7f75d80c9640 2020-11-16T22:32:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8164820 con 0x7f75d8061680 2020-11-16T22:32:40.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41784/0 (socket says 172.21.15.181:41784) 2020-11-16T22:32:40.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 -- 172.21.15.181:0/1729746829 learned_addr learned my addr 172.21.15.181:0/1729746829 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:40.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75ddb67700 1 --2- 172.21.15.181:0/1729746829 >> v2:172.21.15.175:3300/0 conn(0x7f75d80c9640 0x7f75d8064970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 -- 172.21.15.181:0/1729746829 >> v2:172.21.15.175:3300/0 conn(0x7f75d80c9640 msgr2=0x7f75d8064970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:40.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 --2- 172.21.15.181:0/1729746829 >> v2:172.21.15.175:3300/0 conn(0x7f75d80c9640 0x7f75d8064970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 -- 172.21.15.181:0/1729746829 >> v1:172.21.15.175:6789/0 conn(0x7f75d80612a0 legacy=0x7f75d8062ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 -- 172.21.15.181:0/1729746829 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8164ac0 con 0x7f75d8061680 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75dcb65700 1 --2- 172.21.15.181:0/1729746829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75c8006fd0 tx=0x7f75c8006eb0).ready entity=mon.2 client_cookie=7988381dd6195794 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75c7fff700 1 -- 172.21.15.181:0/1729746829 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8009810 con 0x7f75d8061680 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75c7fff700 1 -- 172.21.15.181:0/1729746829 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8009970 con 0x7f75d8061680 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75c7fff700 1 -- 172.21.15.181:0/1729746829 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8009cf0 con 0x7f75d8061680 2020-11-16T22:32:40.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- 172.21.15.181:0/1729746829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 msgr2=0x7f75d80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 --2- 172.21.15.181:0/1729746829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75c8006fd0 tx=0x7f75c8006eb0).stop 2020-11-16T22:32:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- 172.21.15.181:0/1729746829 shutdown_connections 2020-11-16T22:32:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 --2- 172.21.15.181:0/1729746829 >> v2:172.21.15.175:3300/0 conn(0x7f75d80c9640 0x7f75d8064970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 --2- 172.21.15.181:0/1729746829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d80c9150 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- 172.21.15.181:0/1729746829 >> 172.21.15.181:0/1729746829 conn(0x7f75d80cf890 msgr2=0x7f75d80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- 172.21.15.181:0/1729746829 shutdown_connections 2020-11-16T22:32:40.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.021+0000 7f75df5ca700 1 -- 172.21.15.181:0/1729746829 wait complete. 2020-11-16T22:32:40.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 Processor -- start 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- start start 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d80612a0 0x7f75d80b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 0x7f75d80600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8173b20 con 0x7f75d80612a0 2020-11-16T22:32:40.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8173dc0 con 0x7f75d80c9640 2020-11-16T22:32:40.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8174060 con 0x7f75d8061680 2020-11-16T22:32:40.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41786/0 (socket says 172.21.15.181:41786) 2020-11-16T22:32:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 -- 172.21.15.181:0/2667033277 learned_addr learned my addr 172.21.15.181:0/2667033277 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75ddb67700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 0x7f75d80600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dd366700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d80612a0 0x7f75d80b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 -- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 msgr2=0x7f75d80600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 0x7f75d80600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 -- 172.21.15.181:0/2667033277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d80612a0 msgr2=0x7f75d80b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d80612a0 0x7f75d80b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 -- 172.21.15.181:0/2667033277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8174300 con 0x7f75d8061680 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75ddb67700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 0x7f75d80600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75dcb65700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f75d8164960 tx=0x7f75c8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8011c80 con 0x7f75d8061680 2020-11-16T22:32:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8011de0 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c801e670 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d81745a0 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.025+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d80b6b70 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d8174a40 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f75c8025030 con 0x7f75d8061680 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75c57fa700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 0x7f75b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:40.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75c806f350 con 0x7f75d8061680 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75dd366700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 0x7f75b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.029+0000 7f75dd366700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 0x7f75b0057520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f75d4003300 tx=0x7f75d4007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.033+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f75c8045550 con 0x7f75d8061680 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.189+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75d80645a0 con 0x7f75d8061680 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.189+0000 7f75c57fa700 1 -- 172.21.15.181:0/2667033277 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f75c8031210 con 0x7f75d8061680 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 msgr2=0x7f75b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 0x7f75b0057520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f75d4003300 tx=0x7f75d4007b90).stop 2020-11-16T22:32:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 msgr2=0x7f75d814c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f75d8164960 tx=0x7f75c8015000).stop 2020-11-16T22:32:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 shutdown_connections 2020-11-16T22:32:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f75b0055060 0x7f75b0057520 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d80c9640 0x7f75d80600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d80612a0 0x7f75d80b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 --2- 172.21.15.181:0/2667033277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d8061680 0x7f75d814c8b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 >> 172.21.15.181:0/2667033277 conn(0x7f75d80cf890 msgr2=0x7f75d80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 shutdown_connections 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:40.193+0000 7f75df5ca700 1 -- 172.21.15.181:0/2667033277 wait complete. 2020-11-16T22:32:40.627 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:40.628 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":80,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:39.488917+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:41.238 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:41.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.381+0000 7f9b4bfff700 1 Processor -- start 2020-11-16T22:32:41.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.381+0000 7f9b4bfff700 1 -- start start 2020-11-16T22:32:41.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b4c1409d0 0x7f9b4c066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c066a20 0x7f9b4c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4c0c94f0 con 0x7f9b4c1405f0 2020-11-16T22:32:41.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4c0c9790 con 0x7f9b4c1409d0 2020-11-16T22:32:41.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0c9a30 con 0x7f9b4c066a20 2020-11-16T22:32:41.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b4c1409d0 0x7f9b4c066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9b4c1405f0 0x7f9b4c0647a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48256/0 (socket says 172.21.15.181:48256) 2020-11-16T22:32:41.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4affd700 1 -- 172.21.15.181:0/3266800634 learned_addr learned my addr 172.21.15.181:0/3266800634 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:41.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4b7fe700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c066a20 0x7f9b4c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 -- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c066a20 msgr2=0x7f9b4c15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:41.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c066a20 0x7f9b4c15f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 -- 172.21.15.181:0/3266800634 >> v1:172.21.15.181:6789/0 conn(0x7f9b4c1405f0 legacy=0x7f9b4c0647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 -- 172.21.15.181:0/3266800634 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0c9cd0 con 0x7f9b4c1409d0 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b49ffb700 1 -- 172.21.15.181:0/3266800634 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623973234 0 0) 0x7f9b4c0c94f0 con 0x7f9b4c1405f0 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4a7fc700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c1409d0 0x7f9b4c066530 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f9b3400b9a0 tx=0x7f9b34006df0).ready entity=mon.1 client_cookie=57ba3b17a813bd86 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b49ffb700 1 -- 172.21.15.181:0/3266800634 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340097b0 con 0x7f9b4c1409d0 2020-11-16T22:32:41.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b49ffb700 1 -- 172.21.15.181:0/3266800634 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34009910 con 0x7f9b4c1409d0 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b49ffb700 1 -- 172.21.15.181:0/3266800634 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34009c90 con 0x7f9b4c1409d0 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c1409d0 msgr2=0x7f9b4c066530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c1409d0 0x7f9b4c066530 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f9b3400b9a0 tx=0x7f9b34006df0).stop 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- 172.21.15.181:0/3266800634 shutdown_connections 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c066a20 0x7f9b4c15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/3266800634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c1409d0 0x7f9b4c066530 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:41.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- 172.21.15.181:0/3266800634 >> 172.21.15.181:0/3266800634 conn(0x7f9b4c0d4590 msgr2=0x7f9b4c0d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- 172.21.15.181:0/3266800634 shutdown_connections 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.385+0000 7f9b4bfff700 1 -- 172.21.15.181:0/3266800634 wait complete. 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 Processor -- start 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 -- start start 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b4c1405f0 0x7f9b4c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0cf6b0 con 0x7f9b4c1405f0 2020-11-16T22:32:41.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0cf950 con 0x7f9b4c066a20 2020-11-16T22:32:41.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0cfbf0 con 0x7f9b4c1409d0 2020-11-16T22:32:41.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41804/0 (socket says 172.21.15.181:41804) 2020-11-16T22:32:41.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4b7fe700 1 -- 172.21.15.181:0/716599947 learned_addr learned my addr 172.21.15.181:0/716599947 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33960/0 (socket says 172.21.15.181:33960) 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4a7fc700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b4c1405f0 0x7f9b4c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 -- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 msgr2=0x7f9b4c170bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 -- 172.21.15.181:0/716599947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b4c1405f0 msgr2=0x7f9b4c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b4c1405f0 0x7f9b4c0cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:41.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 -- 172.21.15.181:0/716599947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0cfe90 con 0x7f9b4c066a20 2020-11-16T22:32:41.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4b7fe700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:41.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.389+0000 7f9b4affd700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f9b44007fd0 tx=0x7f9b44007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b440072e0 con 0x7f9b4c066a20 2020-11-16T22:32:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b44007440 con 0x7f9b4c066a20 2020-11-16T22:32:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c0d0130 con 0x7f9b4c066a20 2020-11-16T22:32:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44005840 con 0x7f9b4c066a20 2020-11-16T22:32:41.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c0cf350 con 0x7f9b4c066a20 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c044f60 con 0x7f9b4c066a20 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9b4401a060 con 0x7f9b4c066a20 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 0x7f9b28057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9b4406d450 con 0x7f9b4c066a20 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.393+0000 7f9b4a7fc700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 0x7f9b28057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.397+0000 7f9b4a7fc700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 0x7f9b28057520 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9b34006e30 tx=0x7f9b34015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:41.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.401+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9b440436b0 con 0x7f9b4c066a20 2020-11-16T22:32:41.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.557+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b4c0d05d0 con 0x7f9b4c066a20 2020-11-16T22:32:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.557+0000 7f9b42ffd700 1 -- 172.21.15.181:0/716599947 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f9b4c0d05d0 con 0x7f9b4c066a20 2020-11-16T22:32:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 msgr2=0x7f9b28057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 0x7f9b28057520 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9b34006e30 tx=0x7f9b34015000).stop 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 msgr2=0x7f9b4c0d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f9b44007fd0 tx=0x7f9b44007a00).stop 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 shutdown_connections 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9b28055060 0x7f9b28057520 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b4c1409d0 0x7f9b4c170bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b4c066a20 0x7f9b4c0d0870 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 --2- 172.21.15.181:0/716599947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b4c1405f0 0x7f9b4c0cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 >> 172.21.15.181:0/716599947 conn(0x7f9b4c0d4590 msgr2=0x7f9b4c15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 shutdown_connections 2020-11-16T22:32:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:41.561+0000 7f9b4bfff700 1 -- 172.21.15.181:0/716599947 wait complete. 2020-11-16T22:32:42.196 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:42.196 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:40.492654+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:43.011 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:32:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.161+0000 7f473a345700 1 Processor -- start 2020-11-16T22:32:43.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.161+0000 7f473a345700 1 -- start start 2020-11-16T22:32:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4734145a30 0x7f4734145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47341492c0 0x7f4734080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4734061b50 con 0x7f47340814c0 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47340618b0 con 0x7f47341492c0 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47340cb930 con 0x7f4734145a30 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4734145a30 0x7f4734145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4734145a30 0x7f4734145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33966/0 (socket says 172.21.15.181:33966) 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 -- 172.21.15.181:0/2558978762 learned_addr learned my addr 172.21.15.181:0/2558978762 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4738b42700 1 --2- 172.21.15.181:0/2558978762 >> v2:172.21.15.175:3300/0 conn(0x7f47341492c0 0x7f4734080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 -- 172.21.15.181:0/2558978762 >> v2:172.21.15.175:3300/0 conn(0x7f47341492c0 msgr2=0x7f4734080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 --2- 172.21.15.181:0/2558978762 >> v2:172.21.15.175:3300/0 conn(0x7f47341492c0 0x7f4734080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 -- 172.21.15.181:0/2558978762 >> v1:172.21.15.175:6789/0 conn(0x7f47340814c0 legacy=0x7f47340cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 -- 172.21.15.181:0/2558978762 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340cbbd0 con 0x7f4734145a30 2020-11-16T22:32:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4739343700 1 --2- 172.21.15.181:0/2558978762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f4734145e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f47340d43d0 tx=0x7f472c009290).ready entity=mon.1 client_cookie=e6db9afc1bcd7fc8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4733fff700 1 -- 172.21.15.181:0/2558978762 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00bbc0 con 0x7f4734145a30 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4733fff700 1 -- 172.21.15.181:0/2558978762 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c00bd20 con 0x7f4734145a30 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f4733fff700 1 -- 172.21.15.181:0/2558978762 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c011610 con 0x7f4734145a30 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- 172.21.15.181:0/2558978762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 msgr2=0x7f4734145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 --2- 172.21.15.181:0/2558978762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f4734145e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f47340d43d0 tx=0x7f472c009290).stop 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- 172.21.15.181:0/2558978762 shutdown_connections 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 --2- 172.21.15.181:0/2558978762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f4734145e10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 --2- 172.21.15.181:0/2558978762 >> v2:172.21.15.175:3300/0 conn(0x7f47341492c0 0x7f4734080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- 172.21.15.181:0/2558978762 >> 172.21.15.181:0/2558978762 conn(0x7f47340d68c0 msgr2=0x7f47340d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- 172.21.15.181:0/2558978762 shutdown_connections 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.165+0000 7f473a345700 1 -- 172.21.15.181:0/2558978762 wait complete. 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 Processor -- start 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 -- start start 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47340814c0 0x7f4734173680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47341492c0 0x7f47340d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473415da30 con 0x7f47340814c0 2020-11-16T22:32:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47340d1750 con 0x7f4734145a30 2020-11-16T22:32:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4734172ce0 con 0x7f47341492c0 2020-11-16T22:32:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33974/0 (socket says 172.21.15.181:33974) 2020-11-16T22:32:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4739b44700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47341492c0 0x7f47340d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 -- 172.21.15.181:0/2209441056 learned_addr learned my addr 172.21.15.181:0/2209441056 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4739343700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47340814c0 0x7f4734173680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 -- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47341492c0 msgr2=0x7f47340d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47341492c0 0x7f47340d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 -- 172.21.15.181:0/2209441056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47340814c0 msgr2=0x7f4734173680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47340814c0 0x7f4734173680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 -- 172.21.15.181:0/2209441056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340d1990 con 0x7f4734145a30 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f4738b42700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f471c00eba0 tx=0x7f471c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00c8e0 con 0x7f4734145a30 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c00ca40 con 0x7f4734145a30 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.169+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47340d1c30 con 0x7f4734145a30 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00cdc0 con 0x7f4734145a30 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47341731e0 con 0x7f4734145a30 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47341733a0 con 0x7f4734145a30 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f471c012600 con 0x7f4734145a30 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f47317fa700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 0x7f4710057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f471c06e080 con 0x7f4734145a30 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f4739343700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 0x7f4710057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.173+0000 7f4739343700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 0x7f4710057520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f472c0092d0 tx=0x7f472c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.181+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f471c044280 con 0x7f4734145a30 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.381+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4734044f60 con 0x7f4734145a30 2020-11-16T22:32:43.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.517+0000 7f47317fa700 1 -- 172.21.15.181:0/2209441056 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7f471c0478a0 con 0x7f4734145a30 2020-11-16T22:32:43.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 msgr2=0x7f4710057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 0x7f4710057520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f472c0092d0 tx=0x7f472c016000).stop 2020-11-16T22:32:43.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 msgr2=0x7f47340d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f471c00eba0 tx=0x7f471c006df0).stop 2020-11-16T22:32:43.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 shutdown_connections 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4710055060 0x7f4710057520 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47341492c0 0x7f47340d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47340814c0 0x7f4734173680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 --2- 172.21.15.181:0/2209441056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4734145a30 0x7f47340d39b0 secure :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0x7f471c00eba0 tx=0x7f471c006df0).stop 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 >> 172.21.15.181:0/2209441056 conn(0x7f47340d68c0 msgr2=0x7f4734060480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 shutdown_connections 2020-11-16T22:32:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.521+0000 7f473a345700 1 -- 172.21.15.181:0/2209441056 wait complete. 2020-11-16T22:32:43.574 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.717+0000 7f71dbfff700 1 Processor -- start 2020-11-16T22:32:43.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.717+0000 7f71dbfff700 1 -- start start 2020-11-16T22:32:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f71dc15dfd0 0x7f71dc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc1611b0 con 0x7f71dc061100 2020-11-16T22:32:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc15f0b0 con 0x7f71dc15dfd0 2020-11-16T22:32:43.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc15dbe0 con 0x7f71dc157a80 2020-11-16T22:32:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41826/0 (socket says 172.21.15.181:41826) 2020-11-16T22:32:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 -- 172.21.15.181:0/1664076047 learned_addr learned my addr 172.21.15.181:0/1664076047 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:43.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71db7fe700 1 --2- 172.21.15.181:0/1664076047 >> v2:172.21.15.175:3300/0 conn(0x7f71dc15dfd0 0x7f71dc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 -- 172.21.15.181:0/1664076047 >> v2:172.21.15.175:3300/0 conn(0x7f71dc15dfd0 msgr2=0x7f71dc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:43.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 --2- 172.21.15.181:0/1664076047 >> v2:172.21.15.175:3300/0 conn(0x7f71dc15dfd0 0x7f71dc15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 -- 172.21.15.181:0/1664076047 >> v1:172.21.15.175:6789/0 conn(0x7f71dc061100 legacy=0x7f71dc061580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71d9ffb700 1 -- 172.21.15.181:0/1664076047 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678984969 0 0) 0x7f71dc1611b0 con 0x7f71dc061100 2020-11-16T22:32:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 -- 172.21.15.181:0/1664076047 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc161450 con 0x7f71dc157a80 2020-11-16T22:32:43.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71daffd700 1 --2- 172.21.15.181:0/1664076047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f71d0006fd0 tx=0x7f71d0006eb0).ready entity=mon.2 client_cookie=4122b05614444445 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:43.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71d9ffb700 1 -- 172.21.15.181:0/1664076047 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0009810 con 0x7f71dc157a80 2020-11-16T22:32:43.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71d9ffb700 1 -- 172.21.15.181:0/1664076047 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0009970 con 0x7f71dc157a80 2020-11-16T22:32:43.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71d9ffb700 1 -- 172.21.15.181:0/1664076047 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0009cf0 con 0x7f71dc157a80 2020-11-16T22:32:43.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- 172.21.15.181:0/1664076047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 msgr2=0x7f71dc0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:43.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 --2- 172.21.15.181:0/1664076047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f71d0006fd0 tx=0x7f71d0006eb0).stop 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- 172.21.15.181:0/1664076047 shutdown_connections 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 --2- 172.21.15.181:0/1664076047 >> v2:172.21.15.175:3300/0 conn(0x7f71dc15dfd0 0x7f71dc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 --2- 172.21.15.181:0/1664076047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc157a80 0x7f71dc0cc0c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- 172.21.15.181:0/1664076047 >> 172.21.15.181:0/1664076047 conn(0x7f71dc0cf630 msgr2=0x7f71dc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- 172.21.15.181:0/1664076047 shutdown_connections 2020-11-16T22:32:43.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- 172.21.15.181:0/1664076047 wait complete. 2020-11-16T22:32:43.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 Processor -- start 2020-11-16T22:32:43.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.721+0000 7f71dbfff700 1 -- start start 2020-11-16T22:32:43.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc061100 0x7f71dc0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71dc157a80 0x7f71dc0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:43.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc1727c0 con 0x7f71dc157a80 2020-11-16T22:32:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc173670 con 0x7f71dc15dfd0 2020-11-16T22:32:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc16ffd0 con 0x7f71dc061100 2020-11-16T22:32:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33988/0 (socket says 172.21.15.181:33988) 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 -- 172.21.15.181:0/2398396336 learned_addr learned my addr 172.21.15.181:0/2398396336 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71daffd700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc061100 0x7f71dc0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71da7fc700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71dc157a80 0x7f71dc0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 -- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc061100 msgr2=0x7f71dc0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc061100 0x7f71dc0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 -- 172.21.15.181:0/2398396336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71dc157a80 msgr2=0x7f71dc0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71dc157a80 0x7f71dc0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 -- 172.21.15.181:0/2398396336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc1729f0 con 0x7f71dc15dfd0 2020-11-16T22:32:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71db7fe700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f71d400ef50 tx=0x7f71d40091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:44.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d400cb40 con 0x7f71dc15dfd0 2020-11-16T22:32:44.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d400cca0 con 0x7f71dc15dfd0 2020-11-16T22:32:44.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc172c90 con 0x7f71dc15dfd0 2020-11-16T22:32:44.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40123f0 con 0x7f71dc15dfd0 2020-11-16T22:32:44.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc1731d0 con 0x7f71dc15dfd0 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc044f60 con 0x7f71dc15dfd0 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.725+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f71d4022070 con 0x7f71dc15dfd0 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.729+0000 7f71c2ffd700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 0x7f71b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.729+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f71d406e400 con 0x7f71dc15dfd0 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.729+0000 7f71daffd700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 0x7f71b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:44.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.729+0000 7f71daffd700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 0x7f71b8057520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f71d0006ef0 tx=0x7f71d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:44.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.733+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f71d4044660 con 0x7f71dc15dfd0 2020-11-16T22:32:44.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71dc0b13b0 con 0x7f71dc15dfd0 2020-11-16T22:32:44.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71c2ffd700 1 -- 172.21.15.181:0/2398396336 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4631 (secure 0 0 0) 0x7f71d4016020 con 0x7f71dc15dfd0 2020-11-16T22:32:44.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 msgr2=0x7f71b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 0x7f71b8057520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f71d0006ef0 tx=0x7f71d0015000).stop 2020-11-16T22:32:44.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 msgr2=0x7f71dc0b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f71d400ef50 tx=0x7f71d40091d0).stop 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 shutdown_connections 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f71b8055060 0x7f71b8057520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71dc15dfd0 0x7f71dc0b5b20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71dc061100 0x7f71dc0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 --2- 172.21.15.181:0/2398396336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71dc157a80 0x7f71dc0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 >> 172.21.15.181:0/2398396336 conn(0x7f71dc0cf630 msgr2=0x7f71dc0d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:44.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 shutdown_connections 2020-11-16T22:32:44.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:43.937+0000 7f71dbfff700 1 -- 172.21.15.181:0/2398396336 wait complete. 2020-11-16T22:32:44.536 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 80 2020-11-16T22:32:44.536 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:44.536 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6798,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:43.504871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:32:44.537 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 Processor -- start 2020-11-16T22:32:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 -- start start 2020-11-16T22:32:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 0x7f604c145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c061b50 con 0x7f604c0816e0 2020-11-16T22:32:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c0618b0 con 0x7f604c1492c0 2020-11-16T22:32:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0cb930 con 0x7f604c145a30 2020-11-16T22:32:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f6049d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f604c1492c0 0x7f604c080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48300/0 (socket says 172.21.15.181:48300) 2020-11-16T22:32:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 0x7f604c145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f6049d9b700 1 -- 172.21.15.181:0/2702283784 learned_addr learned my addr 172.21.15.181:0/2702283784 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513134871 0 0) 0x7f604c0618b0 con 0x7f604c1492c0 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6030003240 con 0x7f604c1492c0 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604a59c700 1 -- 172.21.15.181:0/2702283784 >> v1:172.21.15.181:6789/0 conn(0x7f604c1492c0 legacy=0x7f604c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503696071 0 0) 0x7f604c061b50 con 0x7f604c0816e0 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604a59c700 1 -- 172.21.15.181:0/2702283784 >> v1:172.21.15.175:6789/0 conn(0x7f604c0816e0 legacy=0x7f604c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604a59c700 1 -- 172.21.15.181:0/2702283784 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0cbbd0 con 0x7f604c145a30 2020-11-16T22:32:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604a59c700 1 --2- 172.21.15.181:0/2702283784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 0x7f604c145e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f603c009cc0 tx=0x7f603c0091d0).ready entity=mon.2 client_cookie=f4042418a7aec504 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00bad0 con 0x7f604c145a30 2020-11-16T22:32:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c00bc30 con 0x7f604c145a30 2020-11-16T22:32:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.881+0000 7f604959a700 1 -- 172.21.15.181:0/2702283784 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011400 con 0x7f604c145a30 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- 172.21.15.181:0/2702283784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 msgr2=0x7f604c145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 --2- 172.21.15.181:0/2702283784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 0x7f604c145e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f603c009cc0 tx=0x7f603c0091d0).stop 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- 172.21.15.181:0/2702283784 shutdown_connections 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 --2- 172.21.15.181:0/2702283784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c145a30 0x7f604c145e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- 172.21.15.181:0/2702283784 >> 172.21.15.181:0/2702283784 conn(0x7f604c0d68c0 msgr2=0x7f604c0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- 172.21.15.181:0/2702283784 shutdown_connections 2020-11-16T22:32:44.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- 172.21.15.181:0/2702283784 wait complete. 2020-11-16T22:32:44.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 Processor -- start 2020-11-16T22:32:44.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.885+0000 7f604b59e700 1 -- start start 2020-11-16T22:32:44.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0816e0 0x7f604c0d2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:44.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:44.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c1492c0 0x7f604c1707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:44.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0d17a0 con 0x7f604c0816e0 2020-11-16T22:32:45.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0d1a40 con 0x7f604c145a30 2020-11-16T22:32:45.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0d1ce0 con 0x7f604c1492c0 2020-11-16T22:32:45.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:45.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34002/0 (socket says 172.21.15.181:34002) 2020-11-16T22:32:45.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 -- 172.21.15.181:0/3780694220 learned_addr learned my addr 172.21.15.181:0/3780694220 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:45.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604ad9d700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c1492c0 0x7f604c1707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:45.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604a59c700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0816e0 0x7f604c0d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:45.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 -- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c1492c0 msgr2=0x7f604c1707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:45.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c1492c0 0x7f604c1707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 -- 172.21.15.181:0/3780694220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0816e0 msgr2=0x7f604c0d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:45.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0816e0 0x7f604c0d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 -- 172.21.15.181:0/3780694220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0d1f80 con 0x7f604c145a30 2020-11-16T22:32:45.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f6049d9b700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f604000c710 tx=0x7f6040007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:45.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400072e0 con 0x7f604c145a30 2020-11-16T22:32:45.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040007440 con 0x7f604c145a30 2020-11-16T22:32:45.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c0d2220 con 0x7f604c145a30 2020-11-16T22:32:45.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040005840 con 0x7f604c145a30 2020-11-16T22:32:45.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c0ce220 con 0x7f604c145a30 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c044f60 con 0x7f604c145a30 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.889+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6040010070 con 0x7f604c145a30 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.893+0000 7f603a7fc700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.893+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f604006d330 con 0x7f604c145a30 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.893+0000 7f604a59c700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.893+0000 7f604a59c700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 0x7f6020057520 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f603c009fd0 tx=0x7f603c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:45.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:44.897+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6040043590 con 0x7f604c145a30 2020-11-16T22:32:45.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.045+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f604c068e30 con 0x7f604c145a30 2020-11-16T22:32:45.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f603a7fc700 1 -- 172.21.15.181:0/3780694220 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f6040046bb0 con 0x7f604c145a30 2020-11-16T22:32:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 msgr2=0x7f6020057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 0x7f6020057520 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f603c009fd0 tx=0x7f603c016000).stop 2020-11-16T22:32:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 msgr2=0x7f604c15d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f604000c710 tx=0x7f6040007a20).stop 2020-11-16T22:32:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 shutdown_connections 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f604c1492c0 0x7f604c1707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0816e0 0x7f604c0d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 --2- 172.21.15.181:0/3780694220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f604c145a30 0x7f604c15d420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.049+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 >> 172.21.15.181:0/3780694220 conn(0x7f604c0d68c0 msgr2=0x7f604c15ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.053+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 shutdown_connections 2020-11-16T22:32:45.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:45.053+0000 7f604b59e700 1 -- 172.21.15.181:0/3780694220 wait complete. 2020-11-16T22:32:45.492 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:40.492654+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:46.103 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:32:46.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 Processor -- start 2020-11-16T22:32:46.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 -- start start 2020-11-16T22:32:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f382c145b00 0x7f382c145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 0x7f382c0d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c0d1cd0 con 0x7f382c061900 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c0cc970 con 0x7f382c0d13a0 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f3831d8b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c0609e0 con 0x7f382c145b00 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f382c145b00 0x7f382c145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f382c145b00 0x7f382c145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34008/0 (socket says 172.21.15.181:34008) 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 -- 172.21.15.181:0/4106203939 learned_addr learned my addr 172.21.15.181:0/4106203939 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382bfff700 1 --2- 172.21.15.181:0/4106203939 >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 0x7f382c0d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 -- 172.21.15.181:0/4106203939 >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 msgr2=0x7f382c0d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 --2- 172.21.15.181:0/4106203939 >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 0x7f382c0d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 -- 172.21.15.181:0/4106203939 >> v1:172.21.15.175:6789/0 conn(0x7f382c061900 legacy=0x7f382c0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 -- 172.21.15.181:0/4106203939 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c060c80 con 0x7f382c145b00 2020-11-16T22:32:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382bfff700 1 --2- 172.21.15.181:0/4106203939 >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 0x7f382c0d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.261+0000 7f382b7fe700 1 --2- 172.21.15.181:0/4106203939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c145b00 0x7f382c145ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f382c0d4510 tx=0x7f381c006eb0).ready entity=mon.1 client_cookie=d2ef983d942fb55 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:46.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f382a7fc700 1 -- 172.21.15.181:0/4106203939 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c009890 con 0x7f382c145b00 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f382a7fc700 1 -- 172.21.15.181:0/4106203939 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c0099f0 con 0x7f382c145b00 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f382a7fc700 1 -- 172.21.15.181:0/4106203939 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c009d70 con 0x7f382c145b00 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- 172.21.15.181:0/4106203939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c145b00 msgr2=0x7f382c145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 --2- 172.21.15.181:0/4106203939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c145b00 0x7f382c145ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f382c0d4510 tx=0x7f381c006eb0).stop 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- 172.21.15.181:0/4106203939 shutdown_connections 2020-11-16T22:32:46.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 --2- 172.21.15.181:0/4106203939 >> v2:172.21.15.175:3300/0 conn(0x7f382c0d13a0 0x7f382c0d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 --2- 172.21.15.181:0/4106203939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c145b00 0x7f382c145ee0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- 172.21.15.181:0/4106203939 >> 172.21.15.181:0/4106203939 conn(0x7f382c0d6a00 msgr2=0x7f382c0d6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- 172.21.15.181:0/4106203939 shutdown_connections 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- 172.21.15.181:0/4106203939 wait complete. 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 Processor -- start 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.265+0000 7f3831d8b700 1 -- start start 2020-11-16T22:32:46.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 0x7f382c13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f382c145b00 0x7f382c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c13d2e0 con 0x7f382c145b00 2020-11-16T22:32:46.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c13d580 con 0x7f382c061900 2020-11-16T22:32:46.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c13d820 con 0x7f382c0d13a0 2020-11-16T22:32:46.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34014/0 (socket says 172.21.15.181:34014) 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382b7fe700 1 -- 172.21.15.181:0/2656274954 learned_addr learned my addr 172.21.15.181:0/2656274954 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 0x7f382c13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382bfff700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f382c145b00 0x7f382c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 -- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 msgr2=0x7f382c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 -- 172.21.15.181:0/2656274954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f382c145b00 msgr2=0x7f382c13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f382c145b00 0x7f382c13ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 -- 172.21.15.181:0/2656274954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c142e40 con 0x7f382c0d13a0 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382b7fe700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f382affd700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 0x7f382c13c800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f382000c730 tx=0x7f3820007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000e490 con 0x7f382c0d13a0 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382000e5f0 con 0x7f382c0d13a0 2020-11-16T22:32:46.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000e990 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c1430e0 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c160440 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.269+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c1606c0 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.273+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3820010030 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.273+0000 7f380b7fe700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 0x7f3804057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.273+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f382006dc10 con 0x7f382c0d13a0 2020-11-16T22:32:46.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.273+0000 7f382b7fe700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 0x7f3804057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:46.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.273+0000 7f382b7fe700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 0x7f3804057550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f381c00b1b0 tx=0x7f381c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:46.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.277+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3820043d60 con 0x7f382c0d13a0 2020-11-16T22:32:46.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.485+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f382c069bd0 con 0x7f382c0d13a0 2020-11-16T22:32:46.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.493+0000 7f380b7fe700 1 -- 172.21.15.181:0/2656274954 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3820047380 con 0x7f382c0d13a0 2020-11-16T22:32:46.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 msgr2=0x7f3804057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:46.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 0x7f3804057550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f381c00b1b0 tx=0x7f381c015000).stop 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 msgr2=0x7f382c13c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 0x7f382c13c800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f382000c730 tx=0x7f3820007b70).stop 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 shutdown_connections 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3804055090 0x7f3804057550 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f382c145b00 0x7f382c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f382c061900 0x7f382c15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 --2- 172.21.15.181:0/2656274954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382c0d13a0 0x7f382c13c800 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:46.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 >> 172.21.15.181:0/2656274954 conn(0x7f382c0d6a00 msgr2=0x7f382c0d6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:46.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 shutdown_connections 2020-11-16T22:32:46.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:46.497+0000 7f3831d8b700 1 -- 172.21.15.181:0/2656274954 wait complete. 2020-11-16T22:32:46.872 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:32:46.873 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:47.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 Processor -- start 2020-11-16T22:32:47.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 -- start start 2020-11-16T22:32:47.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:47.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc0cccd0 con 0x7fcbbc061810 2020-11-16T22:32:47.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc0ccf70 con 0x7fcbbc156a90 2020-11-16T22:32:47.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1d82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15e030 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1581700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:47.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1581700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41868/0 (socket says 172.21.15.181:41868) 2020-11-16T22:32:47.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.221+0000 7fcbc1581700 1 -- 172.21.15.181:0/3566320529 learned_addr learned my addr 172.21.15.181:0/3566320529 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:47.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1581700 1 -- 172.21.15.181:0/3566320529 >> v1:172.21.15.181:6789/0 conn(0x7fcbbc156a90 legacy=0x7fcbbc061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1581700 1 -- 172.21.15.181:0/3566320529 >> v1:172.21.15.175:6789/0 conn(0x7fcbbc061810 legacy=0x7fcbbc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1581700 1 -- 172.21.15.181:0/3566320529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15e2d0 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1581700 1 --2- 172.21.15.181:0/3566320529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fcbb4009d80 tx=0x7fcbb4009290).ready entity=mon.2 client_cookie=7d1fe99f7fc7ef32 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbbb7fe700 1 -- 172.21.15.181:0/3566320529 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400bc80 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbbb7fe700 1 -- 172.21.15.181:0/3566320529 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb400bde0 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbbb7fe700 1 -- 172.21.15.181:0/3566320529 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4011610 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- 172.21.15.181:0/3566320529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 msgr2=0x7fcbbc0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/3566320529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fcbb4009d80 tx=0x7fcbb4009290).stop 2020-11-16T22:32:47.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- 172.21.15.181:0/3566320529 shutdown_connections 2020-11-16T22:32:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/3566320529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc15f3c0 0x7fcbbc0cc2c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- 172.21.15.181:0/3566320529 >> 172.21.15.181:0/3566320529 conn(0x7fcbbc0cf630 msgr2=0x7fcbbc0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- 172.21.15.181:0/3566320529 shutdown_connections 2020-11-16T22:32:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- 172.21.15.181:0/3566320529 wait complete. 2020-11-16T22:32:47.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 Processor -- start 2020-11-16T22:32:47.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.225+0000 7fcbc1d82700 1 -- start start 2020-11-16T22:32:47.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:47.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 0x7fcbbc0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:47.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbbc15f3c0 0x7fcbbc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:47.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15a790 con 0x7fcbbc156a90 2020-11-16T22:32:47.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15aa30 con 0x7fcbbc15f3c0 2020-11-16T22:32:47.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15acd0 con 0x7fcbbc061810 2020-11-16T22:32:47.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:47.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1581700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbbc15f3c0 0x7fcbbc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41870/0 (socket says 172.21.15.181:41870) 2020-11-16T22:32:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 -- 172.21.15.181:0/737413797 learned_addr learned my addr 172.21.15.181:0/737413797 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbbbfff700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 0x7fcbbc0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 -- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbbc15f3c0 msgr2=0x7fcbbc15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbbc15f3c0 0x7fcbbc15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 -- 172.21.15.181:0/737413797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 msgr2=0x7fcbbc0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 0x7fcbbc0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 -- 172.21.15.181:0/737413797 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15af70 con 0x7fcbbc061810 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbbbfff700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 0x7fcbbc0d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc0d80700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fcbac00c6e0 tx=0x7fcbac007c30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00e560 con 0x7fcbbc061810 2020-11-16T22:32:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac00e6c0 con 0x7fcbbc061810 2020-11-16T22:32:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc15b210 con 0x7fcbbc061810 2020-11-16T22:32:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00ea40 con 0x7fcbbc061810 2020-11-16T22:32:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc15be40 con 0x7fcbbc061810 2020-11-16T22:32:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.229+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc044f60 con 0x7fcbbc061810 2020-11-16T22:32:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.233+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcbac019030 con 0x7fcbbc061810 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.233+0000 7fcbb8ff9700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 0x7fcb94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.233+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcbac06d140 con 0x7fcbbc061810 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.233+0000 7fcbbbfff700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 0x7fcb94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.233+0000 7fcbbbfff700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 0x7fcb94057520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fcbb00010d0 tx=0x7fcbb0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.237+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcbac043450 con 0x7fcbbc061810 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.437+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbbc15b6b0 con 0x7fcbbc061810 2020-11-16T22:32:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbb8ff9700 1 -- 172.21.15.181:0/737413797 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4631 (secure 0 0 0) 0x7fcbac046a70 con 0x7fcbbc061810 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 msgr2=0x7fcb94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 0x7fcb94057520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fcbb00010d0 tx=0x7fcbb0009040).stop 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 msgr2=0x7fcbbc0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fcbac00c6e0 tx=0x7fcbac007c30).stop 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 shutdown_connections 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb94055060 0x7fcb94057520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbbc15f3c0 0x7fcbbc15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc061810 0x7fcbbc0d19c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 --2- 172.21.15.181:0/737413797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbbc156a90 0x7fcbbc0d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 >> 172.21.15.181:0/737413797 conn(0x7fcbbc0cf630 msgr2=0x7fcbbc060f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 shutdown_connections 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:47.441+0000 7fcbc1d82700 1 -- 172.21.15.181:0/737413797 wait complete. 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 80 2020-11-16T22:32:47.627 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:47.628 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6798,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:43.504871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:32:48.736 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:32:39.489360+0000', 'modified': '2020-11-16T22:32:43.504871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6946}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6946': {'gid': 6946, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6839/1430907052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1430907052}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1430907052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:32:48.736 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:32:48.737 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:32:48.880 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6946,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.3911648529999994,"mdsmap_epoch":80,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":14.747948619000001} 2020-11-16T22:32:48.881 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6946, "mdsmap_epoch": 80, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 9.391164853, "state": "up:active", "uptime": 14.747948619, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:32:48.881 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:32:48.882 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:32:48.882 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:32:48.882 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:32:48.882 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:32:48.883 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:32:48.883 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:32:48.883 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:32:48.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:48.884 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:32:48.903 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:32:48.903 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:32:48.903 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:32:48.903 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.903 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:32:48.904 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.905 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:32:48.906 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:32:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:32:48.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:48.908 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:32:48.939 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:32:48.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:48.943 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:32:49.002 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:32:49.002 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:32:49.002 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:32:49.002 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:32:49.002 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:32:49.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:49.003 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:32:49.004 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:32:49.004 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:32:49.004 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:32:49.005 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:32:49.005 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:32:49.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:49.106 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:32:49.118 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:32:49.119 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:32:49.119 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:32:49.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:49.120 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:32:49.197 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:32:49.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.337+0000 7f5564daf700 1 Processor -- start 2020-11-16T22:32:49.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.337+0000 7f5564daf700 1 -- start start 2020-11-16T22:32:49.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f556015f5b0 0x7f55600d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 0x7f55600629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5560160e40 con 0x7f55600615f0 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55601609a0 con 0x7f556015f5b0 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560160ba0 con 0x7f55600d1830 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f55600615f0 0x7f55600622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48342/0 (socket says 172.21.15.181:48342) 2020-11-16T22:32:49.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555dd9b700 1 -- 172.21.15.181:0/1520245028 learned_addr learned my addr 172.21.15.181:0/1520245028 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:49.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 --2- 172.21.15.181:0/1520245028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 0x7f55600629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555e59c700 1 --2- 172.21.15.181:0/1520245028 >> v2:172.21.15.175:3300/0 conn(0x7f556015f5b0 0x7f55600d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555d59a700 1 -- 172.21.15.181:0/1520245028 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152187835 0 0) 0x7f5560160e40 con 0x7f55600615f0 2020-11-16T22:32:49.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555d59a700 1 -- 172.21.15.181:0/1520245028 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5540003240 con 0x7f55600615f0 2020-11-16T22:32:49.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 -- 172.21.15.181:0/1520245028 >> v2:172.21.15.175:3300/0 conn(0x7f556015f5b0 msgr2=0x7f55600d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:49.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 --2- 172.21.15.181:0/1520245028 >> v2:172.21.15.175:3300/0 conn(0x7f556015f5b0 0x7f55600d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 -- 172.21.15.181:0/1520245028 >> v1:172.21.15.181:6789/0 conn(0x7f55600615f0 legacy=0x7f55600622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 -- 172.21.15.181:0/1520245028 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560060970 con 0x7f55600d1830 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555ed9d700 1 --2- 172.21.15.181:0/1520245028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 0x7f55600629b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f5558009c90 tx=0x7f55580091d0).ready entity=mon.2 client_cookie=d7bf899be0cc08e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555d59a700 1 -- 172.21.15.181:0/1520245028 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800b980 con 0x7f55600d1830 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555d59a700 1 -- 172.21.15.181:0/1520245028 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800bae0 con 0x7f55600d1830 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f555d59a700 1 -- 172.21.15.181:0/1520245028 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800be60 con 0x7f55600d1830 2020-11-16T22:32:49.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- 172.21.15.181:0/1520245028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 msgr2=0x7f55600629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 --2- 172.21.15.181:0/1520245028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 0x7f55600629b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f5558009c90 tx=0x7f55580091d0).stop 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- 172.21.15.181:0/1520245028 shutdown_connections 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 --2- 172.21.15.181:0/1520245028 >> v2:172.21.15.175:3300/0 conn(0x7f556015f5b0 0x7f55600d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 --2- 172.21.15.181:0/1520245028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55600d1830 0x7f55600629b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- 172.21.15.181:0/1520245028 >> 172.21.15.181:0/1520245028 conn(0x7f55600d69a0 msgr2=0x7f55600d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- 172.21.15.181:0/1520245028 shutdown_connections 2020-11-16T22:32:49.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.341+0000 7f5564daf700 1 -- 172.21.15.181:0/1520245028 wait complete. 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 Processor -- start 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 -- start start 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55600615f0 0x7f55600bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556015f5b0 0x7f556016fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55601700c0 con 0x7f55600615f0 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55600ba8f0 con 0x7f55600d1830 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55600bab90 con 0x7f556015f5b0 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34044/0 (socket says 172.21.15.181:34044) 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 -- 172.21.15.181:0/1901980668 learned_addr learned my addr 172.21.15.181:0/1901980668 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555ed9d700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556015f5b0 0x7f556016fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555e59c700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55600615f0 0x7f55600bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 -- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556015f5b0 msgr2=0x7f556016fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556015f5b0 0x7f556016fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 -- 172.21.15.181:0/1901980668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55600615f0 msgr2=0x7f55600bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55600615f0 0x7f55600bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 -- 172.21.15.181:0/1901980668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560170750 con 0x7f55600d1830 2020-11-16T22:32:49.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f555dd9b700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f555000c730 tx=0x7f5550007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:49.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55500072e0 con 0x7f55600d1830 2020-11-16T22:32:49.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5550007440 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55601709f0 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.345+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550005800 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55600baeb0 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5560170570 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5550010070 con 0x7f55600d1830 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f55567fc700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 0x7f553c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f555006d5c0 con 0x7f55600d1830 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f555e59c700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 0x7f553c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.349+0000 7f555e59c700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 0x7f553c057520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5548006fd0 tx=0x7f5548008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.353+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5550043820 con 0x7f55600d1830 2020-11-16T22:32:49.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.509+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5560069eb0 con 0x7f55600d1830 2020-11-16T22:32:49.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.509+0000 7f55567fc700 1 -- 172.21.15.181:0/1901980668 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f5550046e40 con 0x7f55600d1830 2020-11-16T22:32:49.498 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:49.498 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:40.492654+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:32:49.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 msgr2=0x7f553c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 0x7f553c057520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5548006fd0 tx=0x7f5548008040).stop 2020-11-16T22:32:49.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 msgr2=0x7f556016f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:49.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f555000c730 tx=0x7f5550007a80).stop 2020-11-16T22:32:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 shutdown_connections 2020-11-16T22:32:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f553c055060 0x7f553c057520 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556015f5b0 0x7f556016fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55600615f0 0x7f55600bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 --2- 172.21.15.181:0/1901980668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55600d1830 0x7f556016f6e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 >> 172.21.15.181:0/1901980668 conn(0x7f55600d69a0 msgr2=0x7f5560080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 shutdown_connections 2020-11-16T22:32:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:49.513+0000 7f5564daf700 1 -- 172.21.15.181:0/1901980668 wait complete. 2020-11-16T22:32:49.912 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:32:37.187776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:32:38.352981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:32:50.114 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:32:50.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.265+0000 7fa631f9e700 1 Processor -- start 2020-11-16T22:32:50.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.265+0000 7fa631f9e700 1 -- start start 2020-11-16T22:32:50.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa62c161cc0 0x7fa62c0637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:50.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c063ca0 con 0x7fa62c061810 2020-11-16T22:32:50.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c064970 con 0x7fa62c0633d0 2020-11-16T22:32:50.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c065930 con 0x7fa62c161cc0 2020-11-16T22:32:50.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa62c0633d0 0x7fa62c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48354/0 (socket says 172.21.15.181:48354) 2020-11-16T22:32:50.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62b7fe700 1 -- 172.21.15.181:0/3060853707 learned_addr learned my addr 172.21.15.181:0/3060853707 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:50.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62bfff700 1 --2- 172.21.15.181:0/3060853707 >> v2:172.21.15.181:3300/0 conn(0x7fa62c161cc0 0x7fa62c0637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:50.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62bfff700 1 -- 172.21.15.181:0/3060853707 >> v1:172.21.15.181:6789/0 conn(0x7fa62c0633d0 legacy=0x7fa62c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62bfff700 1 -- 172.21.15.181:0/3060853707 >> v1:172.21.15.175:6789/0 conn(0x7fa62c061810 legacy=0x7fa62c062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62bfff700 1 -- 172.21.15.181:0/3060853707 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c064c10 con 0x7fa62c161cc0 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62bfff700 1 --2- 172.21.15.181:0/3060853707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c161cc0 0x7fa62c0637b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fa624009c90 tx=0x7fa6240091d0).ready entity=mon.1 client_cookie=cb5e076a66fd0561 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62affd700 1 -- 172.21.15.181:0/3060853707 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62400bbc0 con 0x7fa62c161cc0 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62affd700 1 -- 172.21.15.181:0/3060853707 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62400bd20 con 0x7fa62c161cc0 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa62affd700 1 -- 172.21.15.181:0/3060853707 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624011610 con 0x7fa62c161cc0 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- 172.21.15.181:0/3060853707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c161cc0 msgr2=0x7fa62c0637b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 --2- 172.21.15.181:0/3060853707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c161cc0 0x7fa62c0637b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fa624009c90 tx=0x7fa6240091d0).stop 2020-11-16T22:32:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- 172.21.15.181:0/3060853707 shutdown_connections 2020-11-16T22:32:50.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 --2- 172.21.15.181:0/3060853707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c161cc0 0x7fa62c0637b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- 172.21.15.181:0/3060853707 >> 172.21.15.181:0/3060853707 conn(0x7fa62c0cf890 msgr2=0x7fa62c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:50.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- 172.21.15.181:0/3060853707 shutdown_connections 2020-11-16T22:32:50.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.269+0000 7fa631f9e700 1 -- 172.21.15.181:0/3060853707 wait complete. 2020-11-16T22:32:50.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 Processor -- start 2020-11-16T22:32:50.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 -- start start 2020-11-16T22:32:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 0x7fa62c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c0633d0 0x7fa62c1709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0b7210 con 0x7fa62c061810 2020-11-16T22:32:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0cc4f0 con 0x7fa62c0633d0 2020-11-16T22:32:50.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa631f9e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0cc790 con 0x7fa62c161cc0 2020-11-16T22:32:50.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa62bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:50.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa622ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c0633d0 0x7fa62c1709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:50.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa62bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41902/0 (socket says 172.21.15.181:41902) 2020-11-16T22:32:50.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa62bfff700 1 -- 172.21.15.181:0/387319248 learned_addr learned my addr 172.21.15.181:0/387319248 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:50.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.273+0000 7fa62b7fe700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 0x7fa62c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:50.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 -- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c0633d0 msgr2=0x7fa62c1709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c0633d0 0x7fa62c1709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 -- 172.21.15.181:0/387319248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 msgr2=0x7fa62c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 0x7fa62c0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 -- 172.21.15.181:0/387319248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0cca30 con 0x7fa62c161cc0 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62b7fe700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 0x7fa62c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:50.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62bfff700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fa624009210 tx=0x7fa624016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:50.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62400be90 con 0x7fa62c161cc0 2020-11-16T22:32:50.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624010b50 con 0x7fa62c161cc0 2020-11-16T22:32:50.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c0cccd0 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62401d480 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c0cb4b0 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c069eb0 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa624024030 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 0x7fa608057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa62406faa0 con 0x7fa62c161cc0 2020-11-16T22:32:50.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62b7fe700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 0x7fa608057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.277+0000 7fa62b7fe700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 0x7fa608057550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fa61c0011a0 tx=0x7fa61c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.289+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa624045e30 con 0x7fa62c161cc0 2020-11-16T22:32:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa62c05d4e0 con 0x7fa62c161cc0 2020-11-16T22:32:50.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa623fff700 1 -- 172.21.15.181:0/387319248 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v81) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa624022070 con 0x7fa62c161cc0 2020-11-16T22:32:50.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 msgr2=0x7fa608057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 0x7fa608057550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fa61c0011a0 tx=0x7fa61c009040).stop 2020-11-16T22:32:50.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 msgr2=0x7fa62c170ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:50.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fa624009210 tx=0x7fa624016000).stop 2020-11-16T22:32:50.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 shutdown_connections 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa608055090 0x7fa608057550 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa62c161cc0 0x7fa62c170ea0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c061810 0x7fa62c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 --2- 172.21.15.181:0/387319248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa62c0633d0 0x7fa62c1709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 >> 172.21.15.181:0/387319248 conn(0x7fa62c0cf890 msgr2=0x7fa62c0cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 shutdown_connections 2020-11-16T22:32:50.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:50.441+0000 7fa631f9e700 1 -- 172.21.15.181:0/387319248 wait complete. 2020-11-16T22:32:50.880 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:32:50.880 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.025+0000 7f6dc284a700 1 Processor -- start 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.025+0000 7f6dc284a700 1 -- start start 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6dbc1492c0 0x7f6dbc0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc061b50 con 0x7f6dbc145a30 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc149700 con 0x7f6dbc1492c0 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0618b0 con 0x7f6dbc0816e0 2020-11-16T22:32:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41908/0 (socket says 172.21.15.181:41908) 2020-11-16T22:32:51.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 -- 172.21.15.181:0/1486907748 learned_addr learned my addr 172.21.15.181:0/1486907748 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:51.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc2049700 1 --2- 172.21.15.181:0/1486907748 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc1492c0 0x7f6dbc0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 -- 172.21.15.181:0/1486907748 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc1492c0 msgr2=0x7f6dbc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 --2- 172.21.15.181:0/1486907748 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc1492c0 0x7f6dbc0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 -- 172.21.15.181:0/1486907748 >> v1:172.21.15.175:6789/0 conn(0x7f6dbc145a30 legacy=0x7f6dbc080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 -- 172.21.15.181:0/1486907748 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0cb930 con 0x7f6dbc0816e0 2020-11-16T22:32:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc1047700 1 --2- 172.21.15.181:0/1486907748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6db8006fd0 tx=0x7f6db8006eb0).ready entity=mon.2 client_cookie=7d83f20e4d032caf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc0846700 1 -- 172.21.15.181:0/1486907748 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8009810 con 0x7f6dbc0816e0 2020-11-16T22:32:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc0846700 1 -- 172.21.15.181:0/1486907748 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8009970 con 0x7f6dbc0816e0 2020-11-16T22:32:51.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc0846700 1 -- 172.21.15.181:0/1486907748 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8009cf0 con 0x7f6dbc0816e0 2020-11-16T22:32:51.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- 172.21.15.181:0/1486907748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 msgr2=0x7f6dbc145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 --2- 172.21.15.181:0/1486907748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6db8006fd0 tx=0x7f6db8006eb0).stop 2020-11-16T22:32:51.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- 172.21.15.181:0/1486907748 shutdown_connections 2020-11-16T22:32:51.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 --2- 172.21.15.181:0/1486907748 >> v2:172.21.15.175:3300/0 conn(0x7f6dbc1492c0 0x7f6dbc0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 --2- 172.21.15.181:0/1486907748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc0816e0 0x7f6dbc145e10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- 172.21.15.181:0/1486907748 >> 172.21.15.181:0/1486907748 conn(0x7f6dbc0d68c0 msgr2=0x7f6dbc0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:51.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- 172.21.15.181:0/1486907748 shutdown_connections 2020-11-16T22:32:51.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.029+0000 7f6dc284a700 1 -- 172.21.15.181:0/1486907748 wait complete. 2020-11-16T22:32:51.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.033+0000 7f6dc284a700 1 Processor -- start 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.033+0000 7f6dc284a700 1 -- start start 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 0x7f6dbc164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 0x7f6dbc13cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc1492c0 0x7f6dbc146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc164d30 con 0x7f6dbc145a30 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc164fd0 con 0x7f6dbc0816e0 2020-11-16T22:32:51.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc165270 con 0x7f6dbc1492c0 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1848700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 0x7f6dbc164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 0x7f6dbc13cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1848700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 0x7f6dbc164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34070/0 (socket says 172.21.15.181:34070) 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1848700 1 -- 172.21.15.181:0/2993172912 learned_addr learned my addr 172.21.15.181:0/2993172912 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc2049700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc1492c0 0x7f6dbc146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 -- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc1492c0 msgr2=0x7f6dbc146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc1492c0 0x7f6dbc146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 -- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 msgr2=0x7f6dbc164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 0x7f6dbc164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 -- 172.21.15.181:0/2993172912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc165510 con 0x7f6dbc145a30 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc1047700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 0x7f6dbc13cee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6db8006fd0 tx=0x7f6db8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8011c80 con 0x7f6dbc145a30 2020-11-16T22:32:51.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0d13e0 con 0x7f6dbc145a30 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8011de0 con 0x7f6dbc145a30 2020-11-16T22:32:51.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db801e670 con 0x7f6dbc145a30 2020-11-16T22:32:51.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.037+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc13d3d0 con 0x7f6dbc145a30 2020-11-16T22:32:51.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc0cc2f0 con 0x7f6dbc145a30 2020-11-16T22:32:51.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6db801a040 con 0x7f6dbc145a30 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6da9ffb700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 0x7f6d94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6db806ef30 con 0x7f6dbc145a30 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6dc1848700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 0x7f6d94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.041+0000 7f6dc1848700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 0x7f6d94057520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f6dac003380 tx=0x7f6dac000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.049+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6db8045240 con 0x7f6dbc145a30 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.200+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dbc0cc2f0 con 0x7f6dbc145a30 2020-11-16T22:32:51.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.200+0000 7f6da9ffb700 1 -- 172.21.15.181:0/2993172912 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dbc0cc2f0 con 0x7f6dbc145a30 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 msgr2=0x7f6d94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 0x7f6d94057520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f6dac003380 tx=0x7f6dac000b80).stop 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 msgr2=0x7f6dbc13cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 0x7f6dbc13cee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6db8006fd0 tx=0x7f6db8015000).stop 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 shutdown_connections 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6d94055060 0x7f6d94057520 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dbc1492c0 0x7f6dbc146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dbc0816e0 0x7f6dbc164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 --2- 172.21.15.181:0/2993172912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dbc145a30 0x7f6dbc13cee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 >> 172.21.15.181:0/2993172912 conn(0x7f6dbc0d68c0 msgr2=0x7f6dbc060ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 shutdown_connections 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.204+0000 7f6dc284a700 1 -- 172.21.15.181:0/2993172912 wait complete. 2020-11-16T22:32:51.717 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:51.718 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.864+0000 7f092d7fe700 1 Processor -- start 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.864+0000 7f092d7fe700 1 -- start start 2020-11-16T22:32:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 0x7f0928145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0928061b50 con 0x7f09280814c0 2020-11-16T22:32:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09280618b0 con 0x7f0928145a30 2020-11-16T22:32:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280cb930 con 0x7f09281492c0 2020-11-16T22:32:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f0927fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 0x7f0928145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41922/0 (socket says 172.21.15.181:41922) 2020-11-16T22:32:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 -- 172.21.15.181:0/2958270397 learned_addr learned my addr 172.21.15.181:0/2958270397 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:51.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 -- 172.21.15.181:0/2958270397 >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 msgr2=0x7f0928145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 --2- 172.21.15.181:0/2958270397 >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 0x7f0928145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 -- 172.21.15.181:0/2958270397 >> v1:172.21.15.181:6789/0 conn(0x7f09280814c0 legacy=0x7f09280cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f09277fe700 1 -- 172.21.15.181:0/2958270397 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674388186 0 0) 0x7f0928061b50 con 0x7f09280814c0 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 -- 172.21.15.181:0/2958270397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280cbbd0 con 0x7f09281492c0 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f0927fff700 1 --2- 172.21.15.181:0/2958270397 >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 0x7f0928145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f091ffff700 1 --2- 172.21.15.181:0/2958270397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f0910006fd0 tx=0x7f0910006eb0).ready entity=mon.2 client_cookie=f12e05cfc44ae7a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:51.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f09277fe700 1 -- 172.21.15.181:0/2958270397 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910009810 con 0x7f09281492c0 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f09277fe700 1 -- 172.21.15.181:0/2958270397 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0910009970 con 0x7f09281492c0 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f09277fe700 1 -- 172.21.15.181:0/2958270397 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910009cf0 con 0x7f09281492c0 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- 172.21.15.181:0/2958270397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 msgr2=0x7f0928080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2958270397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f0910006fd0 tx=0x7f0910006eb0).stop 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- 172.21.15.181:0/2958270397 shutdown_connections 2020-11-16T22:32:52.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2958270397 >> v2:172.21.15.181:3300/0 conn(0x7f0928145a30 0x7f0928145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2958270397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09281492c0 0x7f0928080fd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- 172.21.15.181:0/2958270397 >> 172.21.15.181:0/2958270397 conn(0x7f09280d68c0 msgr2=0x7f09280d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:52.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- 172.21.15.181:0/2958270397 shutdown_connections 2020-11-16T22:32:52.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.868+0000 7f092d7fe700 1 -- 172.21.15.181:0/2958270397 wait complete. 2020-11-16T22:32:52.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 Processor -- start 2020-11-16T22:32:52.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 -- start start 2020-11-16T22:32:52.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0928145a30 0x7f0928140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09281492c0 0x7f0928141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092816f250 con 0x7f09281492c0 2020-11-16T22:32:52.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928170160 con 0x7f09280814c0 2020-11-16T22:32:52.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092d7fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092816f440 con 0x7f0928145a30 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34084/0 (socket says 172.21.15.181:34084) 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 -- 172.21.15.181:0/2000556960 learned_addr learned my addr 172.21.15.181:0/2000556960 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f091ffff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0928145a30 0x7f0928140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f092cffd700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09281492c0 0x7f0928141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 -- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0928145a30 msgr2=0x7f0928140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0928145a30 0x7f0928140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 -- 172.21.15.181:0/2000556960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09281492c0 msgr2=0x7f0928141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09281492c0 0x7f0928141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 -- 172.21.15.181:0/2000556960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092816f6e0 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.872+0000 7f0927fff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f091800ef80 tx=0x7f0918006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800cbb0 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800cd10 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092816f980 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918012600 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092816fec0 con 0x7f09280814c0 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092815fab0 con 0x7f09280814c0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0918012970 con 0x7f09280814c0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 0x7f0904057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f091806e3e0 con 0x7f09280814c0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f091ffff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 0x7f0904057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.876+0000 7f091ffff700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 0x7f0904057520 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f091000b1b0 tx=0x7f0910015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:51.884+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0918044640 con 0x7f09280814c0 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0928044f60 con 0x7f09280814c0 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f0924ff9700 1 -- 172.21.15.181:0/2000556960 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0918047c60 con 0x7f09280814c0 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 msgr2=0x7f0904057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 0x7f0904057520 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f091000b1b0 tx=0x7f0910015000).stop 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 msgr2=0x7f09280d3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f091800ef80 tx=0x7f0918006df0).stop 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 shutdown_connections 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0904055060 0x7f0904057520 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09281492c0 0x7f0928141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09280814c0 0x7f09280d3200 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 --2- 172.21.15.181:0/2000556960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0928145a30 0x7f0928140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 >> 172.21.15.181:0/2000556960 conn(0x7f09280d68c0 msgr2=0x7f09280610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 shutdown_connections 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.032+0000 7f092d7fe700 1 -- 172.21.15.181:0/2000556960 wait complete. 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:32:52.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.632+0000 7fa1209ca700 1 Processor -- start 2020-11-16T22:32:52.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.632+0000 7fa1209ca700 1 -- start start 2020-11-16T22:32:52.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 0x7fa11c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa11c15d2e0 0x7fa11c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0c8910 con 0x7fa11c15d2e0 2020-11-16T22:32:52.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0b3460 con 0x7fa11c0b2900 2020-11-16T22:32:52.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0b31d0 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 0x7fa11c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41936/0 (socket says 172.21.15.181:41936) 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 0x7fa11c0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34092/0 (socket says 172.21.15.181:34092) 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 -- 172.21.15.181:0/3225966499 learned_addr learned my addr 172.21.15.181:0/3225966499 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11b59e700 1 --2- 172.21.15.181:0/3225966499 >> v2:172.21.15.175:3300/0 conn(0x7fa11c15d2e0 0x7fa11c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 -- 172.21.15.181:0/3225966499 >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 msgr2=0x7fa11c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:52.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 --2- 172.21.15.181:0/3225966499 >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 0x7fa11c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 -- 172.21.15.181:0/3225966499 >> v2:172.21.15.175:3300/0 conn(0x7fa11c15d2e0 msgr2=0x7fa11c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:52.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 --2- 172.21.15.181:0/3225966499 >> v2:172.21.15.175:3300/0 conn(0x7fa11c15d2e0 0x7fa11c0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 -- 172.21.15.181:0/3225966499 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0c8130 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa11a59c700 1 --2- 172.21.15.181:0/3225966499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fa11c066500 tx=0x7fa104009290).ready entity=mon.2 client_cookie=e985e5c2054a71c0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa119d9b700 1 -- 172.21.15.181:0/3225966499 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400bad0 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa119d9b700 1 -- 172.21.15.181:0/3225966499 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10400bc30 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa119d9b700 1 -- 172.21.15.181:0/3225966499 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104011400 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- 172.21.15.181:0/3225966499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 msgr2=0x7fa11c15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- 172.21.15.181:0/3225966499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fa11c066500 tx=0x7fa104009290).stop 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- 172.21.15.181:0/3225966499 shutdown_connections 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- 172.21.15.181:0/3225966499 >> v2:172.21.15.175:3300/0 conn(0x7fa11c15d2e0 0x7fa11c0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- 172.21.15.181:0/3225966499 >> v2:172.21.15.181:3300/0 conn(0x7fa11c0b2900 0x7fa11c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 --2- 172.21.15.181:0/3225966499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0cf6b0 0x7fa11c15cdf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- 172.21.15.181:0/3225966499 >> 172.21.15.181:0/3225966499 conn(0x7fa11c0d2dc0 msgr2=0x7fa11c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- 172.21.15.181:0/3225966499 shutdown_connections 2020-11-16T22:32:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.636+0000 7fa1209ca700 1 -- 172.21.15.181:0/3225966499 wait complete. 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 Processor -- start 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 -- start start 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa11c0cf6b0 0x7fa11c1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11c15d2e0 0x7fa11c061bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c061500 con 0x7fa11c15d2e0 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c16f600 con 0x7fa11c0cf6b0 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa1209ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c16f8a0 con 0x7fa11c0b2900 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41940/0 (socket says 172.21.15.181:41940) 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa11c0cf6b0 0x7fa11c1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 -- 172.21.15.181:0/1289483120 learned_addr learned my addr 172.21.15.181:0/1289483120 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11b59e700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11c15d2e0 0x7fa11c061bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 -- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa11c0cf6b0 msgr2=0x7fa11c1488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa11c0cf6b0 0x7fa11c1488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 -- 172.21.15.181:0/1289483120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11c15d2e0 msgr2=0x7fa11c061bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11c15d2e0 0x7fa11c061bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 -- 172.21.15.181:0/1289483120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c16fb40 con 0x7fa11c0b2900 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.640+0000 7fa11ad9d700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa10c00eba0 tx=0x7fa10c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00c9b0 con 0x7fa11c0b2900 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c00cb10 con 0x7fa11c0b2900 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c16fde0 con 0x7fa11c0b2900 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00ce90 con 0x7fa11c0b2900 2020-11-16T22:32:53.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c0cdb20 con 0x7fa11c0b2900 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c044f60 con 0x7fa11c0b2900 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa10c012600 con 0x7fa11c0b2900 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 0x7fa0ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa10c06e200 con 0x7fa11c0b2900 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa11a59c700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 0x7fa0ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.644+0000 7fa11a59c700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 0x7fa0ec057520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fa11c0b1570 tx=0x7fa104016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:53.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.652+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa10c044460 con 0x7fa11c0b2900 2020-11-16T22:32:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.800+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa11c161ec0 con 0x7fa11c0b2900 2020-11-16T22:32:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa112ffd700 1 -- 172.21.15.181:0/1289483120 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa10c047a80 con 0x7fa11c0b2900 2020-11-16T22:32:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 msgr2=0x7fa0ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 0x7fa0ec057520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fa11c0b1570 tx=0x7fa104016000).stop 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 msgr2=0x7fa11c16efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa10c00eba0 tx=0x7fa10c006df0).stop 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 shutdown_connections 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0ec055060 0x7fa0ec057520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa11c15d2e0 0x7fa11c061bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa11c0b2900 0x7fa11c16efe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 --2- 172.21.15.181:0/1289483120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa11c0cf6b0 0x7fa11c1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 >> 172.21.15.181:0/1289483120 conn(0x7fa11c0d2dc0 msgr2=0x7fa11c15cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 shutdown_connections 2020-11-16T22:32:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:52.804+0000 7fa1209ca700 1 -- 172.21.15.181:0/1289483120 wait complete. 2020-11-16T22:32:53.244 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:32:53.245 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:32:53.245 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 Processor -- start 2020-11-16T22:32:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- start start 2020-11-16T22:32:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa53c167ac0 0x7fa53c167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 0x7fa53c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c168390 con 0x7fa53c0b3220 2020-11-16T22:32:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c0b5f00 con 0x7fa53c1672c0 2020-11-16T22:32:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c167730 con 0x7fa53c167ac0 2020-11-16T22:32:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa53c167ac0 0x7fa53c167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa53c167ac0 0x7fa53c167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34106/0 (socket says 172.21.15.181:34106) 2020-11-16T22:32:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 -- 172.21.15.181:0/1070438437 learned_addr learned my addr 172.21.15.181:0/1070438437 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:53.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542899700 1 --2- 172.21.15.181:0/1070438437 >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 0x7fa53c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 -- 172.21.15.181:0/1070438437 >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 msgr2=0x7fa53c0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:53.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 --2- 172.21.15.181:0/1070438437 >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 0x7fa53c0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 -- 172.21.15.181:0/1070438437 >> v1:172.21.15.175:6789/0 conn(0x7fa53c0b3220 legacy=0x7fa53c166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 -- 172.21.15.181:0/1070438437 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0b2060 con 0x7fa53c167ac0 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542899700 1 --2- 172.21.15.181:0/1070438437 >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 0x7fa53c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa542098700 1 --2- 172.21.15.181:0/1070438437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c167ac0 0x7fa53c167ea0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fa53c066b60 tx=0x7fa52c009290).ready entity=mon.1 client_cookie=fa7eeb08daf95eaf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa541096700 1 -- 172.21.15.181:0/1070438437 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00bbc0 con 0x7fa53c167ac0 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa541096700 1 -- 172.21.15.181:0/1070438437 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00bd20 con 0x7fa53c167ac0 2020-11-16T22:32:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa541096700 1 -- 172.21.15.181:0/1070438437 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c011610 con 0x7fa53c167ac0 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- 172.21.15.181:0/1070438437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c167ac0 msgr2=0x7fa53c167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1070438437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c167ac0 0x7fa53c167ea0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fa53c066b60 tx=0x7fa52c009290).stop 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- 172.21.15.181:0/1070438437 shutdown_connections 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1070438437 >> v2:172.21.15.175:3300/0 conn(0x7fa53c1672c0 0x7fa53c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1070438437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c167ac0 0x7fa53c167ea0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- 172.21.15.181:0/1070438437 >> 172.21.15.181:0/1070438437 conn(0x7fa53c060bc0 msgr2=0x7fa53c0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- 172.21.15.181:0/1070438437 shutdown_connections 2020-11-16T22:32:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.392+0000 7fa5442fc700 1 -- 172.21.15.181:0/1070438437 wait complete. 2020-11-16T22:32:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 Processor -- start 2020-11-16T22:32:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 -- start start 2020-11-16T22:32:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa53c0b3220 0x7fa53c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 0x7fa53c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c061a30 con 0x7fa53c0b3220 2020-11-16T22:32:53.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c061cd0 con 0x7fa53c1672c0 2020-11-16T22:32:53.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c061f70 con 0x7fa53c15f610 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34112/0 (socket says 172.21.15.181:34112) 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 -- 172.21.15.181:0/1123734300 learned_addr learned my addr 172.21.15.181:0/1123734300 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa542899700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 0x7fa53c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa542098700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa53c0b3220 0x7fa53c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 -- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 msgr2=0x7fa53c14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 0x7fa53c14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 -- 172.21.15.181:0/1123734300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa53c0b3220 msgr2=0x7fa53c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa53c0b3220 0x7fa53c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 -- 172.21.15.181:0/1123734300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c062210 con 0x7fa53c1672c0 2020-11-16T22:32:53.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa542899700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 0x7fa53c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:53.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa541897700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa53000c710 tx=0x7fa530007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000e490 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53000e5f0 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.396+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c0624b0 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000e990 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c0d2980 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c069eb0 con 0x7fa53c1672c0 2020-11-16T22:32:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa530019030 con 0x7fa53c1672c0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa53a7fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 0x7fa514057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa53006d150 con 0x7fa53c1672c0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa542098700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 0x7fa514057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.400+0000 7fa542098700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 0x7fa514057520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fa53c05ce10 tx=0x7fa52c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.408+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa5300433b0 con 0x7fa53c1672c0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa53c061100 con 0x7fa53c1672c0 2020-11-16T22:32:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa53a7fc700 1 -- 172.21.15.181:0/1123734300 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4631 (secure 0 0 0) 0x7fa530017020 con 0x7fa53c1672c0 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 msgr2=0x7fa514057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 0x7fa514057520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fa53c05ce10 tx=0x7fa52c016000).stop 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 msgr2=0x7fa53c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.612+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa53000c710 tx=0x7fa530007c10).stop 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 shutdown_connections 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa514055060 0x7fa514057520 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa53c15f610 0x7fa53c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa53c0b3220 0x7fa53c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 --2- 172.21.15.181:0/1123734300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa53c1672c0 0x7fa53c14d210 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 >> 172.21.15.181:0/1123734300 conn(0x7fa53c060bc0 msgr2=0x7fa53c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 shutdown_connections 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:53.616+0000 7fa5442fc700 1 -- 172.21.15.181:0/1123734300 wait complete. 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 80 2020-11-16T22:32:53.800 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:53.801 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6798,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:43.504871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:32:54.708 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:32:39.489360+0000', 'modified': '2020-11-16T22:32:43.504871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6946}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6946': {'gid': 6946, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6839/1430907052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1430907052}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1430907052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:32:54.709 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:32:54.709 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:32:54.846 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6946,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.356552284999999,"mdsmap_epoch":80,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":20.713328798999999} 2020-11-16T22:32:54.846 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6946, "mdsmap_epoch": 80, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 15.356552285, "state": "up:active", "uptime": 20.713328799, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:32:54.847 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:32:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:54.848 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:32:54.865 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:32:54.865 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:32:54.982 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:32:54.982 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:32:55.009 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:32:55.009 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:32:55.019 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:32:55.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:32:55.020 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:32:55.050 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:32:55.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 Processor -- start 2020-11-16T22:32:55.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 -- start start 2020-11-16T22:32:55.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 0x7f16fc062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16fc062740 0x7f16fc0644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16fc0c8db0 con 0x7f16fc161480 2020-11-16T22:32:55.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc165030 con 0x7f16fc161860 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17029a1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0c8330 con 0x7f16fc062740 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f170199f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f16fc161480 0x7f16fc164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48424/0 (socket says 172.21.15.181:48424) 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f170199f700 1 -- 172.21.15.181:0/3078244076 learned_addr learned my addr 172.21.15.181:0/3078244076 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f17021a0700 1 --2- 172.21.15.181:0/3078244076 >> v2:172.21.15.181:3300/0 conn(0x7f16fc062740 0x7f16fc0644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.196+0000 7f170119e700 1 --2- 172.21.15.181:0/3078244076 >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 0x7f16fc062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737295104 0 0) 0x7f16fc0c8db0 con 0x7f16fc161480 2020-11-16T22:32:55.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16dc003240 con 0x7f16fc161480 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17021a0700 1 -- 172.21.15.181:0/3078244076 >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 msgr2=0x7f16fc062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17021a0700 1 --2- 172.21.15.181:0/3078244076 >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 0x7f16fc062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17021a0700 1 -- 172.21.15.181:0/3078244076 >> v1:172.21.15.181:6789/0 conn(0x7f16fc161480 legacy=0x7f16fc164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 906532354 0 0) 0x7f16dc003240 con 0x7f16fc161480 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17021a0700 1 -- 172.21.15.181:0/3078244076 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0c85d0 con 0x7f16fc062740 2020-11-16T22:32:55.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170119e700 1 --2- 172.21.15.181:0/3078244076 >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 0x7f16fc062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:55.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17021a0700 1 --2- 172.21.15.181:0/3078244076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc062740 0x7f16fc0644e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f16fc0b8580 tx=0x7f16ec0091d0).ready entity=mon.1 client_cookie=62b50d09ae034ebd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:55.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4001500 con 0x7f16fc062740 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec00b6b0 con 0x7f16fc062740 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f170099d700 1 -- 172.21.15.181:0/3078244076 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00ba30 con 0x7f16fc062740 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 -- 172.21.15.181:0/3078244076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc062740 msgr2=0x7f16fc0644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 --2- 172.21.15.181:0/3078244076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc062740 0x7f16fc0644e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f16fc0b8580 tx=0x7f16ec0091d0).stop 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 -- 172.21.15.181:0/3078244076 shutdown_connections 2020-11-16T22:32:55.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 --2- 172.21.15.181:0/3078244076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc062740 0x7f16fc0644e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 --2- 172.21.15.181:0/3078244076 >> v2:172.21.15.175:3300/0 conn(0x7f16fc161860 0x7f16fc062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 -- 172.21.15.181:0/3078244076 >> 172.21.15.181:0/3078244076 conn(0x7f16fc0d2dc0 msgr2=0x7f16fc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:55.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 -- 172.21.15.181:0/3078244076 shutdown_connections 2020-11-16T22:32:55.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.200+0000 7f17029a1700 1 -- 172.21.15.181:0/3078244076 wait complete. 2020-11-16T22:32:55.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 Processor -- start 2020-11-16T22:32:55.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 -- start start 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 0x7f16fc146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc145d50 con 0x7f16fc062740 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc145ff0 con 0x7f16fc161860 2020-11-16T22:32:55.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.204+0000 7f17029a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc146290 con 0x7f16fc161480 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170119e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170119e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:41970/0 (socket says 172.21.15.181:41970) 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34130/0 (socket says 172.21.15.181:34130) 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170119e700 1 -- 172.21.15.181:0/1088457497 learned_addr learned my addr 172.21.15.181:0/1088457497 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:55.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170199f700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 0x7f16fc146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 -- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 msgr2=0x7f16fc165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 -- 172.21.15.181:0/1088457497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 msgr2=0x7f16fc146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 0x7f16fc146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 -- 172.21.15.181:0/1088457497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc146530 con 0x7f16fc161860 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170119e700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f170199f700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 0x7f16fc146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17021a0700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f16fc0b8580 tx=0x7f16ec016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:55.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec01d450 con 0x7f16fc161860 2020-11-16T22:32:55.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec01d5b0 con 0x7f16fc161860 2020-11-16T22:32:55.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc1467d0 con 0x7f16fc161860 2020-11-16T22:32:55.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec01d970 con 0x7f16fc161860 2020-11-16T22:32:55.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc145a10 con 0x7f16fc161860 2020-11-16T22:32:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f16ec017040 con 0x7f16fc161860 2020-11-16T22:32:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc145ba0 con 0x7f16fc161860 2020-11-16T22:32:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.208+0000 7f16f9ffb700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 0x7f16d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.212+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f16ec06f9f0 con 0x7f16fc161860 2020-11-16T22:32:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.212+0000 7f170199f700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 0x7f16d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.212+0000 7f170199f700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 0x7f16d8057520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f16f40011c0 tx=0x7f16f4001e50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.216+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f16ec071e60 con 0x7f16fc161860 2020-11-16T22:32:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.420+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f16fc044f60 con 0x7f16fc161860 2020-11-16T22:32:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.752+0000 7f16f9ffb700 1 -- 172.21.15.181:0/1088457497 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v81) v1 ==== 105+0+0 (secure 0 0 0) 0x7f16ec021020 con 0x7f16fc161860 2020-11-16T22:32:55.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 msgr2=0x7f16d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 0x7f16d8057520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f16f40011c0 tx=0x7f16f4001e50).stop 2020-11-16T22:32:55.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 msgr2=0x7f16fc0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f16fc0b8580 tx=0x7f16ec016000).stop 2020-11-16T22:32:55.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 shutdown_connections 2020-11-16T22:32:55.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f16d8055060 0x7f16d8057520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16fc161860 0x7f16fc0ccba0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16fc062740 0x7f16fc146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 --2- 172.21.15.181:0/1088457497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16fc161480 0x7f16fc165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 >> 172.21.15.181:0/1088457497 conn(0x7f16fc0d2dc0 msgr2=0x7f16fc0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 shutdown_connections 2020-11-16T22:32:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.756+0000 7f17029a1700 1 -- 172.21.15.181:0/1088457497 wait complete. 2020-11-16T22:32:55.772 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:55.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 Processor -- start 2020-11-16T22:32:55.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- start start 2020-11-16T22:32:55.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d3c0b3220 0x7f3d3c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d3c166bb0 0x7f3d3c167060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c0b5f00 con 0x7f3d3c0b3220 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c0b3b90 con 0x7f3d3c166bb0 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c168390 con 0x7f3d3c167ac0 2020-11-16T22:32:55.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d3c166bb0 0x7f3d3c167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d3bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d3c166bb0 0x7f3d3c167060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34148/0 (socket says 172.21.15.181:34148) 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 -- 172.21.15.181:0/2770332965 learned_addr learned my addr 172.21.15.181:0/2770332965 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d33fff700 1 --2- 172.21.15.181:0/2770332965 >> v2:172.21.15.175:3300/0 conn(0x7f3d3c0b3220 0x7f3d3c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 -- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 msgr2=0x7f3d3c167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 -- 172.21.15.181:0/2770332965 >> v2:172.21.15.175:3300/0 conn(0x7f3d3c0b3220 msgr2=0x7f3d3c0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 --2- 172.21.15.181:0/2770332965 >> v2:172.21.15.175:3300/0 conn(0x7f3d3c0b3220 0x7f3d3c0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 -- 172.21.15.181:0/2770332965 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c0b2060 con 0x7f3d3c166bb0 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d3bfff700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d40c22700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c167060 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f3d3c168990 tx=0x7f3d34009290).ready entity=mon.1 client_cookie=9cdcc452504a6a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d3b7fe700 1 -- 172.21.15.181:0/2770332965 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400bb10 con 0x7f3d3c166bb0 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d3b7fe700 1 -- 172.21.15.181:0/2770332965 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3400bc70 con 0x7f3d3c166bb0 2020-11-16T22:32:55.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d3b7fe700 1 -- 172.21.15.181:0/2770332965 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34011400 con 0x7f3d3c166bb0 2020-11-16T22:32:55.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 msgr2=0x7f3d3c167060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:55.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c167060 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f3d3c168990 tx=0x7f3d34009290).stop 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- 172.21.15.181:0/2770332965 shutdown_connections 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c167060 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- 172.21.15.181:0/2770332965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 --2- 172.21.15.181:0/2770332965 >> v2:172.21.15.175:3300/0 conn(0x7f3d3c0b3220 0x7f3d3c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- 172.21.15.181:0/2770332965 >> 172.21.15.181:0/2770332965 conn(0x7f3d3c060bc0 msgr2=0x7f3d3c0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:55.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- 172.21.15.181:0/2770332965 shutdown_connections 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.900+0000 7f3d42685700 1 -- 172.21.15.181:0/2770332965 wait complete. 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 Processor -- start 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- start start 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 0x7f3d3c0d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c16ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:55.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c16b810 con 0x7f3d3c0b3220 2020-11-16T22:32:55.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c16bab0 con 0x7f3d3c166bb0 2020-11-16T22:32:55.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c1748f0 con 0x7f3d3c167ac0 2020-11-16T22:32:56.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:56.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34150/0 (socket says 172.21.15.181:34150) 2020-11-16T22:32:56.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 -- 172.21.15.181:0/3185588246 learned_addr learned my addr 172.21.15.181:0/3185588246 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:56.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d40c22700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c16ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:56.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d3bfff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 0x7f3d3c0d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 -- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 msgr2=0x7f3d3c16ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c16ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 -- 172.21.15.181:0/3185588246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 msgr2=0x7f3d3c0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 0x7f3d3c0d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 -- 172.21.15.181:0/3185588246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c174b90 con 0x7f3d3c166bb0 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d3bfff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 0x7f3d3c0d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:32:56.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d33fff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f3d2400ebd0 tx=0x7f3d24006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:56.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400b8a0 con 0x7f3d3c166bb0 2020-11-16T22:32:56.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2400ba00 con 0x7f3d3c166bb0 2020-11-16T22:32:56.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c174e30 con 0x7f3d3c166bb0 2020-11-16T22:32:56.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400bd80 con 0x7f3d3c166bb0 2020-11-16T22:32:56.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.904+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c16b1f0 con 0x7f3d3c166bb0 2020-11-16T22:32:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3c069eb0 con 0x7f3d3c166bb0 2020-11-16T22:32:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3d2401a030 con 0x7f3d3c166bb0 2020-11-16T22:32:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d38ff9700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 0x7f3d18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d3bfff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 0x7f3d18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3d2406d9a0 con 0x7f3d3c166bb0 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.908+0000 7f3d3bfff700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 0x7f3d18057520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3d3c1630a0 tx=0x7f3d2c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:55.912+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3d24043cb0 con 0x7f3d3c166bb0 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d3c061100 con 0x7f3d3c166bb0 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d38ff9700 1 -- 172.21.15.181:0/3185588246 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4647 (secure 0 0 0) 0x7f3d24018030 con 0x7f3d3c166bb0 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 msgr2=0x7f3d18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 0x7f3d18057520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3d3c1630a0 tx=0x7f3d2c008040).stop 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 msgr2=0x7f3d3c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f3d2400ebd0 tx=0x7f3d24006df0).stop 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 shutdown_connections 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3d18055060 0x7f3d18057520 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d3c167ac0 0x7f3d3c16ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d3c0b3220 0x7f3d3c0d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 --2- 172.21.15.181:0/3185588246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d3c166bb0 0x7f3d3c0d0bc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 >> 172.21.15.181:0/3185588246 conn(0x7f3d3c060bc0 msgr2=0x7f3d3c067090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 shutdown_connections 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:56.116+0000 7f3d42685700 1 -- 172.21.15.181:0/3185588246 wait complete. 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 82 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:56.308 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":82,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6798,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:55.752858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6946,"mds_1":6943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:creating","state_seq":1,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:32:57.419 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:32:39.489360+0000', 'modified': '2020-11-16T22:32:55.752858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6946, 'mds_1': 6943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6943': {'gid': 6943, 'name': 'd', 'rank': 1, 'incarnation': 82, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.181:6837/1009245241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1009245241}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1009245241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6946': {'gid': 6946, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6839/1430907052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1430907052}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1430907052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:32:57.420 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6943:up:creating 2020-11-16T22:32:58.421 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:32:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 Processor -- start 2020-11-16T22:32:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 -- start start 2020-11-16T22:32:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb520168390 con 0x7fb5201672c0 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5200b2060 con 0x7fb5200b3220 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb520167770 con 0x7fb520167ac0 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42004/0 (socket says 172.21.15.181:42004) 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 -- 172.21.15.181:0/3247540326 learned_addr learned my addr 172.21.15.181:0/3247540326 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:58.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51effd700 1 --1- 172.21.15.181:0/3247540326 >> v1:172.21.15.181:6789/0 conn(0x7fb5200b3220 0x7fb520166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48468/0 (socket says 172.21.15.181:48468) 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 -- 172.21.15.181:0/3247540326 >> v1:172.21.15.181:6789/0 conn(0x7fb5200b3220 legacy=0x7fb520166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 -- 172.21.15.181:0/3247540326 >> v1:172.21.15.175:6789/0 conn(0x7fb5201672c0 legacy=0x7fb520161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 -- 172.21.15.181:0/3247540326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200b2300 con 0x7fb520167ac0 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51f7fe700 1 --2- 172.21.15.181:0/3247540326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb50800ba20 tx=0x7fb508006eb0).ready entity=mon.2 client_cookie=be714b15062e439b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51e7fc700 1 -- 172.21.15.181:0/3247540326 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5080098d0 con 0x7fb520167ac0 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51e7fc700 1 -- 172.21.15.181:0/3247540326 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb508009a30 con 0x7fb520167ac0 2020-11-16T22:32:58.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb51e7fc700 1 -- 172.21.15.181:0/3247540326 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508009db0 con 0x7fb520167ac0 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.564+0000 7fb525f60700 1 -- 172.21.15.181:0/3247540326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 msgr2=0x7fb520167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 --2- 172.21.15.181:0/3247540326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb50800ba20 tx=0x7fb508006eb0).stop 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 -- 172.21.15.181:0/3247540326 shutdown_connections 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 --2- 172.21.15.181:0/3247540326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb520167ac0 0x7fb520167ea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 -- 172.21.15.181:0/3247540326 >> 172.21.15.181:0/3247540326 conn(0x7fb520060bc0 msgr2=0x7fb5200d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 -- 172.21.15.181:0/3247540326 shutdown_connections 2020-11-16T22:32:58.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 -- 172.21.15.181:0/3247540326 wait complete. 2020-11-16T22:32:58.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 Processor -- start 2020-11-16T22:32:58.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.568+0000 7fb525f60700 1 -- start start 2020-11-16T22:32:58.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5200b3220 0x7fb5201714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:58.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:58.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb520167ac0 0x7fb5200d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:58.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520175290 con 0x7fb5200b3220 2020-11-16T22:32:58.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520175530 con 0x7fb520167ac0 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201757d0 con 0x7fb5201672c0 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb520167ac0 0x7fb5200d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42012/0 (socket says 172.21.15.181:42012) 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 -- 172.21.15.181:0/1018827620 learned_addr learned my addr 172.21.15.181:0/1018827620 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:32:58.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51f7fe700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5200b3220 0x7fb5201714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:58.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 -- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb520167ac0 msgr2=0x7fb5200d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb520167ac0 0x7fb5200d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 -- 172.21.15.181:0/1018827620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5200b3220 msgr2=0x7fb5201714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5200b3220 0x7fb5201714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 -- 172.21.15.181:0/1018827620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520175a70 con 0x7fb5201672c0 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb51effd700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fb51000ca90 tx=0x7fb510007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510005ac0 con 0x7fb5201672c0 2020-11-16T22:32:58.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510005c20 con 0x7fb5201672c0 2020-11-16T22:32:58.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb520175d10 con 0x7fb5201672c0 2020-11-16T22:32:58.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510007110 con 0x7fb5201672c0 2020-11-16T22:32:58.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.572+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb520176940 con 0x7fb5201672c0 2020-11-16T22:32:58.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520176b90 con 0x7fb5201672c0 2020-11-16T22:32:58.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb510010030 con 0x7fb5201672c0 2020-11-16T22:32:58.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb5177fe700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 0x7fb4f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:32:58.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb51006dc30 con 0x7fb5201672c0 2020-11-16T22:32:58.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb51f7fe700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 0x7fb4f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:32:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.576+0000 7fb51f7fe700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 0x7fb4f0057520 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fb50800b1b0 tx=0x7fb508015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:32:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.580+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb510043e90 con 0x7fb5201672c0 2020-11-16T22:32:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.784+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb520069eb0 con 0x7fb5201672c0 2020-11-16T22:32:58.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.784+0000 7fb5177fe700 1 -- 172.21.15.181:0/1018827620 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fb5100474b0 con 0x7fb5201672c0 2020-11-16T22:32:58.774 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:32:58.774 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:57.768567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:32:58.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 msgr2=0x7fb4f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 0x7fb4f0057520 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fb50800b1b0 tx=0x7fb508015000).stop 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 msgr2=0x7fb520176450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fb51000ca90 tx=0x7fb510007f90).stop 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 shutdown_connections 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4f0055060 0x7fb4f0057520 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb520167ac0 0x7fb5200d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5200b3220 0x7fb5201714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 --2- 172.21.15.181:0/1018827620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5201672c0 0x7fb520176450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:32:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 >> 172.21.15.181:0/1018827620 conn(0x7fb520060bc0 msgr2=0x7fb5200d2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:32:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 shutdown_connections 2020-11-16T22:32:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:32:58.788+0000 7fb525f60700 1 -- 172.21.15.181:0/1018827620 wait complete. 2020-11-16T22:32:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2020-11-16T22:32:59.700 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:32:39.489360+0000', 'modified': '2020-11-16T22:32:57.768567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6946, 'mds_1': 6943, 'mds_2': 6798}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6798': {'gid': 6798, 'name': 'h', 'rank': 2, 'incarnation': 84, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6841/2661622280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6840', 'nonce': 2661622280}, {'type': 'v1', 'addr': '172.21.15.175:6841', 'nonce': 2661622280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6943': {'gid': 6943, 'name': 'd', 'rank': 1, 'incarnation': 82, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.181:6837/1009245241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1009245241}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1009245241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6946': {'gid': 6946, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6839/1430907052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1430907052}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1430907052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:32:59.701 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:32:59.701 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok --format=json status 2020-11-16T22:32:59.837 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":6798,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.0731468689999999,"mdsmap_epoch":85,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":25.746100550000001} 2020-11-16T22:32:59.837 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6798, "mdsmap_epoch": 85, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 3.073146869, "state": "up:active", "uptime": 25.74610055, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:32:59.838 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:32:59.975 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":6943,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.2172013799999997,"mdsmap_epoch":85,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":25.853492691} 2020-11-16T22:32:59.975 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6943, "mdsmap_epoch": 85, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 4.21720138, "state": "up:active", "uptime": 25.853492691, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:32:59.976 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:33:00.101 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":6946,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.612959638,"mdsmap_epoch":85,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":25.969729969999999} 2020-11-16T22:33:00.102 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 6946, "mdsmap_epoch": 85, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 20.612959638, "state": "up:active", "uptime": 25.96972997, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:33:00.102 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:00.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.236+0000 7fce8e2a7700 1 Processor -- start 2020-11-16T22:33:00.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.236+0000 7fce8e2a7700 1 -- start start 2020-11-16T22:33:00.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 0x7fce8815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce880cccd0 con 0x7fce88156a90 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce880ccf70 con 0x7fce8815f1a0 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8815e030 con 0x7fce88061810 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8caa4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 0x7fce8815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8daa6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fce8815f1a0 0x7fce880cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48486/0 (socket says 172.21.15.181:48486) 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8daa6700 1 -- 172.21.15.181:0/4257490649 learned_addr learned my addr 172.21.15.181:0/4257490649 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:00.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce87fff700 1 -- 172.21.15.181:0/4257490649 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51324714 0 0) 0x7fce880ccf70 con 0x7fce8815f1a0 2020-11-16T22:33:00.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce87fff700 1 -- 172.21.15.181:0/4257490649 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6c003240 con 0x7fce8815f1a0 2020-11-16T22:33:00.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8caa4700 1 -- 172.21.15.181:0/4257490649 >> v1:172.21.15.181:6789/0 conn(0x7fce8815f1a0 legacy=0x7fce880cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8caa4700 1 -- 172.21.15.181:0/4257490649 >> v1:172.21.15.175:6789/0 conn(0x7fce88156a90 legacy=0x7fce88061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8caa4700 1 -- 172.21.15.181:0/4257490649 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8815e2d0 con 0x7fce88061810 2020-11-16T22:33:00.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8caa4700 1 --2- 172.21.15.181:0/4257490649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 0x7fce8815ecb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fce7400b9d0 tx=0x7fce74006df0).ready entity=mon.2 client_cookie=5fe2093be211c84b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:00.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce87fff700 1 -- 172.21.15.181:0/4257490649 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740097b0 con 0x7fce88061810 2020-11-16T22:33:00.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce87fff700 1 -- 172.21.15.181:0/4257490649 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74009910 con 0x7fce88061810 2020-11-16T22:33:00.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce87fff700 1 -- 172.21.15.181:0/4257490649 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74009c90 con 0x7fce88061810 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- 172.21.15.181:0/4257490649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 msgr2=0x7fce8815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/4257490649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 0x7fce8815ecb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fce7400b9d0 tx=0x7fce74006df0).stop 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- 172.21.15.181:0/4257490649 shutdown_connections 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/4257490649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88061810 0x7fce8815ecb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- 172.21.15.181:0/4257490649 >> 172.21.15.181:0/4257490649 conn(0x7fce880cf630 msgr2=0x7fce880cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- 172.21.15.181:0/4257490649 shutdown_connections 2020-11-16T22:33:00.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.240+0000 7fce8e2a7700 1 -- 172.21.15.181:0/4257490649 wait complete. 2020-11-16T22:33:00.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 Processor -- start 2020-11-16T22:33:00.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- start start 2020-11-16T22:33:00.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce88061810 0x7fce880c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:00.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 0x7fce8814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880c8950 con 0x7fce8815f1a0 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8815a760 con 0x7fce88061810 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8815aa00 con 0x7fce88156a90 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42028/0 (socket says 172.21.15.181:42028) 2020-11-16T22:33:00.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 -- 172.21.15.181:0/576229089 learned_addr learned my addr 172.21.15.181:0/576229089 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8d2a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce88061810 0x7fce880c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8daa6700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 0x7fce8814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 -- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce88061810 msgr2=0x7fce880c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce88061810 0x7fce880c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 -- 172.21.15.181:0/576229089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 msgr2=0x7fce8814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 0x7fce8814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 -- 172.21.15.181:0/576229089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8815aca0 con 0x7fce88156a90 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8daa6700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 0x7fce8814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8caa4700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fce740098e0 tx=0x7fce74015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74011d50 con 0x7fce88156a90 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7401e460 con 0x7fce88156a90 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7401e800 con 0x7fce88156a90 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8815af40 con 0x7fce88156a90 2020-11-16T22:33:00.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.244+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8815bb70 con 0x7fce88156a90 2020-11-16T22:33:00.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce88044f60 con 0x7fce88156a90 2020-11-16T22:33:00.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce7401a040 con 0x7fce88156a90 2020-11-16T22:33:00.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce857fa700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 0x7fce60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:00.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce7406f750 con 0x7fce88156a90 2020-11-16T22:33:00.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce8d2a5700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 0x7fce60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:00.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.248+0000 7fce8d2a5700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 0x7fce60057520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fce880cf1b0 tx=0x7fce7c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:00.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.252+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fce74045950 con 0x7fce88156a90 2020-11-16T22:33:00.330 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:00.336 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:00.338 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:00.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.452+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce8815b3e0 con 0x7fce88156a90 2020-11-16T22:33:00.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.452+0000 7fce857fa700 1 -- 172.21.15.181:0/576229089 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fce74023090 con 0x7fce88156a90 2020-11-16T22:33:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:00.442 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:32:57.768567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:00.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 msgr2=0x7fce60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 0x7fce60057520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fce880cf1b0 tx=0x7fce7c009040).stop 2020-11-16T22:33:00.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 msgr2=0x7fce8815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:00.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fce740098e0 tx=0x7fce74015000).stop 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 shutdown_connections 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fce60055060 0x7fce60057520 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce8815f1a0 0x7fce8814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce88061810 0x7fce880c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 --2- 172.21.15.181:0/576229089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce88156a90 0x7fce8815b680 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 >> 172.21.15.181:0/576229089 conn(0x7fce880cf630 msgr2=0x7fce88060f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:00.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 shutdown_connections 2020-11-16T22:33:00.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:00.456+0000 7fce8e2a7700 1 -- 172.21.15.181:0/576229089 wait complete. 2020-11-16T22:33:00.446 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2020-11-16T22:33:00.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:00.457 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:33:00.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:00.521 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:33:04.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:04.145 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:33:04.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:04.182 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:33:04.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:04.218 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:33:04.258 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:04.395 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:33:04.396 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:04.396 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:06.398 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:06.534 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:33:06.535 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:06.535 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:08.537 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:08.677 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:33:08.677 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:08.678 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:10.679 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:10.815 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:33:10.816 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:10.816 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:12.818 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:12.954 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:33:12.955 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:12.955 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:14.956 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:33:15.096 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:33:15.097 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:33:15.098 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:33:15.099 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:33:15.220 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:33:15.221 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:33:15.221 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:33:17.223 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:33:17.360 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:33:17.361 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:33:17.362 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:33:19.364 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:33:19.500 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:33:19.501 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:33:19.501 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:33:21.503 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:33:21.639 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:33:21.640 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:33:21.641 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:33:21.642 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok --format=json get subtrees 2020-11-16T22:33:21.774 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:33:21.775 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:33:21.776 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:33:21.776 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:21.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 Processor -- start 2020-11-16T22:33:21.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 -- start start 2020-11-16T22:33:21.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5d8161480 0x7fa5d8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d80c8db0 con 0x7fa5d8161860 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d80c8330 con 0x7fa5d8161480 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5dd26a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80c85d0 con 0x7fa5d8164e10 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:21.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42034/0 (socket says 172.21.15.181:42034) 2020-11-16T22:33:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 -- 172.21.15.181:0/1524784980 learned_addr learned my addr 172.21.15.181:0/1524784980 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:21.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5d7fff700 1 --2- 172.21.15.181:0/1524784980 >> v2:172.21.15.175:3300/0 conn(0x7fa5d8161480 0x7fa5d8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:21.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 -- 172.21.15.181:0/1524784980 >> v2:172.21.15.175:3300/0 conn(0x7fa5d8161480 msgr2=0x7fa5d8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:21.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 --2- 172.21.15.181:0/1524784980 >> v2:172.21.15.175:3300/0 conn(0x7fa5d8161480 0x7fa5d8164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:21.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 -- 172.21.15.181:0/1524784980 >> v1:172.21.15.175:6789/0 conn(0x7fa5d8161860 legacy=0x7fa5d8062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5d77fe700 1 -- 172.21.15.181:0/1524784980 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729164984 0 0) 0x7fa5d80c8db0 con 0x7fa5d8161860 2020-11-16T22:33:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 -- 172.21.15.181:0/1524784980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80c8870 con 0x7fa5d8164e10 2020-11-16T22:33:21.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5cf7fe700 1 --2- 172.21.15.181:0/1524784980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fa5d80c8260 tx=0x7fa5c0006eb0).ready entity=mon.2 client_cookie=dca0679c5b667b89 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5d77fe700 1 -- 172.21.15.181:0/1524784980 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0009810 con 0x7fa5d8164e10 2020-11-16T22:33:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5d77fe700 1 -- 172.21.15.181:0/1524784980 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0009970 con 0x7fa5d8164e10 2020-11-16T22:33:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.916+0000 7fa5d77fe700 1 -- 172.21.15.181:0/1524784980 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0009cf0 con 0x7fa5d8164e10 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- 172.21.15.181:0/1524784980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 msgr2=0x7fa5d8062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/1524784980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fa5d80c8260 tx=0x7fa5c0006eb0).stop 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- 172.21.15.181:0/1524784980 shutdown_connections 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/1524784980 >> v2:172.21.15.175:3300/0 conn(0x7fa5d8161480 0x7fa5d8164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/1524784980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8164e10 0x7fa5d8062250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- 172.21.15.181:0/1524784980 >> 172.21.15.181:0/1524784980 conn(0x7fa5d80d2dc0 msgr2=0x7fa5d80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- 172.21.15.181:0/1524784980 shutdown_connections 2020-11-16T22:33:22.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- 172.21.15.181:0/1524784980 wait complete. 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 Processor -- start 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.920+0000 7fa5dd26a700 1 -- start start 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5d8161860 0x7fa5d8160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5d8160c40 0x7fa5d80621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8144570 con 0x7fa5d8161860 2020-11-16T22:33:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d81431a0 con 0x7fa5d8160c40 2020-11-16T22:33:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8146810 con 0x7fa5d8161480 2020-11-16T22:33:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42040/0 (socket says 172.21.15.181:42040) 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 -- 172.21.15.181:0/3344524029 learned_addr learned my addr 172.21.15.181:0/3344524029 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5cf7fe700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5d8161860 0x7fa5d8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dca69700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5d8160c40 0x7fa5d80621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 -- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5d8160c40 msgr2=0x7fa5d80621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5d8160c40 0x7fa5d80621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 -- 172.21.15.181:0/3344524029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5d8161860 msgr2=0x7fa5d8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:22.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5d8161860 0x7fa5d8160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 -- 172.21.15.181:0/3344524029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8146ab0 con 0x7fa5d8161480 2020-11-16T22:33:22.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d7fff700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa5c800eba0 tx=0x7fa5c8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:22.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800c8e0 con 0x7fa5d8161480 2020-11-16T22:33:22.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c800ca40 con 0x7fa5d8161480 2020-11-16T22:33:22.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d8146d50 con 0x7fa5d8161480 2020-11-16T22:33:22.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800cdc0 con 0x7fa5d8161480 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d8144a00 con 0x7fa5d8161480 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.924+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d8144bc0 con 0x7fa5d8161480 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.928+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa5c8012600 con 0x7fa5d8161480 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.928+0000 7fa5d4ff9700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 0x7fa5b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.928+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5c806e150 con 0x7fa5d8161480 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.928+0000 7fa5cf7fe700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 0x7fa5b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:22.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.928+0000 7fa5cf7fe700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 0x7fa5b4057520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fa5c0006ef0 tx=0x7fa5c0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:22.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:21.936+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa5c80443b0 con 0x7fa5d8161480 2020-11-16T22:33:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5d8044f60 con 0x7fa5d8161480 2020-11-16T22:33:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5d4ff9700 1 -- 172.21.15.181:0/3344524029 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4663 (secure 0 0 0) 0x7fa5c80479d0 con 0x7fa5d8161480 2020-11-16T22:33:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 msgr2=0x7fa5b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 0x7fa5b4057520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fa5c0006ef0 tx=0x7fa5c0015000).stop 2020-11-16T22:33:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 msgr2=0x7fa5d8146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa5c800eba0 tx=0x7fa5c8006df0).stop 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 shutdown_connections 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa5b4055060 0x7fa5b4057520 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5d8160c40 0x7fa5d80621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5d8161480 0x7fa5d8146030 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 --2- 172.21.15.181:0/3344524029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5d8161860 0x7fa5d8160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.136+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 >> 172.21.15.181:0/3344524029 conn(0x7fa5d80d2dc0 msgr2=0x7fa5d8148710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:22.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.140+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 shutdown_connections 2020-11-16T22:33:22.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:22.140+0000 7fa5dd26a700 1 -- 172.21.15.181:0/3344524029 wait complete. 2020-11-16T22:33:22.728 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 87 2020-11-16T22:33:22.728 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:22.728 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":87,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:19.877195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:33:23.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.075+0000 7f0ef49c0700 1 Processor -- start 2020-11-16T22:33:23.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.075+0000 7f0ef49c0700 1 -- start start 2020-11-16T22:33:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0ef00d0a00 0x7f0ef0140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ef00cef10 0x7f0ef0063ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef00cf380 con 0x7f0ef0140550 2020-11-16T22:33:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef015ebb0 con 0x7f0ef00d0a00 2020-11-16T22:33:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef00cf5d0 con 0x7f0ef00cef10 2020-11-16T22:33:23.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ef00cef10 0x7f0ef0063ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ef00cef10 0x7f0ef0063ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34210/0 (socket says 172.21.15.181:34210) 2020-11-16T22:33:23.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 -- 172.21.15.181:0/495279517 learned_addr learned my addr 172.21.15.181:0/495279517 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/495279517 >> v2:172.21.15.175:3300/0 conn(0x7f0ef00d0a00 0x7f0ef0140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 -- 172.21.15.181:0/495279517 >> v2:172.21.15.175:3300/0 conn(0x7f0ef00d0a00 msgr2=0x7f0ef0140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 --2- 172.21.15.181:0/495279517 >> v2:172.21.15.175:3300/0 conn(0x7f0ef00d0a00 0x7f0ef0140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 -- 172.21.15.181:0/495279517 >> v1:172.21.15.175:6789/0 conn(0x7f0ef0140550 legacy=0x7f0ef00d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 -- 172.21.15.181:0/495279517 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00cf870 con 0x7f0ef00cef10 2020-11-16T22:33:23.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eeffff700 1 --2- 172.21.15.181:0/495279517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef0063ce0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0ef005d280 tx=0x7f0ee8006eb0).ready entity=mon.1 client_cookie=32e8228300c3ba62 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eee7fc700 1 -- 172.21.15.181:0/495279517 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8009810 con 0x7f0ef00cef10 2020-11-16T22:33:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eee7fc700 1 -- 172.21.15.181:0/495279517 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee8009970 con 0x7f0ef00cef10 2020-11-16T22:33:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0eee7fc700 1 -- 172.21.15.181:0/495279517 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8009cf0 con 0x7f0ef00cef10 2020-11-16T22:33:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- 172.21.15.181:0/495279517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 msgr2=0x7f0ef0063ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/495279517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef0063ce0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0ef005d280 tx=0x7f0ee8006eb0).stop 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- 172.21.15.181:0/495279517 shutdown_connections 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/495279517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef0063ce0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/495279517 >> v2:172.21.15.175:3300/0 conn(0x7f0ef00d0a00 0x7f0ef0140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- 172.21.15.181:0/495279517 >> 172.21.15.181:0/495279517 conn(0x7f0ef00d4570 msgr2=0x7f0ef00d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- 172.21.15.181:0/495279517 shutdown_connections 2020-11-16T22:33:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.079+0000 7f0ef49c0700 1 -- 172.21.15.181:0/495279517 wait complete. 2020-11-16T22:33:23.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 Processor -- start 2020-11-16T22:33:23.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- start start 2020-11-16T22:33:23.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef00d3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef0140550 0x7f0ef00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef0152860 con 0x7f0ef0140550 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef0152b00 con 0x7f0ef00cef10 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef0152da0 con 0x7f0ef00d0a00 2020-11-16T22:33:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42054/0 (socket says 172.21.15.181:42054) 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 -- 172.21.15.181:0/1202982076 learned_addr learned my addr 172.21.15.181:0/1202982076 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef00d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeffff700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef0140550 0x7f0ef00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 -- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 msgr2=0x7f0ef00d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef00d3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 -- 172.21.15.181:0/1202982076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef0140550 msgr2=0x7f0ef00b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef0140550 0x7f0ef00b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 -- 172.21.15.181:0/1202982076 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0153040 con 0x7f0ef00d0a00 2020-11-16T22:33:23.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f0ed800ef50 tx=0x7f0ed80091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ee77fe700 1 -- 172.21.15.181:0/1202982076 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800cc10 con 0x7f0ef00d0a00 2020-11-16T22:33:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef01532e0 con 0x7f0ef00d0a00 2020-11-16T22:33:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ee77fe700 1 -- 172.21.15.181:0/1202982076 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800cd70 con 0x7f0ef00d0a00 2020-11-16T22:33:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ee77fe700 1 -- 172.21.15.181:0/1202982076 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8012600 con 0x7f0ef00d0a00 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.083+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef0153a20 con 0x7f0ef00d0a00 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.087+0000 7f0ee77fe700 1 -- 172.21.15.181:0/1202982076 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0ed8022070 con 0x7f0ef00d0a00 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.087+0000 7f0ee77fe700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 0x7f0ecc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.087+0000 7f0ee77fe700 1 -- 172.21.15.181:0/1202982076 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ed806e5e0 con 0x7f0ef00d0a00 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.087+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 0x7f0ecc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.087+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 0x7f0ecc057520 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f0ee0003230 tx=0x7f0ee0000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 Processor -- start 2020-11-16T22:33:23.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 -- start start 2020-11-16T22:33:23.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x27cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 0x27cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26564b0 con 0x25ac7b0 2020-11-16T22:33:23.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x270a930 con 0x27cafd0 2020-11-16T22:33:23.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25ca6f0 con 0x253ebb0 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34220/0 (socket says 172.21.15.181:34220) 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 -- 172.21.15.181:0/1673198729 learned_addr learned my addr 172.21.15.181:0/1673198729 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 0x27cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x27cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 -- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 msgr2=0x27cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 0x27cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 -- 172.21.15.181:0/1673198729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 msgr2=0x27cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x27cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 -- 172.21.15.181:0/1673198729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25c55a0 con 0x27cafd0 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 0x27cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.091+0000 7f0eeffff700 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f0ef00817a0 tx=0x7f0ee8015000).ready entity=mon.1 client_cookie=2ef7f338f5232cf4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0ee4ff9700 1 -- 172.21.15.181:0/1673198729 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8009cf0 con 0x27cafd0 2020-11-16T22:33:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0ee4ff9700 1 -- 172.21.15.181:0/1673198729 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee8006920 con 0x27cafd0 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0ee4ff9700 1 -- 172.21.15.181:0/1673198729 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8011820 con 0x27cafd0 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 msgr2=0x27c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f0ef00817a0 tx=0x7f0ee8015000).stop 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- 172.21.15.181:0/1673198729 shutdown_connections 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27cafd0 0x27c4700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x27cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- 172.21.15.181:0/1673198729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253ebb0 0x27cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- 172.21.15.181:0/1673198729 >> 172.21.15.181:0/1673198729 conn(0x277ae40 msgr2=0x27c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- 172.21.15.181:0/1673198729 shutdown_connections 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- 172.21.15.181:0/1673198729 wait complete. 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- start start 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 Processor -- start 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x27c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27cafd0 0x27c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f76e0 con 0x27cafd0 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f7820 con 0x253ebb0 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26f8a90 con 0x25ac7b0 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0f046b3740 10 client.0 ll_register_callbacks cb 0x26edad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:23.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x27c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34226/0 (socket says 172.21.15.181:34226) 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 -- 172.21.15.181:0/3029469237 learned_addr learned my addr 172.21.15.181:0/3029469237 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eeffff700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27cafd0 0x27c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 -- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 msgr2=0x27c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x27c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 -- 172.21.15.181:0/3029469237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27cafd0 msgr2=0x27c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27cafd0 0x27c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 -- 172.21.15.181:0/3029469237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26e8280 con 0x253ebb0 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x27c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f0ee00159e0 tx=0x7f0ee0019040).ready entity=mon.1 client_cookie=bdde14c886d76b67 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.095+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001c040 con 0x253ebb0 2020-11-16T22:33:23.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ee0017240 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27ab6c0 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee0022e70 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 10 client.7126 fetch_fsmap learned FSMap version 87 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x251dfd0 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27ab6c0 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 87) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f0ee0014620 con 0x253ebb0 2020-11-16T22:33:23.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 10 client.7126 fetch_fsmap finished waiting for FSMap version 87 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 10 client.7126 resolve_mds: resolved ID 'a' to GID 6946 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 20 client.7126 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x282a160 0x27cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 4 client.7126 mds_command: new command op to 6946 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 --> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x25e8c50 con 0x282a160 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x282a160 0x27cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 10 client.7126 ms_handle_connect on v2:172.21.15.181:6838/1430907052 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x282a160 0x27cca70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9594139482d50b66 server_cookie=3e81fb0a9509f599 in_seq=0 out_seq=0 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/3029469237 <== mds.0 v2:172.21.15.181:6838/1430907052 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x25e8c50 con 0x282a160 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 10 client.7126 handle_command_reply: tid=0 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 client.7126 shutdown 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 msgr2=0x27ed870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f0ee00159e0 tx=0x7f0ee0019040).stop 2020-11-16T22:33:23.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 shutdown_connections 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x282a160 0x27cca70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0ec5ffb700 0 client.7126 ms_handle_reset on v2:172.21.15.181:6838/1430907052 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x27cafd0 0x27c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27ed870 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3029469237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x27c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 >> 172.21.15.181:0/3029469237 conn(0x277ae40 msgr2=0x27cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 shutdown_connections 2020-11-16T22:33:23.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 1 -- 172.21.15.181:0/3029469237 wait complete. 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.099+0000 7f0f046b3740 20 client.7126 trim_cache size 0 max 16384 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 Processor -- start 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- start start 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x277ae40 0x27fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x281b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26cfe40 con 0x25ac7b0 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x27ff1e0 con 0x253ebb0 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x27cb5d0 con 0x277ae40 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x277ae40 0x27fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34236/0 (socket says 172.21.15.181:34236) 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 -- 172.21.15.181:0/3991063791 learned_addr learned my addr 172.21.15.181:0/3991063791 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeffff700 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x281b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 -- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x277ae40 msgr2=0x27fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x277ae40 0x27fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 -- 172.21.15.181:0/3991063791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 msgr2=0x281b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x281b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 -- 172.21.15.181:0/3991063791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc0081d0 con 0x253ebb0 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeffff700 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x281b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f0ed8048200 tx=0x7f0ed806e040).ready entity=mon.1 client_cookie=5a1eec905fffb64f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0ee57fa700 1 -- 172.21.15.181:0/3991063791 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80442f0 con 0x253ebb0 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0ee57fa700 1 -- 172.21.15.181:0/3991063791 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8044450 con 0x253ebb0 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0ee57fa700 1 -- 172.21.15.181:0/3991063791 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed806e0b0 con 0x253ebb0 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 msgr2=0x281b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f0ed8048200 tx=0x7f0ed806e040).stop 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- 172.21.15.181:0/3991063791 shutdown_connections 2020-11-16T22:33:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25ac7b0 0x281b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x277ae40 0x27fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 --2- 172.21.15.181:0/3991063791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x281b040 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- 172.21.15.181:0/3991063791 >> 172.21.15.181:0/3991063791 conn(0x7f0ebc002320 msgr2=0x281c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- 172.21.15.181:0/3991063791 shutdown_connections 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.123+0000 7f0f046b3740 1 -- 172.21.15.181:0/3991063791 wait complete. 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- start start 2020-11-16T22:33:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 Processor -- start 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x2876c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x277ae40 0x281b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2815b50 con 0x277ae40 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2880660 con 0x253ebb0 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2880900 con 0x25ac7b0 2020-11-16T22:33:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 10 client.0 ll_register_callbacks cb 0x27cb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34240/0 (socket says 172.21.15.181:34240) 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 -- 172.21.15.181:0/832852595 learned_addr learned my addr 172.21.15.181:0/832852595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x2876c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eeffff700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x277ae40 0x281b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 -- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 msgr2=0x2876c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x2876c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 -- 172.21.15.181:0/832852595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x277ae40 msgr2=0x281b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x277ae40 0x281b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 -- 172.21.15.181:0/832852595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x255c720 con 0x253ebb0 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eef7fe700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f0ee00176e0 tx=0x7f0ee0011000).ready entity=mon.1 client_cookie=cf194de0d02e0af6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0030070 con 0x253ebb0 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ee0018cd0 con 0x253ebb0 2020-11-16T22:33:23.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ee8006920 con 0x253ebb0 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee0018ea0 con 0x253ebb0 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 10 client.7132 fetch_fsmap learned FSMap version 87 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x2567e00 con 0x253ebb0 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee8006920 con 0x253ebb0 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 87) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f0ee0014550 con 0x253ebb0 2020-11-16T22:33:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 10 client.7132 fetch_fsmap finished waiting for FSMap version 87 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 10 client.7132 resolve_mds: resolved ID 'a' to GID 6946 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 20 client.7132 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x27cafd0 0x281b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 4 client.7132 mds_command: new command op to 6946 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 --> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x25eff50 con 0x27cafd0 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x27cafd0 0x281b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0ec5ffb700 10 client.7132 ms_handle_connect on v2:172.21.15.181:6838/1430907052 2020-11-16T22:33:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.127+0000 7f0eeeffd700 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x27cafd0 0x281b530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=22c82aaf7bb4cc35 server_cookie=f5a90bab83d9f032 in_seq=0 out_seq=0 2020-11-16T22:33:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ec5ffb700 1 -- 172.21.15.181:0/832852595 <== mds.0 v2:172.21.15.181:6838/1430907052 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x25eff50 con 0x27cafd0 2020-11-16T22:33:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ec5ffb700 10 client.7132 handle_command_reply: tid=0 2020-11-16T22:33:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 client.7132 shutdown 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 msgr2=0x27bf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f0ee00176e0 tx=0x7f0ee0011000).stop 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 shutdown_connections 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:6838/1430907052,v1:172.21.15.181:6839/1430907052] conn(0x27cafd0 0x281b530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x277ae40 0x281b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x253ebb0 0x27bf6c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ec5ffb700 0 client.7132 ms_handle_reset on v2:172.21.15.181:6838/1430907052 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 --2- 172.21.15.181:0/832852595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25ac7b0 0x2876c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 >> 172.21.15.181:0/832852595 conn(0x7f0ebc002320 msgr2=0x27c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 shutdown_connections 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 1 -- 172.21.15.181:0/832852595 wait complete. 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0f046b3740 20 client.7132 trim_cache size 0 max 16384 2020-11-16T22:33:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 msgr2=0x7f0ecc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 0x7f0ecc057520 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f0ee0003230 tx=0x7f0ee0000b80).stop 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 msgr2=0x7f0ef00b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f0ed800ef50 tx=0x7f0ed80091d0).stop 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 shutdown_connections 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ecc055060 0x7f0ecc057520 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef0140550 0x7f0ef00b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef00cef10 0x7f0ef00d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 --2- 172.21.15.181:0/1202982076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ef00d0a00 0x7f0ef00b4900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 >> 172.21.15.181:0/1202982076 conn(0x7f0ef00d4570 msgr2=0x7f0ef0062bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 shutdown_connections 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.147+0000 7f0ef49c0700 1 -- 172.21.15.181:0/1202982076 wait complete. 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:33:23.720 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:33:23.721 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:33:23.721 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:33:23.721 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 Processor -- start 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 -- start start 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 0x7f45680cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45680cf5d0 con 0x7f4568140550 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45680cf870 con 0x7f45680d0a00 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456d116700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456815ebb0 con 0x7f4568140930 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f45677fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f45680d0a00 0x7f4568063ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48552/0 (socket says 172.21.15.181:48552) 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456c915700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 0x7f45680cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f45677fe700 1 -- 172.21.15.181:0/1014359529 learned_addr learned my addr 172.21.15.181:0/1014359529 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709668010 0 0) 0x7f45680cf870 con 0x7f45680d0a00 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f454c003240 con 0x7f45680d0a00 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472469931 0 0) 0x7f45680cf5d0 con 0x7f4568140550 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45680cf870 con 0x7f4568140550 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456c915700 1 -- 172.21.15.181:0/1014359529 >> v1:172.21.15.181:6789/0 conn(0x7f45680d0a00 legacy=0x7f4568063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456c915700 1 -- 172.21.15.181:0/1014359529 >> v1:172.21.15.175:6789/0 conn(0x7f4568140550 legacy=0x7f45680d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456c915700 1 -- 172.21.15.181:0/1014359529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680cfb10 con 0x7f4568140930 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f456c915700 1 --2- 172.21.15.181:0/1014359529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 0x7f45680cef10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f456000b9d0 tx=0x7f4560006df0).ready entity=mon.2 client_cookie=34bbc58ed46001ba server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560009680 con 0x7f4568140930 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45600097e0 con 0x7f4568140930 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.863+0000 7f4566ffd700 1 -- 172.21.15.181:0/1014359529 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560009b60 con 0x7f4568140930 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- 172.21.15.181:0/1014359529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 msgr2=0x7f45680cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 --2- 172.21.15.181:0/1014359529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 0x7f45680cef10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f456000b9d0 tx=0x7f4560006df0).stop 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- 172.21.15.181:0/1014359529 shutdown_connections 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 --2- 172.21.15.181:0/1014359529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568140930 0x7f45680cef10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- 172.21.15.181:0/1014359529 >> 172.21.15.181:0/1014359529 conn(0x7f45680d4570 msgr2=0x7f45680d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- 172.21.15.181:0/1014359529 shutdown_connections 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- 172.21.15.181:0/1014359529 wait complete. 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 Processor -- start 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.867+0000 7f456d116700 1 -- start start 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4568140550 0x7f45680cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 0x7f45680cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456816dcb0 con 0x7f4568140550 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456816ccf0 con 0x7f4568140930 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456816cf90 con 0x7f45680d0a00 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42096/0 (socket says 172.21.15.181:42096) 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 -- 172.21.15.181:0/3168122309 learned_addr learned my addr 172.21.15.181:0/3168122309 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456c915700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 0x7f45680cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f45677fe700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4568140550 0x7f45680cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 -- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 msgr2=0x7f45680cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 0x7f45680cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 -- 172.21.15.181:0/3168122309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4568140550 msgr2=0x7f45680cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4568140550 0x7f45680cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 -- 172.21.15.181:0/3168122309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456816d230 con 0x7f45680d0a00 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456c915700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 0x7f45680cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f4567fff700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f455800ca90 tx=0x7f4558007d90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558002240 con 0x7f45680d0a00 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45580023a0 con 0x7f45680d0a00 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558002510 con 0x7f45680d0a00 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456816d4d0 con 0x7f45680d0a00 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456816da10 con 0x7f45680d0a00 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.871+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4568044f60 con 0x7f45680d0a00 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.875+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4558019030 con 0x7f45680d0a00 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.875+0000 7f455ffff700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 0x7f4538057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.875+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f455806d620 con 0x7f45680d0a00 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.875+0000 7f45677fe700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 0x7f4538057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.875+0000 7f45677fe700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 0x7f4538057520 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f4568146790 tx=0x7f4550009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:23.879+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4558043770 con 0x7f45680d0a00 2020-11-16T22:33:24.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.083+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45680d0610 con 0x7f45680d0a00 2020-11-16T22:33:24.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f455ffff700 1 -- 172.21.15.181:0/3168122309 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4663 (secure 0 0 0) 0x7f4558046d90 con 0x7f45680d0a00 2020-11-16T22:33:24.075 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:24.075 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":87,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:19.877195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 msgr2=0x7f4538057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 0x7f4538057520 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f4568146790 tx=0x7f4550009040).stop 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 msgr2=0x7f45680b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f455800ca90 tx=0x7f4558007d90).stop 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 shutdown_connections 2020-11-16T22:33:24.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4538055060 0x7f4538057520 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4568140930 0x7f45680cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0a00 0x7f45680b95a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 --2- 172.21.15.181:0/3168122309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4568140550 0x7f45680cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 >> 172.21.15.181:0/3168122309 conn(0x7f45680d4570 msgr2=0x7f456813f460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 shutdown_connections 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.087+0000 7f456d116700 1 -- 172.21.15.181:0/3168122309 wait complete. 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 87 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d flush journal 2020-11-16T22:33:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 Processor -- start 2020-11-16T22:33:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 -- start start 2020-11-16T22:33:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f192c161860 0x7f192c062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192c0c8db0 con 0x7f192c161480 2020-11-16T22:33:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192c165030 con 0x7f192c161860 2020-11-16T22:33:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932933700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c0c8330 con 0x7f192c062740 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42106/0 (socket says 172.21.15.181:42106) 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 -- 172.21.15.181:0/1290208972 learned_addr learned my addr 172.21.15.181:0/1290208972 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1931130700 1 --2- 172.21.15.181:0/1290208972 >> v2:172.21.15.175:3300/0 conn(0x7f192c161860 0x7f192c062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 -- 172.21.15.181:0/1290208972 >> v2:172.21.15.175:3300/0 conn(0x7f192c161860 msgr2=0x7f192c062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 --2- 172.21.15.181:0/1290208972 >> v2:172.21.15.175:3300/0 conn(0x7f192c161860 0x7f192c062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 -- 172.21.15.181:0/1290208972 >> v1:172.21.15.175:6789/0 conn(0x7f192c161480 legacy=0x7f192c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 -- 172.21.15.181:0/1290208972 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c0c85d0 con 0x7f192c062740 2020-11-16T22:33:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f1932132700 1 --2- 172.21.15.181:0/1290208972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f192c0ced60 tx=0x7f1924009290).ready entity=mon.2 client_cookie=e12ae8440b45a8c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f193092f700 1 -- 172.21.15.181:0/1290208972 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192400bb80 con 0x7f192c062740 2020-11-16T22:33:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f193092f700 1 -- 172.21.15.181:0/1290208972 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192400bce0 con 0x7f192c062740 2020-11-16T22:33:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.223+0000 7f193092f700 1 -- 172.21.15.181:0/1290208972 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924011610 con 0x7f192c062740 2020-11-16T22:33:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/1290208972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 msgr2=0x7f192c0644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- 172.21.15.181:0/1290208972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f192c0ced60 tx=0x7f1924009290).stop 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/1290208972 shutdown_connections 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- 172.21.15.181:0/1290208972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c062740 0x7f192c0644e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- 172.21.15.181:0/1290208972 >> v2:172.21.15.175:3300/0 conn(0x7f192c161860 0x7f192c062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/1290208972 >> 172.21.15.181:0/1290208972 conn(0x7f192c0d2dc0 msgr2=0x7f192c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/1290208972 shutdown_connections 2020-11-16T22:33:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/1290208972 wait complete. 2020-11-16T22:33:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 Processor -- start 2020-11-16T22:33:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- start start 2020-11-16T22:33:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c161480 0x7f192c163200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c161860 0x7f192c1636f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c173390 con 0x7f192c161480 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c176e70 con 0x7f192c062740 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c177110 con 0x7f192c161860 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34268/0 (socket says 172.21.15.181:34268) 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 -- 172.21.15.181:0/2548063884 learned_addr learned my addr 172.21.15.181:0/2548063884 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932132700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c161860 0x7f192c1636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931130700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c161480 0x7f192c163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 -- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c161860 msgr2=0x7f192c1636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c161860 0x7f192c1636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 -- 172.21.15.181:0/2548063884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c161480 msgr2=0x7f192c163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c161480 0x7f192c163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 -- 172.21.15.181:0/2548063884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c1773b0 con 0x7f192c062740 2020-11-16T22:33:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1931931700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f192000c730 tx=0x7f1920007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f191e7fc700 1 -- 172.21.15.181:0/2548063884 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000e560 con 0x7f192c062740 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f191e7fc700 1 -- 172.21.15.181:0/2548063884 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192000e6e0 con 0x7f192c062740 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c177650 con 0x7f192c062740 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f191e7fc700 1 -- 172.21.15.181:0/2548063884 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000eaa0 con 0x7f192c062740 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.227+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c164c00 con 0x7f192c062740 2020-11-16T22:33:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.231+0000 7f191e7fc700 1 -- 172.21.15.181:0/2548063884 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1920010030 con 0x7f192c062740 2020-11-16T22:33:24.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.231+0000 7f191e7fc700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 0x7f1900057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.231+0000 7f1931130700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 0x7f1900057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.231+0000 7f191e7fc700 1 -- 172.21.15.181:0/2548063884 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f192006dc30 con 0x7f192c062740 2020-11-16T22:33:24.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.231+0000 7f1931130700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 0x7f1900057520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1928006fd0 tx=0x7f1928008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 Processor -- start 2020-11-16T22:33:24.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 -- start start 2020-11-16T22:33:24.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2241bb0 0x24ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 0x24c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23594b0 con 0x24cdfd0 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x240d930 con 0x22af7b0 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22cd6f0 con 0x2241bb0 2020-11-16T22:33:24.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34276/0 (socket says 172.21.15.181:34276) 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 -- 172.21.15.181:0/1210143380 learned_addr learned my addr 172.21.15.181:0/1210143380 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931130700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2241bb0 0x24ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1932132700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 0x24c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 -- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2241bb0 msgr2=0x24ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2241bb0 0x24ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 -- 172.21.15.181:0/1210143380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 msgr2=0x24c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 0x24c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 -- 172.21.15.181:0/1210143380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22c85a0 con 0x22af7b0 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1932132700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 0x24c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1931931700 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f1920047850 tx=0x7f1920011000).ready entity=mon.1 client_cookie=e48265ab47bba0f5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1916ffd700 1 -- 172.21.15.181:0/1210143380 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920044260 con 0x22af7b0 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1916ffd700 1 -- 172.21.15.181:0/1210143380 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200443c0 con 0x22af7b0 2020-11-16T22:33:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.235+0000 7f1916ffd700 1 -- 172.21.15.181:0/1210143380 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920047250 con 0x22af7b0 2020-11-16T22:33:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 msgr2=0x24d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f1920047850 tx=0x7f1920011000).stop 2020-11-16T22:33:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- 172.21.15.181:0/1210143380 shutdown_connections 2020-11-16T22:33:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24cdfd0 0x24c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22af7b0 0x24d0720 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 --2- 172.21.15.181:0/1210143380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2241bb0 0x24ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- 172.21.15.181:0/1210143380 >> 172.21.15.181:0/1210143380 conn(0x247de40 msgr2=0x24ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- 172.21.15.181:0/1210143380 shutdown_connections 2020-11-16T22:33:24.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- 172.21.15.181:0/1210143380 wait complete. 2020-11-16T22:33:24.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 -- start start 2020-11-16T22:33:24.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.239+0000 7f1942626740 1 Processor -- start 2020-11-16T22:33:24.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x2530060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24cdfd0 0x251bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23fa6e0 con 0x2241bb0 2020-11-16T22:33:24.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x23fa820 con 0x24cdfd0 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23fba90 con 0x22af7b0 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 10 client.0 ll_register_callbacks cb 0x23f0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1932132700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24cdfd0 0x251bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42126/0 (socket says 172.21.15.181:42126) 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 -- 172.21.15.181:0/2749360111 learned_addr learned my addr 172.21.15.181:0/2749360111 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931931700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x2530060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 -- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24cdfd0 msgr2=0x251bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24cdfd0 0x251bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 -- 172.21.15.181:0/2749360111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 msgr2=0x2530060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x2530060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 -- 172.21.15.181:0/2749360111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23eb280 con 0x22af7b0 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931130700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f192c148d20 tx=0x7f1928012000).ready entity=mon.2 client_cookie=6acde8ae3d97c08 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19280105c0 con 0x22af7b0 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1928010720 con 0x22af7b0 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24ae6c0 con 0x22af7b0 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1928010af0 con 0x22af7b0 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 10 client.6566 fetch_fsmap learned FSMap version 87 2020-11-16T22:33:24.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x2220fd0 con 0x22af7b0 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24ae6c0 con 0x22af7b0 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 87) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f1928010c50 con 0x22af7b0 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 10 client.6566 fetch_fsmap finished waiting for FSMap version 87 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 10 client.6566 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 20 client.6566 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x252da50 0x24ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 4 client.6566 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x22ebc50 con 0x252da50 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931931700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x252da50 0x24ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 10 client.6566 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1931931700 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x252da50 0x24ca5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=72a381cf4eb27251 server_cookie=acf3e4e74ca56004 in_seq=0 out_seq=0 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 1 -- 172.21.15.181:0/2749360111 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x22ebc50 con 0x252da50 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 10 client.6566 handle_command_reply: tid=0 2020-11-16T22:33:24.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 client.6566 shutdown 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 msgr2=0x251b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f192c148d20 tx=0x7f1928012000).stop 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 shutdown_connections 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x252da50 0x24ca5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24cdfd0 0x251bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f18fffff700 0 client.6566 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:24.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x2530060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 --2- 172.21.15.181:0/2749360111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251b770 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.243+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 >> 172.21.15.181:0/2749360111 conn(0x247de40 msgr2=0x2503990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.247+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 shutdown_connections 2020-11-16T22:33:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.247+0000 7f1942626740 1 -- 172.21.15.181:0/2749360111 wait complete. 2020-11-16T22:33:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.247+0000 7f1942626740 20 client.6566 trim_cache size 0 max 16384 2020-11-16T22:33:24.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.263+0000 7f1942626740 1 Processor -- start 2020-11-16T22:33:24.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- start start 2020-11-16T22:33:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x24ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d2e40 con 0x2241bb0 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x252cb10 con 0x247de40 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2501bf0 con 0x22af7b0 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1932132700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1932132700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42132/0 (socket says 172.21.15.181:42132) 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1932132700 1 -- 172.21.15.181:0/2508792380 learned_addr learned my addr 172.21.15.181:0/2508792380 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34290/0 (socket says 172.21.15.181:34290) 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931130700 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x24ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 -- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 msgr2=0x251bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 -- 172.21.15.181:0/2508792380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 msgr2=0x24ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x24ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 -- 172.21.15.181:0/2508792380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8008160 con 0x247de40 2020-11-16T22:33:24.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931130700 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x24ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:24.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1931931700 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1920027000 tx=0x7f192000c040).ready entity=mon.1 client_cookie=f8782d0751cbabac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f19177fe700 1 -- 172.21.15.181:0/2508792380 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920044260 con 0x247de40 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f19177fe700 1 -- 172.21.15.181:0/2508792380 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200350e0 con 0x247de40 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f19177fe700 1 -- 172.21.15.181:0/2508792380 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920045d70 con 0x247de40 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 msgr2=0x252d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1920027000 tx=0x7f192000c040).stop 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- 172.21.15.181:0/2508792380 shutdown_connections 2020-11-16T22:33:24.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x252d510 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 --2- 172.21.15.181:0/2508792380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x24ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- 172.21.15.181:0/2508792380 >> 172.21.15.181:0/2508792380 conn(0x7f18f8002320 msgr2=0x251e310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- 172.21.15.181:0/2508792380 shutdown_connections 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- 172.21.15.181:0/2508792380 wait complete. 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 -- start start 2020-11-16T22:33:24.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.267+0000 7f1942626740 1 Processor -- start 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x252f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x24c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x251c0e0 con 0x2241bb0 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c8740 con 0x247de40 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x252d750 con 0x22af7b0 2020-11-16T22:33:24.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 10 client.0 ll_register_callbacks cb 0x25014a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42138/0 (socket says 172.21.15.181:42138) 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 -- 172.21.15.181:0/245654624 learned_addr learned my addr 172.21.15.181:0/245654624 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1932132700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x24c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931931700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x252f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 -- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 msgr2=0x24c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x24c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 -- 172.21.15.181:0/245654624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 msgr2=0x252f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x252f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 -- 172.21.15.181:0/245654624 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x225f720 con 0x22af7b0 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931931700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x252f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931130700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f1928006bd0 tx=0x7f1928015000).ready entity=mon.2 client_cookie=bc36c835704c0e37 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928034070 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f192801dba0 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19200443c0 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1928011380 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 10 client.6572 fetch_fsmap learned FSMap version 87 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x226ae00 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200443c0 con 0x22af7b0 2020-11-16T22:33:24.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 87) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f19280114e0 con 0x22af7b0 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 10 client.6572 fetch_fsmap finished waiting for FSMap version 87 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 10 client.6572 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 20 client.6572 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x24cdfd0 0x257f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 4 client.6572 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x22f2f50 con 0x24cdfd0 2020-11-16T22:33:24.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931931700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x24cdfd0 0x257f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f18fffff700 10 client.6572 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:24.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.271+0000 7f1931931700 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x24cdfd0 0x257f1c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=ad1885be1589a1aa server_cookie=67b6d7038ae2a424 in_seq=0 out_seq=0 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f18fffff700 1 -- 172.21.15.181:0/245654624 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x22f2f50 con 0x24cdfd0 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f18fffff700 10 client.6572 handle_command_reply: tid=0 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 client.6572 shutdown 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 msgr2=0x251bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f1928006bd0 tx=0x7f1928015000).stop 2020-11-16T22:33:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 shutdown_connections 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x24cdfd0 0x257f1c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x247de40 0x24c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f18fffff700 0 client.6572 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2241bb0 0x252f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 --2- 172.21.15.181:0/245654624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22af7b0 0x251bbf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 >> 172.21.15.181:0/245654624 conn(0x7f18f8002320 msgr2=0x251f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.287+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 shutdown_connections 2020-11-16T22:33:24.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1942626740 1 -- 172.21.15.181:0/245654624 wait complete. 2020-11-16T22:33:24.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1942626740 20 client.6572 trim_cache size 0 max 16384 2020-11-16T22:33:24.282 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:33:24.282 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:33:24.282 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:33:24.282 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:33:24.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 msgr2=0x7f1900057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 0x7f1900057520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f1928006fd0 tx=0x7f1928008040).stop 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 msgr2=0x7f192c164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f192000c730 tx=0x7f1920007c30).stop 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 shutdown_connections 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1900055060 0x7f1900057520 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f192c161860 0x7f192c1636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192c062740 0x7f192c164710 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 --2- 172.21.15.181:0/2548063884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c161480 0x7f192c163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 >> 172.21.15.181:0/2548063884 conn(0x7f192c0d2dc0 msgr2=0x7f192c160840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 shutdown_connections 2020-11-16T22:33:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.291+0000 7f1932933700 1 -- 172.21.15.181:0/2548063884 wait complete. 2020-11-16T22:33:24.294 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:24.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 Processor -- start 2020-11-16T22:33:24.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 -- start start 2020-11-16T22:33:24.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac64063620 0x7fac64061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac64160c40 con 0x7fac64142660 2020-11-16T22:33:24.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac64062700 con 0x7fac64142a40 2020-11-16T22:33:24.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac63fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac640629a0 con 0x7fac64063620 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac5a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fac64142a40 0x7fac64160530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48608/0 (socket says 172.21.15.181:48608) 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac5a7fc700 1 -- 172.21.15.181:0/792900868 learned_addr learned my addr 172.21.15.181:0/792900868 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac637fe700 1 --2- 172.21.15.181:0/792900868 >> v2:172.21.15.181:3300/0 conn(0x7fac64063620 0x7fac64061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880390322 0 0) 0x7fac64062700 con 0x7fac64142a40 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac44003240 con 0x7fac64142a40 2020-11-16T22:33:24.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 422219188 0 0) 0x7fac44003240 con 0x7fac64142a40 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac637fe700 1 -- 172.21.15.181:0/792900868 >> v1:172.21.15.181:6789/0 conn(0x7fac64142a40 legacy=0x7fac64160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac637fe700 1 -- 172.21.15.181:0/792900868 >> v1:172.21.15.175:6789/0 conn(0x7fac64142660 legacy=0x7fac64063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac637fe700 1 -- 172.21.15.181:0/792900868 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64062c40 con 0x7fac64063620 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac637fe700 1 --2- 172.21.15.181:0/792900868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64063620 0x7fac64061810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fac5c009c10 tx=0x7fac5c0091d0).ready entity=mon.1 client_cookie=efc91b4ab91ccb21 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0014e0 con 0x7fac64063620 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c00b790 con 0x7fac64063620 2020-11-16T22:33:24.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.427+0000 7fac627fc700 1 -- 172.21.15.181:0/792900868 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00bb10 con 0x7fac64063620 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- 172.21.15.181:0/792900868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64063620 msgr2=0x7fac64061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 --2- 172.21.15.181:0/792900868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64063620 0x7fac64061810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fac5c009c10 tx=0x7fac5c0091d0).stop 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- 172.21.15.181:0/792900868 shutdown_connections 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 --2- 172.21.15.181:0/792900868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64063620 0x7fac64061810 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- 172.21.15.181:0/792900868 >> 172.21.15.181:0/792900868 conn(0x7fac640d2dc0 msgr2=0x7fac640ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- 172.21.15.181:0/792900868 shutdown_connections 2020-11-16T22:33:24.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- 172.21.15.181:0/792900868 wait complete. 2020-11-16T22:33:24.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 Processor -- start 2020-11-16T22:33:24.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- start start 2020-11-16T22:33:24.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac64142660 0x7fac640ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64142a40 0x7fac640cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640cf500 con 0x7fac64142660 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac641671d0 con 0x7fac64142a40 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac63fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640cdea0 con 0x7fac64063620 2020-11-16T22:33:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42152/0 (socket says 172.21.15.181:42152) 2020-11-16T22:33:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 -- 172.21.15.181:0/1911387323 learned_addr learned my addr 172.21.15.181:0/1911387323 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac637fe700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64142a40 0x7fac640cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac5a7fc700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac64142660 0x7fac640ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 -- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64142a40 msgr2=0x7fac640cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64142a40 0x7fac640cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 -- 172.21.15.181:0/1911387323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac64142660 msgr2=0x7fac640ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac64142660 0x7fac640ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 -- 172.21.15.181:0/1911387323 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac640ce140 con 0x7fac64063620 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac62ffd700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fac5400c6e0 tx=0x7fac54007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.431+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007020 con 0x7fac64063620 2020-11-16T22:33:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54007180 con 0x7fac64063620 2020-11-16T22:33:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac640ce3e0 con 0x7fac64063620 2020-11-16T22:33:24.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540064f0 con 0x7fac64063620 2020-11-16T22:33:24.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6413ac90 con 0x7fac64063620 2020-11-16T22:33:24.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fac54010070 con 0x7fac64063620 2020-11-16T22:33:24.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac64167390 con 0x7fac64063620 2020-11-16T22:33:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5b7fe700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 0x7fac40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac5406d030 con 0x7fac64063620 2020-11-16T22:33:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5a7fc700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 0x7fac40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.435+0000 7fac5a7fc700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 0x7fac40057520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fac640b8320 tx=0x7fac4c000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.439+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fac54043290 con 0x7fac64063620 2020-11-16T22:33:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac64044f60 con 0x7fac64063620 2020-11-16T22:33:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac5b7fe700 1 -- 172.21.15.181:0/1911387323 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4663 (secure 0 0 0) 0x7fac540468b0 con 0x7fac64063620 2020-11-16T22:33:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 msgr2=0x7fac40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 0x7fac40057520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fac640b8320 tx=0x7fac4c000b80).stop 2020-11-16T22:33:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 msgr2=0x7fac6413a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fac5400c6e0 tx=0x7fac54007c50).stop 2020-11-16T22:33:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 shutdown_connections 2020-11-16T22:33:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fac40055060 0x7fac40057520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac64142a40 0x7fac640cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac64063620 0x7fac6413a7a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.643+0000 7fac63fff700 1 --2- 172.21.15.181:0/1911387323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac64142660 0x7fac640ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.647+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 >> 172.21.15.181:0/1911387323 conn(0x7fac640d2dc0 msgr2=0x7fac641604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.647+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 shutdown_connections 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:24.647+0000 7fac63fff700 1 -- 172.21.15.181:0/1911387323 wait complete. 2020-11-16T22:33:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 87 2020-11-16T22:33:25.040 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:25.040 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":87,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:19.877195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:25.242 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h flush journal 2020-11-16T22:33:25.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 Processor -- start 2020-11-16T22:33:25.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- start start 2020-11-16T22:33:25.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd52415d8f0 0x7fd5240b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd5240b79f0 0x7fd5240b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240ceb10 con 0x7fd52415d510 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5240cedb0 con 0x7fd5240b79f0 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5240cf050 con 0x7fd52415d8f0 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd52415d8f0 0x7fd5240b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd52415d8f0 0x7fd5240b7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34318/0 (socket says 172.21.15.181:34318) 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 -- 172.21.15.181:0/90546924 learned_addr learned my addr 172.21.15.181:0/90546924 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd523fff700 1 --2- 172.21.15.181:0/90546924 >> v2:172.21.15.175:3300/0 conn(0x7fd5240b79f0 0x7fd5240b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 -- 172.21.15.181:0/90546924 >> v2:172.21.15.175:3300/0 conn(0x7fd5240b79f0 msgr2=0x7fd5240b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 --2- 172.21.15.181:0/90546924 >> v2:172.21.15.175:3300/0 conn(0x7fd5240b79f0 0x7fd5240b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 -- 172.21.15.181:0/90546924 >> v1:172.21.15.175:6789/0 conn(0x7fd52415d510 legacy=0x7fd5240ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:25.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 -- 172.21.15.181:0/90546924 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240cf2f0 con 0x7fd52415d8f0 2020-11-16T22:33:25.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd522ffd700 1 --2- 172.21.15.181:0/90546924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd5240b7500 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fd5240671f0 tx=0x7fd50c006eb0).ready entity=mon.1 client_cookie=14b43322b2420802 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:25.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd5227fc700 1 -- 172.21.15.181:0/90546924 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0098d0 con 0x7fd52415d8f0 2020-11-16T22:33:25.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd5227fc700 1 -- 172.21.15.181:0/90546924 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c009a30 con 0x7fd52415d8f0 2020-11-16T22:33:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd5227fc700 1 -- 172.21.15.181:0/90546924 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c009db0 con 0x7fd52415d8f0 2020-11-16T22:33:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- 172.21.15.181:0/90546924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 msgr2=0x7fd5240b7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 --2- 172.21.15.181:0/90546924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd5240b7500 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fd5240671f0 tx=0x7fd50c006eb0).stop 2020-11-16T22:33:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- 172.21.15.181:0/90546924 shutdown_connections 2020-11-16T22:33:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 --2- 172.21.15.181:0/90546924 >> v2:172.21.15.175:3300/0 conn(0x7fd5240b79f0 0x7fd5240b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 --2- 172.21.15.181:0/90546924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd5240b7500 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- 172.21.15.181:0/90546924 >> 172.21.15.181:0/90546924 conn(0x7fd5240d3680 msgr2=0x7fd5240d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- 172.21.15.181:0/90546924 shutdown_connections 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.387+0000 7fd529c6b700 1 -- 172.21.15.181:0/90546924 wait complete. 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 Processor -- start 2020-11-16T22:33:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- start start 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240b79f0 0x7fd5241701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241735f0 con 0x7fd5240b79f0 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524173890 con 0x7fd52415d8f0 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524173b30 con 0x7fd52415d510 2020-11-16T22:33:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:25.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:25.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34326/0 (socket says 172.21.15.181:34326) 2020-11-16T22:33:25.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42166/0 (socket says 172.21.15.181:42166) 2020-11-16T22:33:25.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 -- 172.21.15.181:0/2419647070 learned_addr learned my addr 172.21.15.181:0/2419647070 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:25.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240b79f0 0x7fd5241701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 -- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 msgr2=0x7fd524147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 -- 172.21.15.181:0/2419647070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240b79f0 msgr2=0x7fd5241701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240b79f0 0x7fd5241701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 -- 172.21.15.181:0/2419647070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52416fa70 con 0x7fd52415d8f0 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd522ffd700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:25.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd523fff700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fd51c0067d0 tx=0x7fd51c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd51b7fe700 1 -- 172.21.15.181:0/2419647070 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00f5e0 con 0x7fd52415d8f0 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd51b7fe700 1 -- 172.21.15.181:0/2419647070 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c00f740 con 0x7fd52415d8f0 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52416fd10 con 0x7fd52415d8f0 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd51b7fe700 1 -- 172.21.15.181:0/2419647070 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00fb00 con 0x7fd52415d8f0 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.391+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd524147980 con 0x7fd52415d8f0 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.395+0000 7fd51b7fe700 1 -- 172.21.15.181:0/2419647070 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd51c012420 con 0x7fd52415d8f0 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.395+0000 7fd51b7fe700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 0x7fd500057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.395+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 0x7fd500057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.395+0000 7fd51b7fe700 1 -- 172.21.15.181:0/2419647070 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd51c06dde0 con 0x7fd52415d8f0 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.395+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 0x7fd500057550 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fd514004250 tx=0x7fd514007e80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 Processor -- start 2020-11-16T22:33:26.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 -- start start 2020-11-16T22:33:26.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21dbbb0 0x249f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 0x24bcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f34b0 con 0x2462470 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x23a7930 con 0x22497b0 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22676f0 con 0x21dbbb0 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34332/0 (socket says 172.21.15.181:34332) 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 -- 172.21.15.181:0/954319726 learned_addr learned my addr 172.21.15.181:0/954319726 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd522ffd700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21dbbb0 0x249f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd523fff700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 0x24bcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 -- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21dbbb0 msgr2=0x249f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21dbbb0 0x249f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 -- 172.21.15.181:0/954319726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 msgr2=0x24bcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 0x24bcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 -- 172.21.15.181:0/954319726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22625a0 con 0x22497b0 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd523fff700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 0x24bcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd5237fe700 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fd514014df0 tx=0x7fd514010000).ready entity=mon.1 client_cookie=6903dc4ed4db8e27 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:26.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd518ff9700 1 -- 172.21.15.181:0/954319726 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140115b0 con 0x22497b0 2020-11-16T22:33:26.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd518ff9700 1 -- 172.21.15.181:0/954319726 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514011710 con 0x22497b0 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.399+0000 7fd518ff9700 1 -- 172.21.15.181:0/954319726 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514011ad0 con 0x22497b0 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 msgr2=0x2460cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fd514014df0 tx=0x7fd514010000).stop 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/954319726 shutdown_connections 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2462470 0x24bcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2460cd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- 172.21.15.181:0/954319726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21dbbb0 0x249f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/954319726 >> 172.21.15.181:0/954319726 conn(0x2417e40 msgr2=0x24bc580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/954319726 shutdown_connections 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/954319726 wait complete. 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- start start 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 Processor -- start 2020-11-16T22:33:26.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x248a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x24ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:26.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23946e0 con 0x21dbbb0 2020-11-16T22:33:26.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2394820 con 0x2462470 2020-11-16T22:33:26.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2395a90 con 0x22497b0 2020-11-16T22:33:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 10 client.0 ll_register_callbacks cb 0x238aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x24ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34338/0 (socket says 172.21.15.181:34338) 2020-11-16T22:33:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 -- 172.21.15.181:0/2761020815 learned_addr learned my addr 172.21.15.181:0/2761020815 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x248a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 -- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 msgr2=0x24ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x24ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 -- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 msgr2=0x248a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x248a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 -- 172.21.15.181:0/2761020815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2385280 con 0x2462470 2020-11-16T22:33:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x248a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd523fff700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fd51c047ac0 tx=0x7fd51c016000).ready entity=mon.1 client_cookie=d14690e39a8d71d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd4f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c043980 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x245ce10 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd51c043ae0 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd51c043ca0 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 10 client.7174 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.403+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x21bafd0 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x245ce10 con 0x2462470 2020-11-16T22:33:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fd51c046d00 con 0x2462470 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 10 client.7174 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 10 client.7174 resolve_mds: resolved ID 'h' to GID 6798 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 20 client.7174 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x24c9520 0x24b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 4 client.7174 mds_command: new command op to 6798 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 --> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2285c50 con 0x24c9520 2020-11-16T22:33:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x24c9520 0x24b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 10 client.7174 ms_handle_connect on v2:172.21.15.175:6840/2661622280 2020-11-16T22:33:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5237fe700 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x24c9520 0x24b8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=98cc33a4a976836 server_cookie=ad96af1449289707 in_seq=0 out_seq=0 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/2761020815 <== mds.2 v2:172.21.15.175:6840/2661622280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2285c50 con 0x24c9520 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 10 client.7174 handle_command_reply: tid=0 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 client.7174 shutdown 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 msgr2=0x24b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fd51c047ac0 tx=0x7fd51c016000).stop 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 shutdown_connections 2020-11-16T22:33:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x24c9520 0x24b8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2462470 0x24b7bc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd4f9ffb700 0 client.7174 ms_handle_reset on v2:172.21.15.175:6840/2661622280 2020-11-16T22:33:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x248a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 --2- 172.21.15.181:0/2761020815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x24ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 >> 172.21.15.181:0/2761020815 conn(0x2417e40 msgr2=0x249f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:27.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 shutdown_connections 2020-11-16T22:33:27.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 1 -- 172.21.15.181:0/2761020815 wait complete. 2020-11-16T22:33:27.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.407+0000 7fd5386fc740 20 client.7174 trim_cache size 0 max 16384 2020-11-16T22:33:27.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.427+0000 7fd5386fc740 1 Processor -- start 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.427+0000 7fd5386fc740 1 -- start start 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 0x24a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 0x24675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x2467ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x236ce40 con 0x2417e40 2020-11-16T22:33:27.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24caab0 con 0x21dbbb0 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24ca740 con 0x22497b0 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 0x24675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x2467ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 0x24675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34348/0 (socket says 172.21.15.181:34348) 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd522ffd700 1 -- 172.21.15.181:0/3184389560 learned_addr learned my addr 172.21.15.181:0/3184389560 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5237fe700 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 0x24a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 -- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 msgr2=0x24675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:27.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 0x24675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 -- 172.21.15.181:0/3184389560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 msgr2=0x24a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:27.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 0x24a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 -- 172.21.15.181:0/3184389560 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00081d0 con 0x22497b0 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5237fe700 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 0x24a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd523fff700 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x2467ac0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd51c047ac0 tx=0x7fd51c047800).ready entity=mon.2 client_cookie=13cb5d014960fce4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5197fa700 1 -- 172.21.15.181:0/3184389560 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c046d00 con 0x22497b0 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5197fa700 1 -- 172.21.15.181:0/3184389560 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c046e60 con 0x22497b0 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5197fa700 1 -- 172.21.15.181:0/3184389560 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c006090 con 0x22497b0 2020-11-16T22:33:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 msgr2=0x2467ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x2467ac0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd51c047ac0 tx=0x7fd51c047800).stop 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- 172.21.15.181:0/3184389560 shutdown_connections 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22497b0 0x2467ac0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2417e40 0x24a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- 172.21.15.181:0/3184389560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x21dbbb0 0x24675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- 172.21.15.181:0/3184389560 >> 172.21.15.181:0/3184389560 conn(0x7fd4f0002320 msgr2=0x2469140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- 172.21.15.181:0/3184389560 shutdown_connections 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- 172.21.15.181:0/3184389560 wait complete. 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- start start 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 Processor -- start 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x2461d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 0x25164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24609d0 con 0x21dbbb0 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x251a010 con 0x22497b0 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.431+0000 7fd5386fc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24604d0 con 0x2417e40 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 10 client.0 ll_register_callbacks cb 0x24a13c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd523fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 0x25164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34352/0 (socket says 172.21.15.181:34352) 2020-11-16T22:33:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 -- 172.21.15.181:0/272840746 learned_addr learned my addr 172.21.15.181:0/272840746 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5237fe700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x2461d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 -- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 msgr2=0x25164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 0x25164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 -- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 msgr2=0x2461d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x2461d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 -- 172.21.15.181:0/272840746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21f9720 con 0x22497b0 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd523fff700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 0x25164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd522ffd700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fd50c012db0 tx=0x7fd50c015000).ready entity=mon.1 client_cookie=f80e361ef9d1528a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c024030 con 0x22497b0 2020-11-16T22:33:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd50c00b6d0 con 0x22497b0 2020-11-16T22:33:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd514011710 con 0x22497b0 2020-11-16T22:33:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd50c01f650 con 0x22497b0 2020-11-16T22:33:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 10 client.7180 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x2204e00 con 0x22497b0 2020-11-16T22:33:28.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514011710 con 0x22497b0 2020-11-16T22:33:28.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fd50c033030 con 0x22497b0 2020-11-16T22:33:28.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 10 client.7180 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 10 client.7180 resolve_mds: resolved ID 'h' to GID 6798 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 20 client.7180 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x2462470 0x24b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 4 client.7180 mds_command: new command op to 6798 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 --> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x228cf50 con 0x2462470 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5237fe700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x2462470 0x24b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd4f9ffb700 10 client.7180 ms_handle_connect on v2:172.21.15.175:6840/2661622280 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.435+0000 7fd5237fe700 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x2462470 0x24b9d80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=5e4abcc280337f0a server_cookie=fb0a6816e44b7ae in_seq=0 out_seq=0 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd4f9ffb700 1 -- 172.21.15.181:0/272840746 <== mds.2 v2:172.21.15.175:6840/2661622280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x228cf50 con 0x2462470 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd4f9ffb700 10 client.7180 handle_command_reply: tid=0 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 client.7180 shutdown 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 msgr2=0x2519b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fd50c012db0 tx=0x7fd50c015000).stop 2020-11-16T22:33:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 shutdown_connections 2020-11-16T22:33:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:6840/2661622280,v1:172.21.15.175:6841/2661622280] conn(0x2462470 0x24b9d80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2417e40 0x25164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x21dbbb0 0x2461d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 --2- 172.21.15.181:0/272840746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22497b0 0x2519b20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd4f9ffb700 0 client.7180 ms_handle_reset on v2:172.21.15.175:6840/2661622280 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 >> 172.21.15.181:0/272840746 conn(0x7fd4f0002320 msgr2=0x2469140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 shutdown_connections 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 1 -- 172.21.15.181:0/272840746 wait complete. 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd5386fc740 20 client.7180 trim_cache size 0 max 16384 2020-11-16T22:33:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 msgr2=0x7fd500057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 0x7fd500057550 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fd514004250 tx=0x7fd514007e80).stop 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 msgr2=0x7fd524146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fd51c0067d0 tx=0x7fd51c008040).stop 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 shutdown_connections 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd500055090 0x7fd500057550 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd52415d8f0 0x7fd524146c90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5240b79f0 0x7fd5241701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 --2- 172.21.15.181:0/2419647070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd52415d510 0x7fd524147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 >> 172.21.15.181:0/2419647070 conn(0x7fd5240d3680 msgr2=0x7fd524155810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 shutdown_connections 2020-11-16T22:33:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:25.455+0000 7fd529c6b700 1 -- 172.21.15.181:0/2419647070 wait complete. 2020-11-16T22:33:28.451 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:33:28.452 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:33:28.452 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:33:28.452 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:33:28.452 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:28.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.599+0000 7f68597e5700 1 Processor -- start 2020-11-16T22:33:28.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.599+0000 7f68597e5700 1 -- start start 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68597e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68597e5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6854061810 0x7f6854160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68597e5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685415d180 con 0x7f685415fdc0 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68597e5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6854160730 con 0x7f6854061810 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68597e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685413f140 con 0x7f685413e870 2020-11-16T22:33:28.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42202/0 (socket says 172.21.15.181:42202) 2020-11-16T22:33:28.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 -- 172.21.15.181:0/3116243950 learned_addr learned my addr 172.21.15.181:0/3116243950 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:28.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6858fe4700 1 --2- 172.21.15.181:0/3116243950 >> v2:172.21.15.175:3300/0 conn(0x7f6854061810 0x7f6854160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 -- 172.21.15.181:0/3116243950 >> v2:172.21.15.175:3300/0 conn(0x7f6854061810 msgr2=0x7f6854160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:28.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 --2- 172.21.15.181:0/3116243950 >> v2:172.21.15.175:3300/0 conn(0x7f6854061810 0x7f6854160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 -- 172.21.15.181:0/3116243950 >> v1:172.21.15.175:6789/0 conn(0x7f685415fdc0 legacy=0x7f6854061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 -- 172.21.15.181:0/3116243950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685415d810 con 0x7f685413e870 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f6853fff700 1 --2- 172.21.15.181:0/3116243950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f6844006fd0 tx=0x7f6844006eb0).ready entity=mon.2 client_cookie=451e489079e438c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68537fe700 1 -- 172.21.15.181:0/3116243950 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440098d0 con 0x7f685413e870 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68537fe700 1 -- 172.21.15.181:0/3116243950 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844009a30 con 0x7f685413e870 2020-11-16T22:33:28.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.603+0000 7f68537fe700 1 -- 172.21.15.181:0/3116243950 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844009db0 con 0x7f685413e870 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- 172.21.15.181:0/3116243950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 msgr2=0x7f685413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 --2- 172.21.15.181:0/3116243950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f6844006fd0 tx=0x7f6844006eb0).stop 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- 172.21.15.181:0/3116243950 shutdown_connections 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 --2- 172.21.15.181:0/3116243950 >> v2:172.21.15.175:3300/0 conn(0x7f6854061810 0x7f6854160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 --2- 172.21.15.181:0/3116243950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685413e870 0x7f685413ec50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- 172.21.15.181:0/3116243950 >> 172.21.15.181:0/3116243950 conn(0x7f68540cf630 msgr2=0x7f68540cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:28.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- 172.21.15.181:0/3116243950 shutdown_connections 2020-11-16T22:33:28.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- 172.21.15.181:0/3116243950 wait complete. 2020-11-16T22:33:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 Processor -- start 2020-11-16T22:33:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.607+0000 7f68597e5700 1 -- start start 2020-11-16T22:33:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685413e870 0x7f685415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 0x7f685415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b7290 con 0x7f685413e870 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b6d50 con 0x7f6854061810 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b6ff0 con 0x7f685415fdc0 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6858fe4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 0x7f685415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34366/0 (socket says 172.21.15.181:34366) 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6858fe4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 0x7f685415b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42212/0 (socket says 172.21.15.181:42212) 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 -- 172.21.15.181:0/1319661624 learned_addr learned my addr 172.21.15.181:0/1319661624 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f684b7fe700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685413e870 0x7f685415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 -- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 msgr2=0x7f685415b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.802 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 0x7f685415b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 -- 172.21.15.181:0/1319661624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685413e870 msgr2=0x7f685415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685413e870 0x7f685415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 -- 172.21.15.181:0/1319661624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68540c7c00 con 0x7f6854061810 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6853fff700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f6844000dd0 tx=0x7f6844015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844011dd0 con 0x7f6854061810 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684401e460 con 0x7f6854061810 2020-11-16T22:33:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68540c7ea0 con 0x7f6854061810 2020-11-16T22:33:28.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684401e800 con 0x7f6854061810 2020-11-16T22:33:28.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.611+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68540b8b30 con 0x7f6854061810 2020-11-16T22:33:28.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6854044f60 con 0x7f6854061810 2020-11-16T22:33:28.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6844025030 con 0x7f6854061810 2020-11-16T22:33:28.804 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f6850ff9700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 0x7f6830057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f684406f680 con 0x7f6854061810 2020-11-16T22:33:29.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f684b7fe700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 0x7f6830057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.615+0000 7f684b7fe700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 0x7f6830057550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6854166b10 tx=0x7f683c007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.623+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f68440457d0 con 0x7f6854061810 2020-11-16T22:33:29.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.823+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685405c310 con 0x7f6854061810 2020-11-16T22:33:29.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f6850ff9700 1 -- 172.21.15.181:0/1319661624 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4665 (secure 0 0 0) 0x7f6844048df0 con 0x7f6854061810 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 msgr2=0x7f6830057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 0x7f6830057550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6854166b10 tx=0x7f683c007b90).stop 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 msgr2=0x7f68540b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f6844000dd0 tx=0x7f6844015000).stop 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 shutdown_connections 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6830055090 0x7f6830057550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685415fdc0 0x7f685415b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6854061810 0x7f68540b8640 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 --2- 172.21.15.181:0/1319661624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685413e870 0x7f685415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 >> 172.21.15.181:0/1319661624 conn(0x7f68540cf630 msgr2=0x7f68540d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 shutdown_connections 2020-11-16T22:33:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:28.827+0000 7f68597e5700 1 -- 172.21.15.181:0/1319661624 wait complete. 2020-11-16T22:33:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 88 2020-11-16T22:33:29.211 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:29.211 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":88,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:24.911953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:29.212 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub start /d1/d2/d3 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 Processor -- start 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 -- start start 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09d4167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09d40b3220 0x7f09d40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d4168390 con 0x7f09d4166bb0 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d4167770 con 0x7f09d40b3220 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09db881700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40b5f00 con 0x7f09d4167ac0 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f09d4166bb0 0x7f09d4167060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48678/0 (socket says 172.21.15.181:48678) 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 -- 172.21.15.181:0/2810120905 learned_addr learned my addr 172.21.15.181:0/2810120905 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d961d700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09d4167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/2810120905 >> v2:172.21.15.175:3300/0 conn(0x7f09d40b3220 0x7f09d40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089822556 0 0) 0x7f09d4168390 con 0x7f09d4166bb0 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b8003240 con 0x7f09d4166bb0 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2585447470 0 0) 0x7f09b8003240 con 0x7f09d4166bb0 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 msgr2=0x7f09d4167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09d4167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 >> v2:172.21.15.175:3300/0 conn(0x7f09d40b3220 msgr2=0x7f09d40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 --2- 172.21.15.181:0/2810120905 >> v2:172.21.15.175:3300/0 conn(0x7f09d40b3220 0x7f09d40b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d40b2060 con 0x7f09d4166bb0 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f09cc001860 con 0x7f09d4166bb0 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d961d700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09d4167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 >> v1:172.21.15.181:6789/0 conn(0x7f09d4166bb0 legacy=0x7f09d4167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8003760 0x7f09b8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80044e0 0x7f09b8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09b8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4167770 con 0x7f09b80044e0 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40b5f00 con 0x7f09b8003760 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8004260 con 0x7f09d4167ac0 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09b8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d961d700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8003760 0x7f09b8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80044e0 0x7f09b8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 -- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8003760 msgr2=0x7f09b8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8003760 0x7f09b8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 -- 172.21.15.181:0/2810120905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80044e0 msgr2=0x7f09b8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80044e0 0x7f09b8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 -- 172.21.15.181:0/2810120905 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8003240 con 0x7f09d4167ac0 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b80044e0 0x7f09b8004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09b8004e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f09cc00c840 tx=0x7f09d40b4e10).ready entity=mon.2 client_cookie=f83209585942cdce server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc005b80 con 0x7f09d4167ac0 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.351+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc005ce0 con 0x7f09d4167ac0 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09c7fff700 1 -- 172.21.15.181:0/2810120905 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0072e0 con 0x7f09d4167ac0 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 -- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 msgr2=0x7f09b8004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 --2- 172.21.15.181:0/2810120905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d4167ac0 0x7f09b8004e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f09cc00c840 tx=0x7f09d40b4e10).stop 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09d9e1e700 1 -- 172.21.15.181:0/2810120905 reap_dead start 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 -- 172.21.15.181:0/2810120905 shutdown_connections 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 -- 172.21.15.181:0/2810120905 >> 172.21.15.181:0/2810120905 conn(0x7f09d4060bc0 msgr2=0x7f09d40d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 -- 172.21.15.181:0/2810120905 shutdown_connections 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.355+0000 7f09db881700 1 -- 172.21.15.181:0/2810120905 wait complete. 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 Processor -- start 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- start start 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 0x7f09d41664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d40b03f0 0x7f09d4176050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d40b0870 0x7f09d4176540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4151b10 con 0x7f09d40b0870 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4174ed0 con 0x7f09d40b03f0 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 0x7f09d41664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d41669b0 con 0x7f09d41660e0 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 0x7f09d41664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42228/0 (socket says 172.21.15.181:42228) 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d961d700 1 -- 172.21.15.181:0/3259589872 learned_addr learned my addr 172.21.15.181:0/3259589872 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d40b03f0 0x7f09d4176050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d40b0870 0x7f09d4176540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 msgr2=0x7f09d41664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 0x7f09d41664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3259589872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d40b0870 msgr2=0x7f09d4176540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09d40b0870 0x7f09d4176540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3259589872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4175130 con 0x7f09d40b03f0 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d961d700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09d41660e0 0x7f09d41664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d40b03f0 0x7f09d4176050 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f09d0006dc0 tx=0x7f09d0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09c57fa700 1 -- 172.21.15.181:0/3259589872 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0017070 con 0x7f09d40b03f0 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09c57fa700 1 -- 172.21.15.181:0/3259589872 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0012570 con 0x7f09d40b03f0 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- 172.21.15.181:0/3259589872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d41753d0 con 0x7f09d40b03f0 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09c57fa700 1 -- 172.21.15.181:0/3259589872 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00128f0 con 0x7f09d40b03f0 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.359+0000 7f09db881700 1 -- 172.21.15.181:0/3259589872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d4175bb0 con 0x7f09d40b03f0 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.363+0000 7f09c57fa700 1 -- 172.21.15.181:0/3259589872 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f09d0012c60 con 0x7f09d40b03f0 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.363+0000 7f09c57fa700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09ac055060 0x7f09ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.363+0000 7f09d961d700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09ac055060 0x7f09ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.363+0000 7f09c57fa700 1 -- 172.21.15.181:0/3259589872 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09d006d9b0 con 0x7f09d40b03f0 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.363+0000 7f09d961d700 1 --2- 172.21.15.181:0/3259589872 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09ac055060 0x7f09ac057520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f09c8006a90 tx=0x7f09c8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 Processor -- start 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 -- start start 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e8bbf0 0x214f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x216ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa34f0 con 0x1e8bbf0 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2057970 con 0x1ef97f0 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f17730 con 0x21124b0 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x216ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34394/0 (socket says 172.21.15.181:34394) 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 -- 172.21.15.181:0/2474373170 learned_addr learned my addr 172.21.15.181:0/2474373170 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e8bbf0 0x214f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 -- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 msgr2=0x216ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x216ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 -- 172.21.15.181:0/2474373170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e8bbf0 msgr2=0x214f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e8bbf0 0x214f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 -- 172.21.15.181:0/2474373170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f125e0 con 0x1ef97f0 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.367+0000 7f09d961d700 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f09c8012f60 tx=0x7f09c8017000).ready entity=mon.1 client_cookie=30cf287a1d98d6a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09b67fc700 1 -- 172.21.15.181:0/2474373170 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8014e30 con 0x1ef97f0 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09b67fc700 1 -- 172.21.15.181:0/2474373170 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c801f440 con 0x1ef97f0 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09b67fc700 1 -- 172.21.15.181:0/2474373170 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c801f800 con 0x1ef97f0 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 msgr2=0x2110d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f09c8012f60 tx=0x7f09c8017000).stop 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- 172.21.15.181:0/2474373170 shutdown_connections 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x216ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x2110d10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 --2- 172.21.15.181:0/2474373170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e8bbf0 0x214f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- 172.21.15.181:0/2474373170 >> 172.21.15.181:0/2474373170 conn(0x20c7e80 msgr2=0x216c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- 172.21.15.181:0/2474373170 shutdown_connections 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- 172.21.15.181:0/2474373170 wait complete. 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 -- start start 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.371+0000 7f09ea312740 1 Processor -- start 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x2176ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef97f0 0x213b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2044720 con 0x1ef97f0 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2044860 con 0x1e8bbf0 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2045ad0 con 0x21124b0 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 10 client.0 ll_register_callbacks cb 0x203ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x2176ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42244/0 (socket says 172.21.15.181:42244) 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x2176ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34400/0 (socket says 172.21.15.181:34400) 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 -- 172.21.15.181:0/249501509 learned_addr learned my addr 172.21.15.181:0/249501509 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef97f0 0x213b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 -- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 msgr2=0x2176ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x2176ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 -- 172.21.15.181:0/249501509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef97f0 msgr2=0x213b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef97f0 0x213b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 -- 172.21.15.181:0/249501509 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x20352c0 con 0x21124b0 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f09cc00a000 tx=0x7f09cc015000).ready entity=mon.2 client_cookie=9e6c53d5ba392be8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc001450 con 0x21124b0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09cc001b00 con 0x21124b0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x210ce50 con 0x21124b0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc0072e0 con 0x21124b0 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 10 client.6608 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x1e6b010 con 0x21124b0 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x210ce50 con 0x21124b0 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f09cc013450 con 0x21124b0 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 10 client.6608 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 10 client.6608 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 20 client.6608 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2115330 0x2168fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 4 client.6608 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1f35c90 con 0x2115330 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d961d700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2115330 0x2168fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 10 client.6608 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09d961d700 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2115330 0x2168fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=bbe24c101c593c2f server_cookie=d5f5e5efadc1b356 in_seq=0 out_seq=0 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 1 -- 172.21.15.181:0/249501509 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1f35c90 con 0x2115330 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 10 client.6608 handle_command_reply: tid=0 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 client.6608 shutdown 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 msgr2=0x2167d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f09cc00a000 tx=0x7f09cc015000).stop 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 shutdown_connections 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2115330 0x2168fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x21124b0 0x2167d30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x2176ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09a37fe700 0 client.6608 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 --2- 172.21.15.181:0/249501509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef97f0 0x213b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 >> 172.21.15.181:0/249501509 conn(0x20c7e80 msgr2=0x213d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 shutdown_connections 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.375+0000 7f09ea312740 1 -- 172.21.15.181:0/249501509 wait complete. 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.379+0000 7f09ea312740 20 client.6608 trim_cache size 0 max 16384 2020-11-16T22:33:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 Processor -- start 2020-11-16T22:33:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- start start 2020-11-16T22:33:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x2166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x213e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2112b60 con 0x20c7e80 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x201ce80 con 0x1e8bbf0 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x214fb60 con 0x1ef97f0 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42250/0 (socket says 172.21.15.181:42250) 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 -- 172.21.15.181:0/634890843 learned_addr learned my addr 172.21.15.181:0/634890843 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x213e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d961d700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x2166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 -- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 msgr2=0x213e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x213e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 -- 172.21.15.181:0/634890843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 msgr2=0x2166ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x2166ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 -- 172.21.15.181:0/634890843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80081d0 con 0x1ef97f0 2020-11-16T22:33:29.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d961d700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x2166ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:29.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f09cc00e240 tx=0x7f09cc021000).ready entity=mon.2 client_cookie=3c8999ce697be296 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09b6ffd700 1 -- 172.21.15.181:0/634890843 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc001450 con 0x1ef97f0 2020-11-16T22:33:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09b6ffd700 1 -- 172.21.15.181:0/634890843 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc0015b0 con 0x1ef97f0 2020-11-16T22:33:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09b6ffd700 1 -- 172.21.15.181:0/634890843 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc001cd0 con 0x1ef97f0 2020-11-16T22:33:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 msgr2=0x213e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f09cc00e240 tx=0x7f09cc021000).stop 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- 172.21.15.181:0/634890843 shutdown_connections 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ef97f0 0x213e860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x2166ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 --2- 172.21.15.181:0/634890843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e8bbf0 0x213e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- 172.21.15.181:0/634890843 >> 172.21.15.181:0/634890843 conn(0x7f09a8002320 msgr2=0x2151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- 172.21.15.181:0/634890843 shutdown_connections 2020-11-16T22:33:29.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.399+0000 7f09ea312740 1 -- 172.21.15.181:0/634890843 wait complete. 2020-11-16T22:33:29.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 -- start start 2020-11-16T22:33:29.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 Processor -- start 2020-11-16T22:33:29.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e8bbf0 0x2114830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x213e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x213e580 con 0x20c7e80 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x213e7c0 con 0x1ef97f0 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x213ea60 con 0x1e8bbf0 2020-11-16T22:33:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09ea312740 10 client.0 ll_register_callbacks cb 0x2151470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d961d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e8bbf0 0x2114830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34416/0 (socket says 172.21.15.181:34416) 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3065829343 learned_addr learned my addr 172.21.15.181:0/3065829343 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d9e1e700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x213e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e8bbf0 msgr2=0x2114830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e8bbf0 0x2114830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3065829343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 msgr2=0x213e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x213e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 -- 172.21.15.181:0/3065829343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ea9760 con 0x1ef97f0 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09d8e1c700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f09d0043d50 tx=0x7f09d0015000).ready entity=mon.1 client_cookie=e652ac88e2337758 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.403+0000 7f09a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d002d070 con 0x1ef97f0 2020-11-16T22:33:29.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09d0046a40 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c801f440 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09d0046c00 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 10 client.7201 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x1eb4e40 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c801f440 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f09d006d400 con 0x1ef97f0 2020-11-16T22:33:29.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 10 client.7201 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 10 client.7201 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 20 client.7201 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x21124b0 0x213c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 4 client.7201 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",}] 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " }) v1 -- 0x1f3cf90 con 0x21124b0 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09d961d700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x21124b0 0x213c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 10 client.7201 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09d961d700 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x21124b0 0x213c5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=6d337563e5f3d32a server_cookie=37befba057f89a60 in_seq=0 out_seq=0 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 1 -- 172.21.15.181:0/3065829343 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x1f3cf90 con 0x21124b0 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 10 client.7201 handle_command_reply: tid=0 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 client.7201 shutdown 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 msgr2=0x21c0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f09d0043d50 tx=0x7f09d0015000).stop 2020-11-16T22:33:29.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 shutdown_connections 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x21124b0 0x213c5b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20c7e80 0x213e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09a37fe700 0 client.7201 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e8bbf0 0x2114830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 --2- 172.21.15.181:0/3065829343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef97f0 0x21c0c30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 >> 172.21.15.181:0/3065829343 conn(0x7f09a8002320 msgr2=0x213b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 shutdown_connections 2020-11-16T22:33:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 1 -- 172.21.15.181:0/3065829343 wait complete. 2020-11-16T22:33:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.407+0000 7f09ea312740 20 client.7201 trim_cache size 0 max 16384 2020-11-16T22:33:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:Error EXDEV: Not rank 0 2020-11-16T22:33:29.405 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:33:29.406 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub abort 2020-11-16T22:33:29.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 Processor -- start 2020-11-16T22:33:29.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 -- start start 2020-11-16T22:33:29.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a0061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a0160c40 con 0x7f49a0142a40 2020-11-16T22:33:29.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a0062700 con 0x7f49a0142660 2020-11-16T22:33:29.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a6cc6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a00629a0 con 0x7f49a0063620 2020-11-16T22:33:29.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a64c5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a5cc4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f49a0142660 0x7f49a0063ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48726/0 (socket says 172.21.15.181:48726) 2020-11-16T22:33:29.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a5cc4700 1 -- 172.21.15.181:0/2187594509 learned_addr learned my addr 172.21.15.181:0/2187594509 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:29.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920707443 0 0) 0x7f49a0062700 con 0x7f49a0142660 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4988003240 con 0x7f49a0142660 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861428061 0 0) 0x7f49a0160c40 con 0x7f49a0142a40 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a0062700 con 0x7f49a0142a40 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a64c5700 1 -- 172.21.15.181:0/2187594509 >> v1:172.21.15.181:6789/0 conn(0x7f49a0142660 legacy=0x7f49a0063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.547+0000 7f49a64c5700 1 -- 172.21.15.181:0/2187594509 >> v1:172.21.15.175:6789/0 conn(0x7f49a0142a40 legacy=0x7f49a0160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a64c5700 1 -- 172.21.15.181:0/2187594509 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0062c40 con 0x7f49a0063620 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a64c5700 1 --2- 172.21.15.181:0/2187594509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a0061810 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f4998009cc0 tx=0x7f49980091d0).ready entity=mon.2 client_cookie=b5359d3ad0f76b1f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800ba20 con 0x7f49a0063620 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499800bb80 con 0x7f49a0063620 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a4cc2700 1 -- 172.21.15.181:0/2187594509 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998011400 con 0x7f49a0063620 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- 172.21.15.181:0/2187594509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 msgr2=0x7f49a0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 --2- 172.21.15.181:0/2187594509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a0061810 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f4998009cc0 tx=0x7f49980091d0).stop 2020-11-16T22:33:29.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- 172.21.15.181:0/2187594509 shutdown_connections 2020-11-16T22:33:29.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 --2- 172.21.15.181:0/2187594509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a0061810 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- 172.21.15.181:0/2187594509 >> 172.21.15.181:0/2187594509 conn(0x7f49a00d2dc0 msgr2=0x7f49a00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- 172.21.15.181:0/2187594509 shutdown_connections 2020-11-16T22:33:29.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- 172.21.15.181:0/2187594509 wait complete. 2020-11-16T22:33:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 Processor -- start 2020-11-16T22:33:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- start start 2020-11-16T22:33:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a00b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0142660 0x7f49a01601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49a0142a40 0x7f49a01606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a0172530 con 0x7f49a0142a40 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a01727d0 con 0x7f49a0142660 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a6cc6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a0172a70 con 0x7f49a0063620 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0142660 0x7f49a01601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0142660 0x7f49a01601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34430/0 (socket says 172.21.15.181:34430) 2020-11-16T22:33:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a54c3700 1 -- 172.21.15.181:0/582824030 learned_addr learned my addr 172.21.15.181:0/582824030 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a00b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.551+0000 7f49a64c5700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49a0142a40 0x7f49a01606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 -- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0142660 msgr2=0x7f49a01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0142660 0x7f49a01601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 -- 172.21.15.181:0/582824030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49a0142a40 msgr2=0x7f49a01606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49a0142a40 0x7f49a01606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 -- 172.21.15.181:0/582824030 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0172d10 con 0x7f49a0063620 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49a0063620 0x7f49a00b1de0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f499c00c6e0 tx=0x7f499c0079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 -- 172.21.15.181:0/582824030 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c0072e0 con 0x7f49a0063620 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 -- 172.21.15.181:0/582824030 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c007440 con 0x7f49a0063620 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 -- 172.21.15.181:0/582824030 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c005820 con 0x7f49a0063620 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a6cc6700 1 -- 172.21.15.181:0/582824030 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a0172fb0 con 0x7f49a0063620 2020-11-16T22:33:29.548 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a6cc6700 1 -- 172.21.15.181:0/582824030 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a00b22d0 con 0x7f49a0063620 2020-11-16T22:33:29.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 -- 172.21.15.181:0/582824030 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f499c010070 con 0x7f49a0063620 2020-11-16T22:33:29.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4978055060 0x7f4978057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f4991ffb700 1 -- 172.21.15.181:0/582824030 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f499c06d350 con 0x7f49a0063620 2020-11-16T22:33:29.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.555+0000 7f49a54c3700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4978055060 0x7f4978057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 --2- 172.21.15.181:0/582824030 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4978055060 0x7f4978057520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f49a0064ed0 tx=0x7f4994007a90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 Processor -- start 2020-11-16T22:33:29.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 -- start start 2020-11-16T22:33:29.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e7b7a0 0x209c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x20936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f254a0 con 0x1e7b7a0 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd9920 con 0x2099fc0 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e996e0 con 0x1e0dba0 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42278/0 (socket says 172.21.15.181:42278) 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 -- 172.21.15.181:0/1728291267 learned_addr learned my addr 172.21.15.181:0/1728291267 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a64c5700 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x20936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e7b7a0 0x209c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 -- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 msgr2=0x20936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x20936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 -- 172.21.15.181:0/1728291267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e7b7a0 msgr2=0x209c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e7b7a0 0x209c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.559+0000 7f49a54c3700 1 -- 172.21.15.181:0/1728291267 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e94590 con 0x1e0dba0 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49a54c3700 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f4994015b80 tx=0x7f499400a000).ready entity=mon.2 client_cookie=c27219b41825e1f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f497effd700 1 -- 172.21.15.181:0/1728291267 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994017a70 con 0x1e0dba0 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f497effd700 1 -- 172.21.15.181:0/1728291267 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994017bd0 con 0x1e0dba0 2020-11-16T22:33:29.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f497effd700 1 -- 172.21.15.181:0/1728291267 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940123d0 con 0x1e0dba0 2020-11-16T22:33:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 msgr2=0x209a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f4994015b80 tx=0x7f499400a000).stop 2020-11-16T22:33:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- 172.21.15.181:0/1728291267 shutdown_connections 2020-11-16T22:33:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x20936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e7b7a0 0x209c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1728291267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e0dba0 0x209a660 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- 172.21.15.181:0/1728291267 >> 172.21.15.181:0/1728291267 conn(0x2049e30 msgr2=0x2096390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- 172.21.15.181:0/1728291267 shutdown_connections 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- 172.21.15.181:0/1728291267 wait complete. 2020-11-16T22:33:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- start start 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 Processor -- start 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x20b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x2096360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fc66d0 con 0x1e0dba0 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fc6810 con 0x2099fc0 2020-11-16T22:33:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fc7a80 con 0x1e7b7a0 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.563+0000 7f49b69b9740 10 client.0 ll_register_callbacks cb 0x1fbcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a64c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x2096360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42286/0 (socket says 172.21.15.181:42286) 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 -- 172.21.15.181:0/150157836 learned_addr learned my addr 172.21.15.181:0/150157836 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x20b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 -- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 msgr2=0x2096360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x2096360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 -- 172.21.15.181:0/150157836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 msgr2=0x20b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x20b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 -- 172.21.15.181:0/150157836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1fb7270 con 0x1e7b7a0 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a54c3700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f4994012000 tx=0x7f499401b000).ready entity=mon.2 client_cookie=13f501dba4c63212 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994027070 con 0x1e7b7a0 2020-11-16T22:33:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4994012800 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x207a6b0 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4994012bd0 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 10 client.6626 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x1decfc0 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x207a6b0 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f499401e070 con 0x1e7b7a0 2020-11-16T22:33:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 10 client.6626 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 10 client.6626 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 20 client.6626 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x20fc940 0x20fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 4 client.6626 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1eb7c40 con 0x20fc940 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x20fc940 0x20fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 10 client.6626 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x20fc940 0x20fcd20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=4ccd64e9ce043f05 server_cookie=c278d7328d106411 in_seq=0 out_seq=0 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 1 -- 172.21.15.181:0/150157836 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1eb7c40 con 0x20fc940 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 10 client.6626 handle_command_reply: tid=0 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 client.6626 shutdown 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 msgr2=0x2095e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f4994012000 tx=0x7f499401b000).stop 2020-11-16T22:33:29.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 shutdown_connections 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x20fc940 0x20fcd20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2099fc0 0x2096360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x20b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 --2- 172.21.15.181:0/150157836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x2095e70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 >> 172.21.15.181:0/150157836 conn(0x2049e30 msgr2=0x20cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f4973fff700 0 client.6626 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 shutdown_connections 2020-11-16T22:33:29.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 1 -- 172.21.15.181:0/150157836 wait complete. 2020-11-16T22:33:29.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.567+0000 7f49b69b9740 20 client.6626 trim_cache size 0 max 16384 2020-11-16T22:33:29.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.587+0000 7f49b69b9740 1 Processor -- start 2020-11-16T22:33:29.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.587+0000 7f49b69b9740 1 -- start start 2020-11-16T22:33:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x208ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e7b7a0 0x2098d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f9ee30 con 0x1e0dba0 2020-11-16T22:33:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x20eaa00 con 0x1e7b7a0 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x20eae90 con 0x2049e30 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a64c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e7b7a0 0x2098d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42294/0 (socket says 172.21.15.181:42294) 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 -- 172.21.15.181:0/1062651663 learned_addr learned my addr 172.21.15.181:0/1062651663 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a54c3700 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x208ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 -- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e7b7a0 msgr2=0x2098d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e7b7a0 0x2098d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 -- 172.21.15.181:0/1062651663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 msgr2=0x208ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x208ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 -- 172.21.15.181:0/1062651663 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c0081d0 con 0x2049e30 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f499c046ed0 tx=0x7f499c049040).ready entity=mon.2 client_cookie=7254688a3de6d50e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f497f7fe700 1 -- 172.21.15.181:0/1062651663 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c0453b0 con 0x2049e30 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f497f7fe700 1 -- 172.21.15.181:0/1062651663 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c045610 con 0x2049e30 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f497f7fe700 1 -- 172.21.15.181:0/1062651663 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c043980 con 0x2049e30 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 msgr2=0x20fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f499c046ed0 tx=0x7f499c049040).stop 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- 172.21.15.181:0/1062651663 shutdown_connections 2020-11-16T22:33:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e7b7a0 0x2098d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2049e30 0x20fb220 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 --2- 172.21.15.181:0/1062651663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e0dba0 0x208ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- 172.21.15.181:0/1062651663 >> 172.21.15.181:0/1062651663 conn(0x7f496c002320 msgr2=0x20ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- 172.21.15.181:0/1062651663 shutdown_connections 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- 172.21.15.181:0/1062651663 wait complete. 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 -- start start 2020-11-16T22:33:29.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.591+0000 7f49b69b9740 1 Processor -- start 2020-11-16T22:33:29.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x20fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2049e30 0x2148f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x20e7560 con 0x2049e30 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x20fb6f0 con 0x1e0dba0 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x208ef10 con 0x1e7b7a0 2020-11-16T22:33:29.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 10 client.0 ll_register_callbacks cb 0x209a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a54c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x20fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34458/0 (socket says 172.21.15.181:34458) 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 -- 172.21.15.181:0/3010245882 learned_addr learned my addr 172.21.15.181:0/3010245882 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a64c5700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2049e30 0x2148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 -- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 msgr2=0x20fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x20fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 -- 172.21.15.181:0/3010245882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2049e30 msgr2=0x2148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2049e30 0x2148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 -- 172.21.15.181:0/3010245882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e2b710 con 0x1e0dba0 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49a5cc4700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f499c046f70 tx=0x7f499c055000).ready entity=mon.1 client_cookie=6b505663216f57f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c06bb20 con 0x1e0dba0 2020-11-16T22:33:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f499c06bc80 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4994017bd0 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f499c06be40 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 10 client.7219 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x1e36df0 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994017bd0 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f499c01b020 con 0x1e0dba0 2020-11-16T22:33:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 10 client.7219 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 10 client.7219 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 20 client.7219 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2099fc0 0x20ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 4 client.7219 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,a,b,o,r,t,",}] 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.595+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " s c r u b a b o r t " }) v1 -- 0x1ebef40 con 0x2099fc0 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49a54c3700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2099fc0 0x20ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49a54c3700 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2099fc0 0x20ca900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=2dd96852813b11ad server_cookie=c4a5e8cfbe1ad4ae in_seq=0 out_seq=0 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f4973fff700 10 client.7219 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f4973fff700 1 -- 172.21.15.181:0/3010245882 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x1ebef40 con 0x2099fc0 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f4973fff700 10 client.7219 handle_command_reply: tid=0 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 client.7219 shutdown 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 msgr2=0x208e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f499c046f70 tx=0x7f499c055000).stop 2020-11-16T22:33:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 shutdown_connections 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x2099fc0 0x20ca900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2049e30 0x2148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f4973fff700 0 client.7219 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e0dba0 0x208e960 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 --2- 172.21.15.181:0/3010245882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e7b7a0 0x20fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 >> 172.21.15.181:0/3010245882 conn(0x7f496c002320 msgr2=0x20cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 shutdown_connections 2020-11-16T22:33:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 1 -- 172.21.15.181:0/3010245882 wait complete. 2020-11-16T22:33:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.599+0000 7f49b69b9740 20 client.7219 trim_cache size 0 max 16384 2020-11-16T22:33:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:Error EXDEV: Not rank 0 2020-11-16T22:33:29.596 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:33:29.597 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub pause 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 Processor -- start 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 -- start start 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fabd01606c0 0x7fabd0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c8e30 0x7fabd00b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabd00c9290 0x7fabd00b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd0160f90 con 0x7fabd01606c0 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00c9800 con 0x7fabd00c9290 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd65d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00c9aa0 con 0x7fabd00c8e30 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabd00c9290 0x7fabd00b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c8e30 0x7fabd00b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabd00c9290 0x7fabd00b7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34468/0 (socket says 172.21.15.181:34468) 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 -- 172.21.15.181:0/2477705212 learned_addr learned my addr 172.21.15.181:0/2477705212 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd55d4700 1 --2- 172.21.15.181:0/2477705212 >> v2:172.21.15.175:3300/0 conn(0x7fabd01606c0 0x7fabd0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 -- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c8e30 msgr2=0x7fabd00b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c8e30 0x7fabd00b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 -- 172.21.15.181:0/2477705212 >> v2:172.21.15.175:3300/0 conn(0x7fabd01606c0 msgr2=0x7fabd0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/2477705212 >> v2:172.21.15.175:3300/0 conn(0x7fabd01606c0 0x7fabd0160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 -- 172.21.15.181:0/2477705212 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00c9d40 con 0x7fabd00c9290 2020-11-16T22:33:29.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c9290 0x7fabd00b7680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fabd0062480 tx=0x7fabc8006eb0).ready entity=mon.1 client_cookie=b18afd7571e327c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:29.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabcffff700 1 -- 172.21.15.181:0/2477705212 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8009720 con 0x7fabd00c9290 2020-11-16T22:33:29.936 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabcffff700 1 -- 172.21.15.181:0/2477705212 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc8009880 con 0x7fabd00c9290 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.739+0000 7fabcffff700 1 -- 172.21.15.181:0/2477705212 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8009c00 con 0x7fabd00c9290 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c9290 msgr2=0x7fabd00b7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 --2- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c9290 0x7fabd00b7680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fabd0062480 tx=0x7fabc8006eb0).stop 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- 172.21.15.181:0/2477705212 shutdown_connections 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 --2- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c9290 0x7fabd00b7680 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 --2- 172.21.15.181:0/2477705212 >> v2:172.21.15.175:3300/0 conn(0x7fabd01606c0 0x7fabd0160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 --2- 172.21.15.181:0/2477705212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c8e30 0x7fabd00b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:29.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- 172.21.15.181:0/2477705212 >> 172.21.15.181:0/2477705212 conn(0x7fabd00d33f0 msgr2=0x7fabd00d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:29.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- 172.21.15.181:0/2477705212 shutdown_connections 2020-11-16T22:33:29.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- 172.21.15.181:0/2477705212 wait complete. 2020-11-16T22:33:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 Processor -- start 2020-11-16T22:33:29.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.743+0000 7fabd65d6700 1 -- start start 2020-11-16T22:33:29.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c8e30 0x7fabd00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c9290 0x7fabd00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabd01606c0 0x7fabd00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015dfd0 con 0x7fabd01606c0 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015e270 con 0x7fabd00c8e30 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015e510 con 0x7fabd00c9290 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c9290 0x7fabd00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c8e30 0x7fabd00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c9290 0x7fabd00ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42314/0 (socket says 172.21.15.181:42314) 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 -- 172.21.15.181:0/1515939992 learned_addr learned my addr 172.21.15.181:0/1515939992 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabd01606c0 0x7fabd00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 -- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c8e30 msgr2=0x7fabd00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabd00c8e30 0x7fabd00d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 -- 172.21.15.181:0/1515939992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabd01606c0 msgr2=0x7fabd00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabd01606c0 0x7fabd00cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 -- 172.21.15.181:0/1515939992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd015e7b0 con 0x7fabd00c9290 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabd00c9290 0x7fabd00ced10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fabc000ecf0 tx=0x7fabc00091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabcd7fa700 1 -- 172.21.15.181:0/1515939992 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000ca10 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabcd7fa700 1 -- 172.21.15.181:0/1515939992 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000cb70 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 -- 172.21.15.181:0/1515939992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd015ea50 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabcd7fa700 1 -- 172.21.15.181:0/1515939992 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00123f0 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabd65d6700 1 -- 172.21.15.181:0/1515939992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd00cf6f0 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.747+0000 7fabcd7fa700 1 -- 172.21.15.181:0/1515939992 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fabc0022070 con 0x7fabd00c9290 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabcd7fa700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabac055060 0x7fabac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabcd7fa700 1 -- 172.21.15.181:0/1515939992 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabc006e380 con 0x7fabd00c9290 2020-11-16T22:33:30.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabac055060 0x7fabac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- 172.21.15.181:0/1515939992 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fabac055060 0x7fabac057520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fabb8006fd0 tx=0x7fabb8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:30.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 Processor -- start 2020-11-16T22:33:30.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 -- start start 2020-11-16T22:33:30.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xec1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xeba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4c4a0 con 0xec0fc0 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe00920 con 0xca27a0 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xcc06e0 con 0xc34ba0 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xec1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34478/0 (socket says 172.21.15.181:34478) 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 -- 172.21.15.181:0/3947427354 learned_addr learned my addr 172.21.15.181:0/3947427354 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xeba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 -- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 msgr2=0xec1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xec1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 -- 172.21.15.181:0/3947427354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 msgr2=0xeba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xeba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 -- 172.21.15.181:0/3947427354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcbb590 con 0xca27a0 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xec1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.751+0000 7fabd55d4700 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fabb8014e50 tx=0x7fabb8013000).ready entity=mon.1 client_cookie=ab25bdac48fa9476 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabc67fc700 1 -- 172.21.15.181:0/3947427354 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8011a50 con 0xca27a0 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabc67fc700 1 -- 172.21.15.181:0/3947427354 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb8011bb0 con 0xca27a0 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabc67fc700 1 -- 172.21.15.181:0/3947427354 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8020410 con 0xca27a0 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 msgr2=0xec3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fabb8014e50 tx=0x7fabb8013000).stop 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- 172.21.15.181:0/3947427354 shutdown_connections 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xeba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xca27a0 0xec3710 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 --2- 172.21.15.181:0/3947427354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xec1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- 172.21.15.181:0/3947427354 >> 172.21.15.181:0/3947427354 conn(0xe70e30 msgr2=0xebd390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- 172.21.15.181:0/3947427354 shutdown_connections 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- 172.21.15.181:0/3947427354 wait complete. 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 -- start start 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.755+0000 7fabe62c9740 1 Processor -- start 2020-11-16T22:33:30.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 0xec2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:30.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xded6d0 con 0xec0fc0 2020-11-16T22:33:30.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xded810 con 0xc34ba0 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xdeea80 con 0xec2a80 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 10 client.0 ll_register_callbacks cb 0xde3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd5dd5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 0xec2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34484/0 (socket says 172.21.15.181:34484) 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 -- 172.21.15.181:0/4042857250 learned_addr learned my addr 172.21.15.181:0/4042857250 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 -- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 msgr2=0xec2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 0xec2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 -- 172.21.15.181:0/4042857250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 msgr2=0xebd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:30.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:30.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 -- 172.21.15.181:0/4042857250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdde270 con 0xc34ba0 2020-11-16T22:33:30.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 0xec2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd55d4700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fabb800b3b0 tx=0x7fabb801e000).ready entity=mon.1 client_cookie=709d7d4824a3ca92 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8020800 con 0xc34ba0 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabb8020960 con 0xc34ba0 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xea16b0 con 0xc34ba0 2020-11-16T22:33:30.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb8020d30 con 0xc34ba0 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 10 client.7231 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0xc13fc0 con 0xc34ba0 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xea16b0 con 0xc34ba0 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fabb8013460 con 0xc34ba0 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 10 client.7231 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 10 client.7231 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 20 client.7231 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xf24190 0xf20400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 4 client.7231 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xcdec40 con 0xf24190 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xf24190 0xf20400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fab9f7fe700 10 client.7231 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.759+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xf24190 0xf20400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=d8ad8bfbf0cfd997 server_cookie=e12128838b54b993 in_seq=0 out_seq=0 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fab9f7fe700 1 -- 172.21.15.181:0/4042857250 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xcdec40 con 0xf24190 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fab9f7fe700 10 client.7231 handle_command_reply: tid=0 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 client.7231 shutdown 2020-11-16T22:33:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 msgr2=0xf15100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fabb800b3b0 tx=0x7fabb801e000).stop 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 shutdown_connections 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xf24190 0xf20400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xec2a80 0xec2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf15100 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 --2- 172.21.15.181:0/4042857250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 >> 172.21.15.181:0/4042857250 conn(0xe70e30 msgr2=0xef4650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:31.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fab9f7fe700 0 client.7231 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:31.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 shutdown_connections 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 1 -- 172.21.15.181:0/4042857250 wait complete. 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.763+0000 7fabe62c9740 20 client.7231 trim_cache size 0 max 16384 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 Processor -- start 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- start start 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 0xec20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xf23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xdc5e30 con 0xe70e30 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xebd6a0 con 0xec0fc0 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf1fcb0 con 0xc34ba0 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34492/0 (socket says 172.21.15.181:34492) 2020-11-16T22:33:31.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 -- 172.21.15.181:0/1900718110 learned_addr learned my addr 172.21.15.181:0/1900718110 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:31.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xf23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd55d4700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 0xec20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 -- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 msgr2=0xf23890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xf23890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 -- 172.21.15.181:0/1900718110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 msgr2=0xec20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 0xec20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 -- 172.21.15.181:0/1900718110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba40081d0 con 0xec0fc0 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd55d4700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 0xec20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fabd00624b0 tx=0x7fabc8015000).ready entity=mon.1 client_cookie=7e027c3cb24cea22 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabc6ffd700 1 -- 172.21.15.181:0/1900718110 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8024070 con 0xec0fc0 2020-11-16T22:33:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabc6ffd700 1 -- 172.21.15.181:0/1900718110 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc8025040 con 0xec0fc0 2020-11-16T22:33:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabc6ffd700 1 -- 172.21.15.181:0/1900718110 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8009c00 con 0xec0fc0 2020-11-16T22:33:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 msgr2=0xec0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fabd00624b0 tx=0x7fabc8015000).stop 2020-11-16T22:33:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- 172.21.15.181:0/1900718110 shutdown_connections 2020-11-16T22:33:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xec0fc0 0xec0220 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe70e30 0xec20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 --2- 172.21.15.181:0/1900718110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc34ba0 0xf23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- 172.21.15.181:0/1900718110 >> 172.21.15.181:0/1900718110 conn(0x7faba4002320 msgr2=0xeb5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:31.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- 172.21.15.181:0/1900718110 shutdown_connections 2020-11-16T22:33:31.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.779+0000 7fabe62c9740 1 -- 172.21.15.181:0/1900718110 wait complete. 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- start start 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 Processor -- start 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf241d0 con 0xec0fc0 2020-11-16T22:33:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xf216f0 con 0xc34ba0 2020-11-16T22:33:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xece740 con 0xe70e30 2020-11-16T22:33:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 10 client.0 ll_register_callbacks cb 0xec1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd55d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42342/0 (socket says 172.21.15.181:42342) 2020-11-16T22:33:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 -- 172.21.15.181:0/853139344 learned_addr learned my addr 172.21.15.181:0/853139344 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 -- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 msgr2=0xf7f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf7f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 -- 172.21.15.181:0/853139344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 msgr2=0xebb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 -- 172.21.15.181:0/853139344 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc52710 con 0xe70e30 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd5dd5700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd4dd3700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fabc0009fd0 tx=0x7fabc001d040).ready entity=mon.2 client_cookie=7e5a6850c9856444 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0046710 con 0xe70e30 2020-11-16T22:33:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabc00449b0 con 0xe70e30 2020-11-16T22:33:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabb8011bb0 con 0xe70e30 2020-11-16T22:33:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabc0044b70 con 0xe70e30 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 10 client.6650 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0xc5ddf0 con 0xe70e30 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb8011bb0 con 0xe70e30 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fabc006cb50 con 0xe70e30 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 10 client.6650 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 10 client.6650 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 20 client.6650 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xec2a80 0xf7b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 4 client.6650 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0xce5f40 con 0xec2a80 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd55d4700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xec2a80 0xf7b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fab9f7fe700 10 client.6650 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.783+0000 7fabd55d4700 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xec2a80 0xf7b0e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=622ce7a15b86536 server_cookie=206253f22d61be78 in_seq=0 out_seq=0 2020-11-16T22:33:32.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fab9f7fe700 1 -- 172.21.15.181:0/853139344 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0xce5f40 con 0xec2a80 2020-11-16T22:33:32.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fab9f7fe700 10 client.6650 handle_command_reply: tid=0 2020-11-16T22:33:32.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 client.6650 shutdown 2020-11-16T22:33:32.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 msgr2=0xebb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fabc0009fd0 tx=0x7fabc001d040).stop 2020-11-16T22:33:32.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 shutdown_connections 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xec2a80 0xf7b0e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xec0fc0 0xebb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fab9f7fe700 0 client.6650 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xc34ba0 0xf7f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 --2- 172.21.15.181:0/853139344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe70e30 0xebb0c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 >> 172.21.15.181:0/853139344 conn(0x7faba4002320 msgr2=0xf14510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 shutdown_connections 2020-11-16T22:33:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 1 -- 172.21.15.181:0/853139344 wait complete. 2020-11-16T22:33:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:29.787+0000 7fabe62c9740 20 client.6650 trim_cache size 0 max 16384 2020-11-16T22:33:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:Error EXDEV: Not rank 0 2020-11-16T22:33:32.404 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:33:32.405 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub resume 2020-11-16T22:33:32.406 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:32.413 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:32.415 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:33:32.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 Processor -- start 2020-11-16T22:33:32.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 -- start start 2020-11-16T22:33:32.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5640164e10 0x7f5640062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56400c8db0 con 0x7f5640161860 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56400c8330 con 0x7f5640164e10 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400c85d0 con 0x7f5640161480 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42348/0 (socket says 172.21.15.181:42348) 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563e59c700 1 -- 172.21.15.181:0/1559954585 learned_addr learned my addr 172.21.15.181:0/1559954585 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.555+0000 7f563dd9b700 1 --2- 172.21.15.181:0/1559954585 >> v2:172.21.15.175:3300/0 conn(0x7f5640164e10 0x7f5640062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563e59c700 1 -- 172.21.15.181:0/1559954585 >> v2:172.21.15.175:3300/0 conn(0x7f5640164e10 msgr2=0x7f5640062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563e59c700 1 --2- 172.21.15.181:0/1559954585 >> v2:172.21.15.175:3300/0 conn(0x7f5640164e10 0x7f5640062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563e59c700 1 -- 172.21.15.181:0/1559954585 >> v1:172.21.15.175:6789/0 conn(0x7f5640161860 legacy=0x7f5640062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563e59c700 1 -- 172.21.15.181:0/1559954585 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400c8870 con 0x7f5640161480 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563e59c700 1 --2- 172.21.15.181:0/1559954585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5628006fd0 tx=0x7f5628006eb0).ready entity=mon.2 client_cookie=916a89a78d3ee19f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.549 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563d59a700 1 -- 172.21.15.181:0/1559954585 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628009810 con 0x7f5640161480 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563d59a700 1 -- 172.21.15.181:0/1559954585 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628009970 con 0x7f5640161480 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563d59a700 1 -- 172.21.15.181:0/1559954585 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628009cf0 con 0x7f5640161480 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- 172.21.15.181:0/1559954585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 msgr2=0x7f5640164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 --2- 172.21.15.181:0/1559954585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5628006fd0 tx=0x7f5628006eb0).stop 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- 172.21.15.181:0/1559954585 shutdown_connections 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 --2- 172.21.15.181:0/1559954585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f5640164920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 --2- 172.21.15.181:0/1559954585 >> v2:172.21.15.175:3300/0 conn(0x7f5640164e10 0x7f5640062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- 172.21.15.181:0/1559954585 >> 172.21.15.181:0/1559954585 conn(0x7f56400d2dc0 msgr2=0x7f56400ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- 172.21.15.181:0/1559954585 shutdown_connections 2020-11-16T22:33:32.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- 172.21.15.181:0/1559954585 wait complete. 2020-11-16T22:33:32.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 Processor -- start 2020-11-16T22:33:32.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.559+0000 7f563f59e700 1 -- start start 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f56400c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5640161860 0x7f56400d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400d1b90 0x7f56400cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5640165f50 con 0x7f5640161860 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5640160f20 con 0x7f56400d1b90 2020-11-16T22:33:32.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400d1fd0 con 0x7f5640161480 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f56400c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400d1b90 0x7f56400cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f56400c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42354/0 (socket says 172.21.15.181:42354) 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400d1b90 0x7f56400cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34516/0 (socket says 172.21.15.181:34516) 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563e59c700 1 -- 172.21.15.181:0/590369926 learned_addr learned my addr 172.21.15.181:0/590369926 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563dd9b700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5640161860 0x7f56400d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 -- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 msgr2=0x7f56400c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5640161480 0x7f56400c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 -- 172.21.15.181:0/590369926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5640161860 msgr2=0x7f56400d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5640161860 0x7f56400d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 -- 172.21.15.181:0/590369926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400c9b50 con 0x7f56400d1b90 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563ed9d700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400d1b90 0x7f56400cb830 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f5640161c90 tx=0x7f5630007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f56367fc700 1 -- 172.21.15.181:0/590369926 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630013410 con 0x7f56400d1b90 2020-11-16T22:33:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f56367fc700 1 -- 172.21.15.181:0/590369926 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630013570 con 0x7f56400d1b90 2020-11-16T22:33:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 -- 172.21.15.181:0/590369926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56400c9df0 con 0x7f56400d1b90 2020-11-16T22:33:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f56367fc700 1 -- 172.21.15.181:0/590369926 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630013930 con 0x7f56400d1b90 2020-11-16T22:33:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f563f59e700 1 -- 172.21.15.181:0/590369926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400b6e90 con 0x7f56400d1b90 2020-11-16T22:33:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.563+0000 7f56367fc700 1 -- 172.21.15.181:0/590369926 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5630010030 con 0x7f56400d1b90 2020-11-16T22:33:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f56367fc700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5610055060 0x7f5610057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f56367fc700 1 -- 172.21.15.181:0/590369926 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f563006da10 con 0x7f56400d1b90 2020-11-16T22:33:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f563e59c700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5610055060 0x7f5610057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f563e59c700 1 --2- 172.21.15.181:0/590369926 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5610055060 0x7f5610057520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f562800b1b0 tx=0x7f5628015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f5653514740 1 Processor -- start 2020-11-16T22:33:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.567+0000 7f5653514740 1 -- start start 2020-11-16T22:33:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd2dbb0 0xfba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb9fd0 0xfb3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xe454b0 con 0xfb9fd0 2020-11-16T22:33:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xef9930 con 0xd2dbb0 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xdb96f0 con 0xd9b7b0 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd2dbb0 0xfba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42364/0 (socket says 172.21.15.181:42364) 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 -- 172.21.15.181:0/4219472010 learned_addr learned my addr 172.21.15.181:0/4219472010 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563ed9d700 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb9fd0 0xfb3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 -- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd2dbb0 msgr2=0xfba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd2dbb0 0xfba670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 -- 172.21.15.181:0/4219472010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb9fd0 msgr2=0xfb3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb9fd0 0xfb3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 -- 172.21.15.181:0/4219472010 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xdb45a0 con 0xd9b7b0 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f563e59c700 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f562800b750 tx=0x7f562801f000).ready entity=mon.2 client_cookie=204147323a4dced6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f562f7fe700 1 -- 172.21.15.181:0/4219472010 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628011c80 con 0xd9b7b0 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f562f7fe700 1 -- 172.21.15.181:0/4219472010 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628011de0 con 0xd9b7b0 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f562f7fe700 1 -- 172.21.15.181:0/4219472010 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562802a680 con 0xd9b7b0 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 msgr2=0xfbc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f562800b750 tx=0x7f562801f000).stop 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- 172.21.15.181:0/4219472010 shutdown_connections 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xfb9fd0 0xfb3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0xfbc720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 --2- 172.21.15.181:0/4219472010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd2dbb0 0xfba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- 172.21.15.181:0/4219472010 >> 172.21.15.181:0/4219472010 conn(0xf69e40 msgr2=0xfb63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- 172.21.15.181:0/4219472010 shutdown_connections 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- 172.21.15.181:0/4219472010 wait complete. 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 -- start start 2020-11-16T22:33:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.571+0000 7f5653514740 1 Processor -- start 2020-11-16T22:33:32.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x100b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 0x1007a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xee66e0 con 0xd2dbb0 2020-11-16T22:33:32.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xee6820 con 0xd9b7b0 2020-11-16T22:33:32.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xee7a90 con 0xfb9fd0 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 10 client.0 ll_register_callbacks cb 0xedcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 0x1007a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34528/0 (socket says 172.21.15.181:34528) 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 0x1007a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42372/0 (socket says 172.21.15.181:42372) 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 -- 172.21.15.181:0/638481088 learned_addr learned my addr 172.21.15.181:0/638481088 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563e59c700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x100b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 -- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 msgr2=0x1007a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 0x1007a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 -- 172.21.15.181:0/638481088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 msgr2=0x100b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x100b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 -- 172.21.15.181:0/638481088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xed7280 con 0xd9b7b0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563dd9b700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f5638014df0 tx=0x7f56380091d0).ready entity=mon.1 client_cookie=2176f4270b532de6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638012a00 con 0xd9b7b0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5638012b60 con 0xd9b7b0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf9a6c0 con 0xd9b7b0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f563801e470 con 0xd9b7b0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 10 client.7246 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0xd0cfd0 con 0xd9b7b0 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf9a6c0 con 0xd9b7b0 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f563801e5d0 con 0xd9b7b0 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 10 client.7246 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 10 client.7246 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 20 client.7246 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x101cec0 0xfdbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 4 client.7246 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xdd7c50 con 0x101cec0 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563e59c700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x101cec0 0xfdbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f563e59c700 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x101cec0 0xfdbea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=fe30f9b489f6766 server_cookie=beeda43acbbe7b17 in_seq=0 out_seq=0 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.575+0000 7f562cff9700 10 client.7246 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f562cff9700 1 -- 172.21.15.181:0/638481088 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xdd7c50 con 0x101cec0 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f562cff9700 10 client.7246 handle_command_reply: tid=0 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 client.7246 shutdown 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 msgr2=0xfb6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f5638014df0 tx=0x7f56380091d0).stop 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 shutdown_connections 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0x101cec0 0xfdbea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfb9fd0 0x1007a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x100b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 --2- 172.21.15.181:0/638481088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0xfb6530 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 >> 172.21.15.181:0/638481088 conn(0xf69e40 msgr2=0xfbc370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f562cff9700 0 client.7246 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 shutdown_connections 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 1 -- 172.21.15.181:0/638481088 wait complete. 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.579+0000 7f5653514740 20 client.7246 trim_cache size 0 max 16384 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.595+0000 7f5653514740 1 Processor -- start 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.595+0000 7f5653514740 1 -- start start 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf69e40 0x10063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 0x1062230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1029db0 con 0xf69e40 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xebee40 con 0xd9b7b0 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x10068e0 con 0xd2dbb0 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 0x1062230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34536/0 (socket says 172.21.15.181:34536) 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 -- 172.21.15.181:0/3511567098 learned_addr learned my addr 172.21.15.181:0/3511567098 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.782 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563e59c700 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf69e40 0x10063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 -- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 msgr2=0x1062230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 0x1062230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 -- 172.21.15.181:0/3511567098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf69e40 msgr2=0x10063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf69e40 0x10063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 -- 172.21.15.181:0/3511567098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080081d0 con 0xd9b7b0 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563dd9b700 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 0x1062230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:32.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f563ed9d700 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f5630047a80 tx=0x7f563000a000).ready entity=mon.1 client_cookie=8eeb8a0e57bdb1d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f562ffff700 1 -- 172.21.15.181:0/3511567098 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630045da0 con 0xd9b7b0 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f562ffff700 1 -- 172.21.15.181:0/3511567098 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630013d60 con 0xd9b7b0 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f562ffff700 1 -- 172.21.15.181:0/3511567098 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630046e20 con 0xd9b7b0 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 msgr2=0x1062720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f5630047a80 tx=0x7f563000a000).stop 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- 172.21.15.181:0/3511567098 shutdown_connections 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd9b7b0 0x1062720 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf69e40 0x10063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 --2- 172.21.15.181:0/3511567098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd2dbb0 0x1062230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- 172.21.15.181:0/3511567098 >> 172.21.15.181:0/3511567098 conn(0x7f5608002320 msgr2=0xfb5950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- 172.21.15.181:0/3511567098 shutdown_connections 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- 172.21.15.181:0/3511567098 wait complete. 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 -- start start 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.599+0000 7f5653514740 1 Processor -- start 2020-11-16T22:33:32.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x104ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf69e40 0x10038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x104d370 con 0xd2dbb0 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xfeb1a0 con 0xf69e40 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1004b50 con 0xd9b7b0 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 10 client.0 ll_register_callbacks cb 0xfbaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:33:32.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42384/0 (socket says 172.21.15.181:42384) 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 -- 172.21.15.181:0/3111543278 learned_addr learned my addr 172.21.15.181:0/3111543278 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563ed9d700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf69e40 0x10038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563e59c700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x104ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 -- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf69e40 msgr2=0x10038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf69e40 0x10038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 -- 172.21.15.181:0/3111543278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 msgr2=0x104ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:32.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x104ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 -- 172.21.15.181:0/3111543278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd4b720 con 0xd9b7b0 2020-11-16T22:33:32.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563dd9b700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f563801d9b0 tx=0x7f563801e000).ready entity=mon.2 client_cookie=6fde298b057bde2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563801eb20 con 0xd9b7b0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5638012610 con 0xd9b7b0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5628011de0 con 0xd9b7b0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5638002030 con 0xd9b7b0 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 10 client.6668 fetch_fsmap learned FSMap version 88 2020-11-16T22:33:32.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0xd56e00 con 0xd9b7b0 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628011de0 con 0xd9b7b0 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 88) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f563801ed30 con 0xd9b7b0 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 10 client.6668 fetch_fsmap finished waiting for FSMap version 88 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 10 client.6668 resolve_mds: resolved ID 'd' to GID 6943 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 20 client.6668 populate_metadata read hostname 'smithi181' 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xfb9fd0 0x106b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:32.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 4 client.6668 mds_command: new command op to 6943 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,r,e,s,u,m,e,",}] 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 --> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] -- command(tid 0: { " p r e f i x " : " s c r u b r e s u m e " }) v1 -- 0xddef50 con 0xfb9fd0 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563e59c700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xfb9fd0 0x106b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 10 client.6668 ms_handle_connect on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f563e59c700 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xfb9fd0 0x106b740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=b9959e8b35640261 server_cookie=db17c2247d3a3d46 in_seq=0 out_seq=0 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 1 -- 172.21.15.181:0/3111543278 <== mds.1 v2:172.21.15.181:6836/1009245241 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0xddef50 con 0xfb9fd0 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f562cff9700 10 client.6668 handle_command_reply: tid=0 2020-11-16T22:33:32.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.603+0000 7f5653514740 1 client.6668 shutdown 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 msgr2=0x106f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f563801d9b0 tx=0x7f563801e000).stop 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 shutdown_connections 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:6836/1009245241,v1:172.21.15.181:6837/1009245241] conn(0xfb9fd0 0x106b740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf69e40 0x10038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xd2dbb0 0x104ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f562cff9700 0 client.6668 ms_handle_reset on v2:172.21.15.181:6836/1009245241 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 --2- 172.21.15.181:0/3111543278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xd9b7b0 0x106f730 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 >> 172.21.15.181:0/3111543278 conn(0x7f5608002320 msgr2=0xfb5950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 shutdown_connections 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 1 -- 172.21.15.181:0/3111543278 wait complete. 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:32.607+0000 7f5653514740 20 client.6668 trim_cache size 0 max 16384 2020-11-16T22:33:32.995 INFO:teuthology.orchestra.run.smithi181.stderr:Error EXDEV: Not rank 0 2020-11-16T22:33:32.995 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:33:32.996 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:33:32.996 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:33.033 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:33.050 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:33.068 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:33:33.068 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:33:33.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:33:33.069 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:33:33.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:33:33.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:33:33.164 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:33:33.183 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:33:33.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 Processor -- start 2020-11-16T22:33:33.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 -- start start 2020-11-16T22:33:33.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c04066230 0x7f2c04066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c040632c0 0x7f2c04063750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c04066b00 con 0x7f2c040c8470 2020-11-16T22:33:33.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04063d00 con 0x7f2c040632c0 2020-11-16T22:33:33.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04063fa0 con 0x7f2c04066230 2020-11-16T22:33:33.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c04066230 0x7f2c04066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c04066230 0x7f2c04066610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34548/0 (socket says 172.21.15.181:34548) 2020-11-16T22:33:33.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 -- 172.21.15.181:0/1362850086 learned_addr learned my addr 172.21.15.181:0/1362850086 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:33.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c02ffd700 1 --2- 172.21.15.181:0/1362850086 >> v2:172.21.15.175:3300/0 conn(0x7f2c040632c0 0x7f2c04063750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 -- 172.21.15.181:0/1362850086 >> v2:172.21.15.175:3300/0 conn(0x7f2c040632c0 msgr2=0x7f2c04063750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 --2- 172.21.15.181:0/1362850086 >> v2:172.21.15.175:3300/0 conn(0x7f2c040632c0 0x7f2c04063750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 -- 172.21.15.181:0/1362850086 >> v1:172.21.15.175:6789/0 conn(0x7f2c040c8470 legacy=0x7f2c040c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 -- 172.21.15.181:0/1362850086 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04064240 con 0x7f2c04066230 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c037fe700 1 --2- 172.21.15.181:0/1362850086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c04066230 0x7f2c04066610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f2c040622a0 tx=0x7f2bfc009290).ready entity=mon.1 client_cookie=4df028eaec5d6b79 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c027fc700 1 -- 172.21.15.181:0/1362850086 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00bc50 con 0x7f2c04066230 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c027fc700 1 -- 172.21.15.181:0/1362850086 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc00bdb0 con 0x7f2c04066230 2020-11-16T22:33:33.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c027fc700 1 -- 172.21.15.181:0/1362850086 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc011610 con 0x7f2c04066230 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 -- 172.21.15.181:0/1362850086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c04066230 msgr2=0x7f2c04066610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.331+0000 7f2c08c90700 1 --2- 172.21.15.181:0/1362850086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c04066230 0x7f2c04066610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f2c040622a0 tx=0x7f2bfc009290).stop 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- 172.21.15.181:0/1362850086 shutdown_connections 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 --2- 172.21.15.181:0/1362850086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c04066230 0x7f2c04066610 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 --2- 172.21.15.181:0/1362850086 >> v2:172.21.15.175:3300/0 conn(0x7f2c040632c0 0x7f2c04063750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- 172.21.15.181:0/1362850086 >> 172.21.15.181:0/1362850086 conn(0x7f2c040cfe90 msgr2=0x7f2c040d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:33.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- 172.21.15.181:0/1362850086 shutdown_connections 2020-11-16T22:33:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- 172.21.15.181:0/1362850086 wait complete. 2020-11-16T22:33:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 Processor -- start 2020-11-16T22:33:33.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- start start 2020-11-16T22:33:33.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c04066230 0x7f2c0415f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 0x7f2c040ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040cb1d0 con 0x7f2c04066230 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040cb470 con 0x7f2c040632c0 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.335+0000 7f2c08c90700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040cb710 con 0x7f2c040c8470 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34554/0 (socket says 172.21.15.181:34554) 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c037fe700 1 -- 172.21.15.181:0/3821459009 learned_addr learned my addr 172.21.15.181:0/3821459009 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 0x7f2c040ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c02ffd700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c04066230 0x7f2c0415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 -- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 msgr2=0x7f2c0415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 -- 172.21.15.181:0/3821459009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c04066230 msgr2=0x7f2c0415f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c04066230 0x7f2c0415f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 -- 172.21.15.181:0/3821459009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c040cb9b0 con 0x7f2c040c8470 2020-11-16T22:33:33.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c037fe700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:33.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c03fff700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 0x7f2c040ca4d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f2bf400c6e0 tx=0x7f2bf4007a60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4013410 con 0x7f2c040c8470 2020-11-16T22:33:33.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf4013570 con 0x7f2c040c8470 2020-11-16T22:33:33.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4013930 con 0x7f2c040c8470 2020-11-16T22:33:33.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c040cbc50 con 0x7f2c040c8470 2020-11-16T22:33:33.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c040ca9c0 con 0x7f2c040c8470 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2bf4019020 con 0x7f2c040c8470 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c04044f60 con 0x7f2c040c8470 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.339+0000 7f2bfb7fe700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 0x7f2be0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.343+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2bf406d910 con 0x7f2c040c8470 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.343+0000 7f2c037fe700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 0x7f2be0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.343+0000 7f2c037fe700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 0x7f2be0057520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f2bfc009e50 tx=0x7f2bfc016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.347+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2bf4043b70 con 0x7f2c040c8470 2020-11-16T22:33:33.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.499+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2c040d0d10 con 0x7f2c040c8470 2020-11-16T22:33:33.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2bfb7fe700 1 -- 172.21.15.181:0/3821459009 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2bf4047190 con 0x7f2c040c8470 2020-11-16T22:33:33.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 msgr2=0x7f2be0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 0x7f2be0057520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f2bfc009e50 tx=0x7f2bfc016000).stop 2020-11-16T22:33:33.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 msgr2=0x7f2c040ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 0x7f2c040ca4d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f2bf400c6e0 tx=0x7f2bf4007a60).stop 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 shutdown_connections 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2be0055060 0x7f2be0057520 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c040c8470 0x7f2c040ca4d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c040632c0 0x7f2c0415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 --2- 172.21.15.181:0/3821459009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c04066230 0x7f2c0415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 >> 172.21.15.181:0/3821459009 conn(0x7f2c040cfe90 msgr2=0x7f2c040cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:33.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 shutdown_connections 2020-11-16T22:33:33.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.515+0000 7f2c08c90700 1 -- 172.21.15.181:0/3821459009 wait complete. 2020-11-16T22:33:33.519 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2020-11-16T22:33:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 Processor -- start 2020-11-16T22:33:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 -- start start 2020-11-16T22:33:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51c00b7cd0 0x7f51c00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c015e760 con 0x7f51c00b5ca0 2020-11-16T22:33:33.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c015ea00 con 0x7f51c00b58c0 2020-11-16T22:33:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51c5da3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00b8110 con 0x7f51c00b7cd0 2020-11-16T22:33:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51bffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51c00b7cd0 0x7f51c00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51bf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f51c00b58c0 0x7f51c015dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48870/0 (socket says 172.21.15.181:48870) 2020-11-16T22:33:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51bf7fe700 1 -- 172.21.15.181:0/101221591 learned_addr learned my addr 172.21.15.181:0/101221591 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.651+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463683380 0 0) 0x7f51c015ea00 con 0x7f51c00b58c0 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac003240 con 0x7f51c00b58c0 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51bffff700 1 -- 172.21.15.181:0/101221591 >> v1:172.21.15.181:6789/0 conn(0x7f51c00b58c0 legacy=0x7f51c015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597488246 0 0) 0x7f51c015e760 con 0x7f51c00b5ca0 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51bffff700 1 -- 172.21.15.181:0/101221591 >> v1:172.21.15.175:6789/0 conn(0x7f51c00b5ca0 legacy=0x7f51c00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51bffff700 1 -- 172.21.15.181:0/101221591 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c015eca0 con 0x7f51c00b7cd0 2020-11-16T22:33:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51bffff700 1 --2- 172.21.15.181:0/101221591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00cd9a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f51b8009c90 tx=0x7f51b80091d0).ready entity=mon.1 client_cookie=49b942ba1cb3c9fc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800ba70 con 0x7f51c00b7cd0 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800bbd0 con 0x7f51c00b7cd0 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51be7fc700 1 -- 172.21.15.181:0/101221591 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8011400 con 0x7f51c00b7cd0 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- 172.21.15.181:0/101221591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 msgr2=0x7f51c00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 --2- 172.21.15.181:0/101221591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00cd9a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f51b8009c90 tx=0x7f51b80091d0).stop 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- 172.21.15.181:0/101221591 shutdown_connections 2020-11-16T22:33:33.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 --2- 172.21.15.181:0/101221591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00cd9a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- 172.21.15.181:0/101221591 >> 172.21.15.181:0/101221591 conn(0x7f51c0060bc0 msgr2=0x7f51c00d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:33.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- 172.21.15.181:0/101221591 shutdown_connections 2020-11-16T22:33:33.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- 172.21.15.181:0/101221591 wait complete. 2020-11-16T22:33:33.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 Processor -- start 2020-11-16T22:33:33.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.655+0000 7f51c5da3700 1 -- start start 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 0x7f51c00d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c016f420 con 0x7f51c00b5ca0 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c016f6c0 con 0x7f51c00b7cd0 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c016f960 con 0x7f51c00b58c0 2020-11-16T22:33:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42412/0 (socket says 172.21.15.181:42412) 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 -- 172.21.15.181:0/3454505248 learned_addr learned my addr 172.21.15.181:0/3454505248 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51beffd700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 0x7f51c00d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 -- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 msgr2=0x7f51c00d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 -- 172.21.15.181:0/3454505248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 msgr2=0x7f51c00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 0x7f51c00d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 -- 172.21.15.181:0/3454505248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c016fc00 con 0x7f51c00b58c0 2020-11-16T22:33:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bffff700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51beffd700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 0x7f51c00d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51bf7fe700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f51b4001e90 tx=0x7f51b4007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40072e0 con 0x7f51c00b58c0 2020-11-16T22:33:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b4007440 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c016fea0 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005820 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0157ae0 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.659+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0170030 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.663+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51b4010070 con 0x7f51c00b58c0 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.663+0000 7f519f7fe700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 0x7f5198057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.663+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51b406d440 con 0x7f51c00b58c0 2020-11-16T22:33:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.663+0000 7f51beffd700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 0x7f5198057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.663+0000 7f51beffd700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 0x7f5198057520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f51c0146270 tx=0x7f51b0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:33.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.667+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f51b4043590 con 0x7f51c00b58c0 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.871+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7f51c0069eb0 con 0x7f51c00b58c0 2020-11-16T22:33:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.959+0000 7f519f7fe700 1 -- 172.21.15.181:0/3454505248 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v326) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51b4018070 con 0x7f51c00b58c0 2020-11-16T22:33:33.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 msgr2=0x7f5198057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 0x7f5198057520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f51c0146270 tx=0x7f51b0009040).stop 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 msgr2=0x7f51c0157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f51b4001e90 tx=0x7f51b4007ac0).stop 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 shutdown_connections 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5198055060 0x7f5198057520 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c00b7cd0 0x7f51c00d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c00b58c0 0x7f51c0157450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 --2- 172.21.15.181:0/3454505248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51c00b5ca0 0x7f51c00d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 >> 172.21.15.181:0/3454505248 conn(0x7f51c0060bc0 msgr2=0x7f51c0155240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 shutdown_connections 2020-11-16T22:33:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:33.963+0000 7f51c5da3700 1 -- 172.21.15.181:0/3454505248 wait complete. 2020-11-16T22:33:33.967 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:33:33.968 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:33:33.968 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.107+0000 7f74fc926700 1 Processor -- start 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.107+0000 7f74fc926700 1 -- start start 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.107+0000 7f74fc926700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74f815f800 0x7f74f815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.107+0000 7f74fc926700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 0x7f74f8065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.107+0000 7f74fc926700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f8064040 con 0x7f74f80656f0 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f8067c30 con 0x7f74f815f800 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f80642e0 con 0x7f74f815fbe0 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 0x7f74f8065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74f815f800 0x7f74f815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 0x7f74f8065200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42420/0 (socket says 172.21.15.181:42420) 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f659c700 1 -- 172.21.15.181:0/1399317242 learned_addr learned my addr 172.21.15.181:0/1399317242 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 -- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 msgr2=0x7f74f8065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 --2- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 0x7f74f8065200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 -- 172.21.15.181:0/1399317242 >> v1:172.21.15.175:6789/0 conn(0x7f74f80656f0 legacy=0x7f74f8067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 -- 172.21.15.181:0/1399317242 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8064580 con 0x7f74f815f800 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f6d9d700 1 --2- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815f800 0x7f74f815d560 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f74e800dd30 tx=0x7f74e800d270).ready entity=mon.1 client_cookie=b7ee3d8d145fbb9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f5d9b700 1 -- 172.21.15.181:0/1399317242 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800fc50 con 0x7f74f815f800 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f5d9b700 1 -- 172.21.15.181:0/1399317242 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800fdb0 con 0x7f74f815f800 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74f5d9b700 1 -- 172.21.15.181:0/1399317242 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015610 con 0x7f74f815f800 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815f800 msgr2=0x7f74f815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 --2- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815f800 0x7f74f815d560 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f74e800dd30 tx=0x7f74e800d270).stop 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- 172.21.15.181:0/1399317242 shutdown_connections 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 --2- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815f800 0x7f74f815d560 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 --2- 172.21.15.181:0/1399317242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f815fbe0 0x7f74f8065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- 172.21.15.181:0/1399317242 >> 172.21.15.181:0/1399317242 conn(0x7f74f80d2310 msgr2=0x7f74f80d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- 172.21.15.181:0/1399317242 shutdown_connections 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.111+0000 7f74fc926700 1 -- 172.21.15.181:0/1399317242 wait complete. 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 Processor -- start 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 -- start start 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f815f800 0x7f74f8142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815fbe0 0x7f74f816c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f815db40 con 0x7f74f815f800 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f815dde0 con 0x7f74f815fbe0 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74fc926700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f815e080 con 0x7f74f80656f0 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42424/0 (socket says 172.21.15.181:42424) 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 -- 172.21.15.181:0/4259028380 learned_addr learned my addr 172.21.15.181:0/4259028380 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f759e700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815fbe0 0x7f74f816c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f659c700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f815f800 0x7f74f8142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 -- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815fbe0 msgr2=0x7f74f816c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815fbe0 0x7f74f816c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 -- 172.21.15.181:0/4259028380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f815f800 msgr2=0x7f74f8142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f815f800 0x7f74f8142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 -- 172.21.15.181:0/4259028380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f815e320 con 0x7f74f80656f0 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74f6d9d700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f74e800d2b0 tx=0x7f74e801a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.115+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e801f640 con 0x7f74f80656f0 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e801f7a0 con 0x7f74f80656f0 2020-11-16T22:33:34.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f815e5c0 con 0x7f74f80656f0 2020-11-16T22:33:34.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e801fb20 con 0x7f74f80656f0 2020-11-16T22:33:34.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f80cd0d0 con 0x7f74f80656f0 2020-11-16T22:33:34.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f815ea60 con 0x7f74f80656f0 2020-11-16T22:33:34.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f74e801b040 con 0x7f74f80656f0 2020-11-16T22:33:34.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74deffd700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 0x7f74c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:34.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f74e80739a0 con 0x7f74f80656f0 2020-11-16T22:33:34.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74f659c700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 0x7f74c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:34.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.119+0000 7f74f659c700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 0x7f74c8057520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f74ec006a10 tx=0x7f74ec008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:34.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.123+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f74f815ea60 con 0x7f74f80656f0 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.323+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f74f8044f60 con 0x7f74f80656f0 2020-11-16T22:33:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.959+0000 7f74deffd700 1 -- 172.21.15.181:0/4259028380 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v327) v1 ==== 157+0+0 (secure 0 0 0) 0x7f74e80190e0 con 0x7f74f80656f0 2020-11-16T22:33:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 msgr2=0x7f74c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 0x7f74c8057520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f74ec006a10 tx=0x7f74ec008040).stop 2020-11-16T22:33:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 msgr2=0x7f74f80ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f74e800d2b0 tx=0x7f74e801a000).stop 2020-11-16T22:33:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 shutdown_connections 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f74c8055060 0x7f74c8057520 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f815fbe0 0x7f74f816c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74f80656f0 0x7f74f80ccbe0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 --2- 172.21.15.181:0/4259028380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f815f800 0x7f74f8142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 >> 172.21.15.181:0/4259028380 conn(0x7f74f80d2310 msgr2=0x7f74f80d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 shutdown_connections 2020-11-16T22:33:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:34.963+0000 7f74fc926700 1 -- 172.21.15.181:0/4259028380 wait complete. 2020-11-16T22:33:34.971 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:33:35.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.111+0000 7fa1dd22f700 1 Processor -- start 2020-11-16T22:33:35.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 -- start start 2020-11-16T22:33:35.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1d80615f0 0x7fa1d80622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1d8062790 0x7fa1d8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d8160e40 con 0x7fa1d815f5b0 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d81609a0 con 0x7fa1d80615f0 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d8160ba0 con 0x7fa1d8062790 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1d8062790 0x7fa1d8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa1d815f5b0 0x7fa1d80d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48894/0 (socket says 172.21.15.181:48894) 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d6d9d700 1 -- 172.21.15.181:0/3290573373 learned_addr learned my addr 172.21.15.181:0/3290573373 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:35.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d659c700 1 --2- 172.21.15.181:0/3290573373 >> v2:172.21.15.175:3300/0 conn(0x7fa1d80615f0 0x7fa1d80622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 -- 172.21.15.181:0/3290573373 >> v2:172.21.15.175:3300/0 conn(0x7fa1d80615f0 msgr2=0x7fa1d80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:35.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 --2- 172.21.15.181:0/3290573373 >> v2:172.21.15.175:3300/0 conn(0x7fa1d80615f0 0x7fa1d80622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 -- 172.21.15.181:0/3290573373 >> v1:172.21.15.181:6789/0 conn(0x7fa1d815f5b0 legacy=0x7fa1d80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 -- 172.21.15.181:0/3290573373 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8060970 con 0x7fa1d8062790 2020-11-16T22:33:35.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d5d9b700 1 -- 172.21.15.181:0/3290573373 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496178455 0 0) 0x7fa1d8160e40 con 0x7fa1d815f5b0 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d759e700 1 --2- 172.21.15.181:0/3290573373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d8062790 0x7fa1d8080fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fa1d80d2ea0 tx=0x7fa1d00091d0).ready entity=mon.1 client_cookie=4f317694b3fb9c41 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d5d9b700 1 -- 172.21.15.181:0/3290573373 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000ba70 con 0x7fa1d8062790 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d5d9b700 1 -- 172.21.15.181:0/3290573373 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000bbd0 con 0x7fa1d8062790 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1d5d9b700 1 -- 172.21.15.181:0/3290573373 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0011400 con 0x7fa1d8062790 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 -- 172.21.15.181:0/3290573373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d8062790 msgr2=0x7fa1d8080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.115+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/3290573373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d8062790 0x7fa1d8080fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fa1d80d2ea0 tx=0x7fa1d00091d0).stop 2020-11-16T22:33:35.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 -- 172.21.15.181:0/3290573373 shutdown_connections 2020-11-16T22:33:35.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/3290573373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d8062790 0x7fa1d8080fd0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/3290573373 >> v2:172.21.15.175:3300/0 conn(0x7fa1d80615f0 0x7fa1d80622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 -- 172.21.15.181:0/3290573373 >> 172.21.15.181:0/3290573373 conn(0x7fa1d80d69a0 msgr2=0x7fa1d80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:35.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 -- 172.21.15.181:0/3290573373 shutdown_connections 2020-11-16T22:33:35.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 -- 172.21.15.181:0/3290573373 wait complete. 2020-11-16T22:33:35.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 Processor -- start 2020-11-16T22:33:35.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.119+0000 7fa1dd22f700 1 -- start start 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d80615f0 0x7fa1d816f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d8062790 0x7fa1d8166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80babd0 con 0x7fa1d80615f0 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80ba930 con 0x7fa1d815f5b0 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81706d0 con 0x7fa1d8062790 2020-11-16T22:33:35.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34600/0 (socket says 172.21.15.181:34600) 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 -- 172.21.15.181:0/2482510498 learned_addr learned my addr 172.21.15.181:0/2482510498 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d659c700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d8062790 0x7fa1d8166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d6d9d700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d80615f0 0x7fa1d816f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 -- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d8062790 msgr2=0x7fa1d8166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d8062790 0x7fa1d8166850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 -- 172.21.15.181:0/2482510498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d80615f0 msgr2=0x7fa1d816f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d80615f0 0x7fa1d816f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 -- 172.21.15.181:0/2482510498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8170970 con 0x7fa1d815f5b0 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1d759e700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fa1d8160ce0 tx=0x7fa1d0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d001c5f0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d001c750 con 0x7fa1d815f5b0 2020-11-16T22:33:35.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d8170c10 con 0x7fa1d815f5b0 2020-11-16T22:33:35.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d001cad0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d816fda0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.123+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d8069eb0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.127+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa1d0024030 con 0x7fa1d815f5b0 2020-11-16T22:33:35.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.127+0000 7fa1beffd700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 0x7fa1b4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.127+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1d006f8a0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.127+0000 7fa1d6d9d700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 0x7fa1b4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.127+0000 7fa1d6d9d700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 0x7fa1b4057550 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fa1cc002d70 tx=0x7fa1cc0016d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.135+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa1d0045c30 con 0x7fa1d815f5b0 2020-11-16T22:33:35.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.287+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fa1d81642f0 con 0x7fa1d815f5b0 2020-11-16T22:33:35.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1beffd700 1 -- 172.21.15.181:0/2482510498 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1d0022020 con 0x7fa1d815f5b0 2020-11-16T22:33:35.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 msgr2=0x7fa1b4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 0x7fa1b4057550 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fa1cc002d70 tx=0x7fa1cc0016d0).stop 2020-11-16T22:33:35.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 msgr2=0x7fa1d8166d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fa1d8160ce0 tx=0x7fa1d0016000).stop 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 shutdown_connections 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa1b4055090 0x7fa1b4057550 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d815f5b0 0x7fa1d8166d40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d80615f0 0x7fa1d816f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.299+0000 7fa1dd22f700 1 --2- 172.21.15.181:0/2482510498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d8062790 0x7fa1d8166850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.303+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 >> 172.21.15.181:0/2482510498 conn(0x7fa1d80d69a0 msgr2=0x7fa1d80d47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.303+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 shutdown_connections 2020-11-16T22:33:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.303+0000 7fa1dd22f700 1 -- 172.21.15.181:0/2482510498 wait complete. 2020-11-16T22:33:35.306 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:33:35.307 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:35.348 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:35.366 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:33:35.383 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:33:35.383 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:33:35.384 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:35.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 Processor -- start 2020-11-16T22:33:35.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 -- start start 2020-11-16T22:33:35.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fead8145e10 0x7fead81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead8061b50 con 0x7fead8145a30 2020-11-16T22:33:35.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead80618b0 con 0x7fead80816e0 2020-11-16T22:33:35.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadd655700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead80cb930 con 0x7fead8145e10 2020-11-16T22:33:35.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feacffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fead80816e0 0x7fead80cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48908/0 (socket says 172.21.15.181:48908) 2020-11-16T22:33:35.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feacffff700 1 -- 172.21.15.181:0/1135349896 learned_addr learned my addr 172.21.15.181:0/1135349896 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:35.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadce54700 1 --2- 172.21.15.181:0/1135349896 >> v2:172.21.15.175:3300/0 conn(0x7fead8145e10 0x7fead81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7fead77fe700 1 -- 172.21.15.181:0/1135349896 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773565194 0 0) 0x7fead80618b0 con 0x7fead80816e0 2020-11-16T22:33:35.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7fead77fe700 1 -- 172.21.15.181:0/1135349896 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab8003240 con 0x7fead80816e0 2020-11-16T22:33:35.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadce54700 1 -- 172.21.15.181:0/1135349896 >> v1:172.21.15.181:6789/0 conn(0x7fead80816e0 legacy=0x7fead80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadce54700 1 -- 172.21.15.181:0/1135349896 >> v1:172.21.15.175:6789/0 conn(0x7fead8145a30 legacy=0x7fead8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadce54700 1 -- 172.21.15.181:0/1135349896 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80cbbd0 con 0x7fead8145e10 2020-11-16T22:33:35.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7feadce54700 1 --2- 172.21.15.181:0/1135349896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145e10 0x7fead81492c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fead0009c90 tx=0x7fead00091d0).ready entity=mon.0 client_cookie=746b7947a62264d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7fead77fe700 1 -- 172.21.15.181:0/1135349896 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000ba70 con 0x7fead8145e10 2020-11-16T22:33:35.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7fead77fe700 1 -- 172.21.15.181:0/1135349896 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead000bbd0 con 0x7fead8145e10 2020-11-16T22:33:35.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.567+0000 7fead77fe700 1 -- 172.21.15.181:0/1135349896 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0011400 con 0x7fead8145e10 2020-11-16T22:33:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 -- 172.21.15.181:0/1135349896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145e10 msgr2=0x7fead81492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 --2- 172.21.15.181:0/1135349896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145e10 0x7fead81492c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fead0009c90 tx=0x7fead00091d0).stop 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 -- 172.21.15.181:0/1135349896 shutdown_connections 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 --2- 172.21.15.181:0/1135349896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145e10 0x7fead81492c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 -- 172.21.15.181:0/1135349896 >> 172.21.15.181:0/1135349896 conn(0x7fead80d68c0 msgr2=0x7fead80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 -- 172.21.15.181:0/1135349896 shutdown_connections 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.571+0000 7feadd655700 1 -- 172.21.15.181:0/1135349896 wait complete. 2020-11-16T22:33:35.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 Processor -- start 2020-11-16T22:33:35.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 -- start start 2020-11-16T22:33:35.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145a30 0x7fead80d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead8145e10 0x7fead80d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead81702c0 con 0x7fead8145a30 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead81707a0 con 0x7fead8145e10 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadd655700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead80d5e30 con 0x7fead80816e0 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feadce54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead8145e10 0x7fead80d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42452/0 (socket says 172.21.15.181:42452) 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 -- 172.21.15.181:0/2610421308 learned_addr learned my addr 172.21.15.181:0/2610421308 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:35.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7feacffff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145a30 0x7fead80d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 -- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead8145e10 msgr2=0x7fead80d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead8145e10 0x7fead80d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 -- 172.21.15.181:0/2610421308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145a30 msgr2=0x7fead80d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145a30 0x7fead80d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 -- 172.21.15.181:0/2610421308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80d6070 con 0x7fead80816e0 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.575+0000 7fead7fff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7feac800c770 tx=0x7feac8007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800e490 con 0x7fead80816e0 2020-11-16T22:33:35.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac800e610 con 0x7fead80816e0 2020-11-16T22:33:35.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead8165850 con 0x7fead80816e0 2020-11-16T22:33:35.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800e9d0 con 0x7fead80816e0 2020-11-16T22:33:35.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead8166030 con 0x7fead80816e0 2020-11-16T22:33:35.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feac8019030 con 0x7fead80816e0 2020-11-16T22:33:35.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead81659e0 con 0x7fead80816e0 2020-11-16T22:33:35.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 0x7feab4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:35.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feac806d120 con 0x7fead80816e0 2020-11-16T22:33:35.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7feacffff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 0x7feab4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:35.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.579+0000 7feacffff700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 0x7feab4057520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fead81454e0 tx=0x7feac0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:35.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.583+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7feac8043320 con 0x7fead80816e0 2020-11-16T22:33:35.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.783+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fead8044f60 con 0x7fead80816e0 2020-11-16T22:33:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.783+0000 7fead4ff9700 1 -- 172.21.15.181:0/2610421308 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4665 (secure 0 0 0) 0x7feac8017020 con 0x7fead80816e0 2020-11-16T22:33:35.774 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:35.774 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":88,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:24.911953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 msgr2=0x7feab4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 0x7feab4057520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fead81454e0 tx=0x7feac0009040).stop 2020-11-16T22:33:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 msgr2=0x7fead80d5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7feac800c770 tx=0x7feac8007c10).stop 2020-11-16T22:33:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 shutdown_connections 2020-11-16T22:33:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feab4055060 0x7feab4057520 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead8145e10 0x7fead80d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead80816e0 0x7fead80d5940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 --2- 172.21.15.181:0/2610421308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fead8145a30 0x7fead80d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 >> 172.21.15.181:0/2610421308 conn(0x7fead80d68c0 msgr2=0x7fead80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 shutdown_connections 2020-11-16T22:33:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:35.787+0000 7feadd655700 1 -- 172.21.15.181:0/2610421308 wait complete. 2020-11-16T22:33:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 88 2020-11-16T22:33:36.184 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:33:36.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.315+0000 7fcaea8bb700 1 Processor -- start 2020-11-16T22:33:36.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.315+0000 7fcaea8bb700 1 -- start start 2020-11-16T22:33:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcaea8bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae4061580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:36.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcaea8bb700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae40cc7d0 con 0x7fcae415dfd0 2020-11-16T22:33:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcaea8bb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae415eb10 con 0x7fcae4157a80 2020-11-16T22:33:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcaea8bb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae415f0b0 con 0x7fcae4061100 2020-11-16T22:33:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae98b9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcae4157a80 0x7fcae40cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48922/0 (socket says 172.21.15.181:48922) 2020-11-16T22:33:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae98b9700 1 -- 172.21.15.181:0/283674191 learned_addr learned my addr 172.21.15.181:0/283674191 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:36.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae90b8700 1 --2- 172.21.15.181:0/283674191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae4061580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:36.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727173400 0 0) 0x7fcae415eb10 con 0x7fcae4157a80 2020-11-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad0003240 con 0x7fcae4157a80 2020-11-16T22:33:36.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187549926 0 0) 0x7fcae40cc7d0 con 0x7fcae415dfd0 2020-11-16T22:33:36.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae415eb10 con 0x7fcae415dfd0 2020-11-16T22:33:36.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae90b8700 1 -- 172.21.15.181:0/283674191 >> v1:172.21.15.181:6789/0 conn(0x7fcae4157a80 legacy=0x7fcae40cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:36.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae90b8700 1 -- 172.21.15.181:0/283674191 >> v1:172.21.15.175:6789/0 conn(0x7fcae415dfd0 legacy=0x7fcae415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:36.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae90b8700 1 -- 172.21.15.181:0/283674191 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae41611b0 con 0x7fcae4061100 2020-11-16T22:33:36.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae90b8700 1 --2- 172.21.15.181:0/283674191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae4061580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fcad800b9d0 tx=0x7fcad8006df0).ready entity=mon.2 client_cookie=b2f9ea499f241a85 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:36.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8009680 con 0x7fcae4061100 2020-11-16T22:33:36.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad80097e0 con 0x7fcae4061100 2020-11-16T22:33:36.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.319+0000 7fcae88b7700 1 -- 172.21.15.181:0/283674191 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8009b60 con 0x7fcae4061100 2020-11-16T22:33:36.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- 172.21.15.181:0/283674191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 msgr2=0x7fcae4061580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:36.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/283674191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae4061580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fcad800b9d0 tx=0x7fcad8006df0).stop 2020-11-16T22:33:36.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- 172.21.15.181:0/283674191 shutdown_connections 2020-11-16T22:33:36.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/283674191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae4061580 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- 172.21.15.181:0/283674191 >> 172.21.15.181:0/283674191 conn(0x7fcae40cf630 msgr2=0x7fcae40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:36.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- 172.21.15.181:0/283674191 shutdown_connections 2020-11-16T22:33:36.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- 172.21.15.181:0/283674191 wait complete. 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 Processor -- start 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.323+0000 7fcaea8bb700 1 -- start start 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae40b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4157a80 0x7fcae40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae40c7c70 con 0x7fcae4157a80 2020-11-16T22:33:36.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae40c7f10 con 0x7fcae415dfd0 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae40c81b0 con 0x7fcae4061100 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34628/0 (socket says 172.21.15.181:34628) 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcae90b8700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4157a80 0x7fcae40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 -- 172.21.15.181:0/2003424376 learned_addr learned my addr 172.21.15.181:0/2003424376 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:36.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcae98b9700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae40b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:36.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 -- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 msgr2=0x7fcae40b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:36.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae40b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 -- 172.21.15.181:0/2003424376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4157a80 msgr2=0x7fcae40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:36.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4157a80 0x7fcae40cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:36.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 -- 172.21.15.181:0/2003424376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae40c8450 con 0x7fcae415dfd0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea0ba700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fcadc00c710 tx=0x7fcadc0079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00e490 con 0x7fcae415dfd0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc00e5f0 con 0x7fcae415dfd0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae40c86f0 con 0x7fcae415dfd0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00e970 con 0x7fcae415dfd0 2020-11-16T22:33:36.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.327+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae40b8c10 con 0x7fcae415dfd0 2020-11-16T22:33:36.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae40ca3d0 con 0x7fcae415dfd0 2020-11-16T22:33:36.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcadc019030 con 0x7fcae415dfd0 2020-11-16T22:33:36.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcae1ffb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 0x7fcabc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:36.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcadc06d1a0 con 0x7fcae415dfd0 2020-11-16T22:33:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcae98b9700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 0x7fcabc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.331+0000 7fcae98b9700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 0x7fcabc057520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fcae40b4a40 tx=0x7fcad4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.339+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcae40ca3d0 con 0x7fcae415dfd0 2020-11-16T22:33:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.495+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcae40ca3d0 con 0x7fcae415dfd0 2020-11-16T22:33:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.495+0000 7fcae1ffb700 1 -- 172.21.15.181:0/2003424376 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fcadc025200 con 0x7fcae415dfd0 2020-11-16T22:33:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 msgr2=0x7fcabc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 0x7fcabc057520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fcae40b4a40 tx=0x7fcad4009040).stop 2020-11-16T22:33:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 msgr2=0x7fcae40cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fcadc00c710 tx=0x7fcadc0079e0).stop 2020-11-16T22:33:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 shutdown_connections 2020-11-16T22:33:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcabc055060 0x7fcabc057520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415dfd0 0x7fcae40cc110 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae4061100 0x7fcae40b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:37.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 --2- 172.21.15.181:0/2003424376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcae4157a80 0x7fcae40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:37.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 >> 172.21.15.181:0/2003424376 conn(0x7fcae40cf630 msgr2=0x7fcae4160740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:37.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 shutdown_connections 2020-11-16T22:33:37.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:36.499+0000 7fcaea8bb700 1 -- 172.21.15.181:0/2003424376 wait complete. 2020-11-16T22:33:37.335 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:37.335 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:40.492654+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:33:38.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:32:37.187776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:32:38.352981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:33:38.558 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:33:38.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 Processor -- start 2020-11-16T22:33:38.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 -- start start 2020-11-16T22:33:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6a6413ec50 0x7f6a64061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a640615f0 0x7f6a6415e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6415d180 con 0x7f6a6413e870 2020-11-16T22:33:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6415d810 con 0x7f6a6413ec50 2020-11-16T22:33:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6ae62700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6415dab0 con 0x7f6a640615f0 2020-11-16T22:33:38.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a640615f0 0x7f6a6415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:38.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a640615f0 0x7f6a6415e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34634/0 (socket says 172.21.15.181:34634) 2020-11-16T22:33:38.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 -- 172.21.15.181:0/1017165129 learned_addr learned my addr 172.21.15.181:0/1017165129 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6965f700 1 --2- 172.21.15.181:0/1017165129 >> v2:172.21.15.175:3300/0 conn(0x7f6a6413ec50 0x7f6a64061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 -- 172.21.15.181:0/1017165129 >> v2:172.21.15.175:3300/0 conn(0x7f6a6413ec50 msgr2=0x7f6a64061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:38.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 --2- 172.21.15.181:0/1017165129 >> v2:172.21.15.175:3300/0 conn(0x7f6a6413ec50 0x7f6a64061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 -- 172.21.15.181:0/1017165129 >> v1:172.21.15.175:6789/0 conn(0x7f6a6413e870 legacy=0x7f6a6415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 -- 172.21.15.181:0/1017165129 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6415dd50 con 0x7f6a640615f0 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a6a661700 1 --2- 172.21.15.181:0/1017165129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a6415e730 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6a640d2b60 tx=0x7f6a5c009290).ready entity=mon.1 client_cookie=4441140d0813bf5e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a68e5e700 1 -- 172.21.15.181:0/1017165129 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00bbc0 con 0x7f6a640615f0 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.711+0000 7f6a68e5e700 1 -- 172.21.15.181:0/1017165129 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00bd20 con 0x7f6a640615f0 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a68e5e700 1 -- 172.21.15.181:0/1017165129 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011610 con 0x7f6a640615f0 2020-11-16T22:33:38.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/1017165129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 msgr2=0x7f6a6415e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/1017165129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a6415e730 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6a640d2b60 tx=0x7f6a5c009290).stop 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/1017165129 shutdown_connections 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/1017165129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a6415e730 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/1017165129 >> v2:172.21.15.175:3300/0 conn(0x7f6a6413ec50 0x7f6a64061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/1017165129 >> 172.21.15.181:0/1017165129 conn(0x7f6a640cf630 msgr2=0x7f6a640cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:38.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/1017165129 shutdown_connections 2020-11-16T22:33:38.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/1017165129 wait complete. 2020-11-16T22:33:38.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 Processor -- start 2020-11-16T22:33:38.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- start start 2020-11-16T22:33:38.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a640cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:38.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a6413ec50 0x7f6a6416f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a641700c0 con 0x7f6a6413ec50 2020-11-16T22:33:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6416f8e0 con 0x7f6a640615f0 2020-11-16T22:33:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6416fb80 con 0x7f6a6413e870 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42480/0 (socket says 172.21.15.181:42480) 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 -- 172.21.15.181:0/479210200 learned_addr learned my addr 172.21.15.181:0/479210200 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a69e60700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a640cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6a661700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a6413ec50 0x7f6a6416f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 -- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 msgr2=0x7f6a640cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a640cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 -- 172.21.15.181:0/479210200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a6413ec50 msgr2=0x7f6a6416f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a6413ec50 0x7f6a6416f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 -- 172.21.15.181:0/479210200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6416fe20 con 0x7f6a6413e870 2020-11-16T22:33:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6965f700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f6a6000db60 tx=0x7f6a60008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60009790 con 0x7f6a6413e870 2020-11-16T22:33:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a600098f0 con 0x7f6a6413e870 2020-11-16T22:33:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a641534c0 con 0x7f6a6413e870 2020-11-16T22:33:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.715+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60009c70 con 0x7f6a6413e870 2020-11-16T22:33:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6416f5f0 con 0x7f6a6413e870 2020-11-16T22:33:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6a60013690 con 0x7f6a6413e870 2020-11-16T22:33:39.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a51ffb700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 0x7f6a3c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a69e60700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 0x7f6a3c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a6006e260 con 0x7f6a6413e870 2020-11-16T22:33:39.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64044f60 con 0x7f6a6413e870 2020-11-16T22:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.719+0000 7f6a69e60700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 0x7f6a3c057520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6a54003300 tx=0x7f6a54007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.727+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6a60044460 con 0x7f6a6413e870 2020-11-16T22:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.883+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6a6405c310 con 0x7f6a6413e870 2020-11-16T22:33:39.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.883+0000 7f6a51ffb700 1 -- 172.21.15.181:0/479210200 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v81) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6a60047a80 con 0x7f6a6413e870 2020-11-16T22:33:39.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 msgr2=0x7f6a3c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 0x7f6a3c057520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6a54003300 tx=0x7f6a54007b90).stop 2020-11-16T22:33:39.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 msgr2=0x7f6a64154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f6a6000db60 tx=0x7f6a60008040).stop 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 shutdown_connections 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6a3c055060 0x7f6a3c057520 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a6413ec50 0x7f6a6416f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a640615f0 0x7f6a640cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 --2- 172.21.15.181:0/479210200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a6413e870 0x7f6a64154810 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 >> 172.21.15.181:0/479210200 conn(0x7f6a640cf630 msgr2=0x7f6a64130800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:39.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 shutdown_connections 2020-11-16T22:33:39.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:38.887+0000 7f6a6ae62700 1 -- 172.21.15.181:0/479210200 wait complete. 2020-11-16T22:33:39.324 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:33:39.324 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:33:39.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.475+0000 7f95bbfff700 1 Processor -- start 2020-11-16T22:33:39.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.475+0000 7f95bbfff700 1 -- start start 2020-11-16T22:33:39.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f95bc145a30 0x7f95bc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc061b50 con 0x7f95bc0814c0 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95bc0618b0 con 0x7f95bc145a30 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0cb930 con 0x7f95bc1492c0 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42492/0 (socket says 172.21.15.181:42492) 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 -- 172.21.15.181:0/114658004 learned_addr learned my addr 172.21.15.181:0/114658004 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:39.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95baffd700 1 --2- 172.21.15.181:0/114658004 >> v2:172.21.15.175:3300/0 conn(0x7f95bc145a30 0x7f95bc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 -- 172.21.15.181:0/114658004 >> v2:172.21.15.175:3300/0 conn(0x7f95bc145a30 msgr2=0x7f95bc145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:39.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 --2- 172.21.15.181:0/114658004 >> v2:172.21.15.175:3300/0 conn(0x7f95bc145a30 0x7f95bc145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 -- 172.21.15.181:0/114658004 >> v1:172.21.15.175:6789/0 conn(0x7f95bc0814c0 legacy=0x7f95bc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 -- 172.21.15.181:0/114658004 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0cbbd0 con 0x7f95bc1492c0 2020-11-16T22:33:39.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95ba7fc700 1 --2- 172.21.15.181:0/114658004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f95ac006fd0 tx=0x7f95ac006eb0).ready entity=mon.2 client_cookie=3caf2e8faf77e72c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:39.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95b9ffb700 1 -- 172.21.15.181:0/114658004 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac009810 con 0x7f95bc1492c0 2020-11-16T22:33:39.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95b9ffb700 1 -- 172.21.15.181:0/114658004 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac009970 con 0x7f95bc1492c0 2020-11-16T22:33:39.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95b9ffb700 1 -- 172.21.15.181:0/114658004 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac009cf0 con 0x7f95bc1492c0 2020-11-16T22:33:39.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- 172.21.15.181:0/114658004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 msgr2=0x7f95bc080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 --2- 172.21.15.181:0/114658004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f95ac006fd0 tx=0x7f95ac006eb0).stop 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- 172.21.15.181:0/114658004 shutdown_connections 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 --2- 172.21.15.181:0/114658004 >> v2:172.21.15.175:3300/0 conn(0x7f95bc145a30 0x7f95bc145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 --2- 172.21.15.181:0/114658004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc1492c0 0x7f95bc080fd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- 172.21.15.181:0/114658004 >> 172.21.15.181:0/114658004 conn(0x7f95bc0d68c0 msgr2=0x7f95bc0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:39.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- 172.21.15.181:0/114658004 shutdown_connections 2020-11-16T22:33:39.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.479+0000 7f95bbfff700 1 -- 172.21.15.181:0/114658004 wait complete. 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 Processor -- start 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 -- start start 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc145a30 0x7f95bc0d5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 0x7f95bc16f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc147f60 con 0x7f95bc1492c0 2020-11-16T22:33:39.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc148200 con 0x7f95bc145a30 2020-11-16T22:33:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0d5240 con 0x7f95bc0814c0 2020-11-16T22:33:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42494/0 (socket says 172.21.15.181:42494) 2020-11-16T22:33:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 -- 172.21.15.181:0/241161044 learned_addr learned my addr 172.21.15.181:0/241161044 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95ba7fc700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc145a30 0x7f95bc0d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bb7fe700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 0x7f95bc16f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 -- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc145a30 msgr2=0x7f95bc0d5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc145a30 0x7f95bc0d5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 -- 172.21.15.181:0/241161044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 msgr2=0x7f95bc16f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 0x7f95bc16f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 -- 172.21.15.181:0/241161044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0d54e0 con 0x7f95bc0814c0 2020-11-16T22:33:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bb7fe700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 0x7f95bc16f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95baffd700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f95a400eba0 tx=0x7f95a4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:40.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a400c9b0 con 0x7f95bc0814c0 2020-11-16T22:33:40.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a400cb10 con 0x7f95bc0814c0 2020-11-16T22:33:40.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc0d5780 con 0x7f95bc0814c0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.483+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a400ce90 con 0x7f95bc0814c0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc0d62b0 con 0x7f95bc0814c0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc044f60 con 0x7f95bc0814c0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f95a4012600 con 0x7f95bc0814c0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95b2ffd700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 0x7f9598057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95ba7fc700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 0x7f9598057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f95a406e2d0 con 0x7f95bc0814c0 2020-11-16T22:33:40.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.487+0000 7f95ba7fc700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 0x7f9598057520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f95ac00b1b0 tx=0x7f95ac015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:40.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.495+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f95a4044530 con 0x7f95bc0814c0 2020-11-16T22:33:40.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.647+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f95bc068e30 con 0x7f95bc0814c0 2020-11-16T22:33:40.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.647+0000 7f95b2ffd700 1 -- 172.21.15.181:0/241161044 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f95a4047b50 con 0x7f95bc0814c0 2020-11-16T22:33:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 msgr2=0x7f9598057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 0x7f9598057520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f95ac00b1b0 tx=0x7f95ac015000).stop 2020-11-16T22:33:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 msgr2=0x7f95bc1708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f95a400eba0 tx=0x7f95a4006df0).stop 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 shutdown_connections 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9598055060 0x7f9598057520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95bc1492c0 0x7f95bc16f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0814c0 0x7f95bc1708e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 --2- 172.21.15.181:0/241161044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc145a30 0x7f95bc0d5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 >> 172.21.15.181:0/241161044 conn(0x7f95bc0d68c0 msgr2=0x7f95bc140fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:40.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 shutdown_connections 2020-11-16T22:33:40.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:39.651+0000 7f95bbfff700 1 -- 172.21.15.181:0/241161044 wait complete. 2020-11-16T22:33:40.292 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:33:40.292 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:33:40.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.443+0000 7f049f59e700 1 Processor -- start 2020-11-16T22:33:40.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- start start 2020-11-16T22:33:40.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04a01492c0 0x7f04a00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a0061b50 con 0x7f04a0145a30 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a0149700 con 0x7f04a01492c0 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a00618b0 con 0x7f04a00816e0 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42502/0 (socket says 172.21.15.181:42502) 2020-11-16T22:33:40.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 -- 172.21.15.181:0/940033106 learned_addr learned my addr 172.21.15.181:0/940033106 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:40.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049ed9d700 1 --2- 172.21.15.181:0/940033106 >> v2:172.21.15.175:3300/0 conn(0x7f04a01492c0 0x7f04a00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 -- 172.21.15.181:0/940033106 >> v2:172.21.15.175:3300/0 conn(0x7f04a01492c0 msgr2=0x7f04a00cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:40.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 --2- 172.21.15.181:0/940033106 >> v2:172.21.15.175:3300/0 conn(0x7f04a01492c0 0x7f04a00cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 -- 172.21.15.181:0/940033106 >> v1:172.21.15.175:6789/0 conn(0x7f04a0145a30 legacy=0x7f04a0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 -- 172.21.15.181:0/940033106 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a00cb930 con 0x7f04a00816e0 2020-11-16T22:33:40.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049dd9b700 1 --2- 172.21.15.181:0/940033106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f0494006fd0 tx=0x7f0494006eb0).ready entity=mon.2 client_cookie=417be80fd5924cbf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:40.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049d59a700 1 -- 172.21.15.181:0/940033106 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494009810 con 0x7f04a00816e0 2020-11-16T22:33:40.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049d59a700 1 -- 172.21.15.181:0/940033106 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494009970 con 0x7f04a00816e0 2020-11-16T22:33:40.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049d59a700 1 -- 172.21.15.181:0/940033106 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494009cf0 con 0x7f04a00816e0 2020-11-16T22:33:40.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- 172.21.15.181:0/940033106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 msgr2=0x7f04a0145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 --2- 172.21.15.181:0/940033106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f0494006fd0 tx=0x7f0494006eb0).stop 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- 172.21.15.181:0/940033106 shutdown_connections 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 --2- 172.21.15.181:0/940033106 >> v2:172.21.15.175:3300/0 conn(0x7f04a01492c0 0x7f04a00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 --2- 172.21.15.181:0/940033106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a0145e10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- 172.21.15.181:0/940033106 >> 172.21.15.181:0/940033106 conn(0x7f04a00d68c0 msgr2=0x7f04a00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- 172.21.15.181:0/940033106 shutdown_connections 2020-11-16T22:33:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.447+0000 7f049f59e700 1 -- 172.21.15.181:0/940033106 wait complete. 2020-11-16T22:33:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 Processor -- start 2020-11-16T22:33:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 -- start start 2020-11-16T22:33:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a013cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 0x7f04a013d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a0143360 con 0x7f04a01492c0 2020-11-16T22:33:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a0141f00 con 0x7f04a0145a30 2020-11-16T22:33:40.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01421a0 con 0x7f04a00816e0 2020-11-16T22:33:40.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34668/0 (socket says 172.21.15.181:34668) 2020-11-16T22:33:40.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049dd9b700 1 -- 172.21.15.181:0/2360430759 learned_addr learned my addr 172.21.15.181:0/2360430759 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049e59c700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a013cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.451+0000 7f049ed9d700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 0x7f04a013d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 -- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 msgr2=0x7f04a013cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a013cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 -- 172.21.15.181:0/2360430759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 msgr2=0x7f04a013d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 0x7f04a013d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:40.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 -- 172.21.15.181:0/2360430759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0142440 con 0x7f04a0145a30 2020-11-16T22:33:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049ed9d700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 0x7f04a013d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:40.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049dd9b700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f049400b1b0 tx=0x7f0494015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494011d50 con 0x7f04a0145a30 2020-11-16T22:33:41.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049401e460 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049401e800 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a01426e0 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a0142c20 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a01456c0 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0494025030 con 0x7f04a0145a30 2020-11-16T22:33:41.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 0x7f0474057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f049406f490 con 0x7f04a0145a30 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049e59c700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 0x7f0474057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.455+0000 7f049e59c700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 0x7f0474057520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0490003300 tx=0x7f0490000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.463+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f04a01456c0 con 0x7f04a0145a30 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04a01456c0 con 0x7f04a0145a30 2020-11-16T22:33:41.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f048e7fc700 1 -- 172.21.15.181:0/2360430759 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+10 (secure 0 0 0) 0x7f04a01456c0 con 0x7f04a0145a30 2020-11-16T22:33:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 msgr2=0x7f0474057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 0x7f0474057520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0490003300 tx=0x7f0490000b80).stop 2020-11-16T22:33:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 msgr2=0x7f04a013d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f049400b1b0 tx=0x7f0494015000).stop 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 shutdown_connections 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0474055060 0x7f0474057520 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04a01492c0 0x7f04a013d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a00816e0 0x7f04a013cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 --2- 172.21.15.181:0/2360430759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a0145a30 0x7f04a013d000 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 >> 172.21.15.181:0/2360430759 conn(0x7f04a00d68c0 msgr2=0x7f04a015e340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.611+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 shutdown_connections 2020-11-16T22:33:41.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:40.615+0000 7f049f59e700 1 -- 172.21.15.181:0/2360430759 wait complete. 2020-11-16T22:33:41.260 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:33:41.260 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 Processor -- start 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 -- start start 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb015f5b0 0x7fcbb00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbb00615f0 0x7fcbb00d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb0160e40 con 0x7fcbb00622a0 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb01609a0 con 0x7fcbb00615f0 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb7543700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0160ba0 con 0x7fcbb015f5b0 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbb00615f0 0x7fcbb00d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbb00615f0 0x7fcbb00d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34678/0 (socket says 172.21.15.181:34678) 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb52df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb015f5b0 0x7fcbb00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 -- 172.21.15.181:0/930444130 learned_addr learned my addr 172.21.15.181:0/930444130 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb5ae0700 1 --1- 172.21.15.181:0/930444130 >> v1:172.21.15.175:6789/0 conn(0x7fcbb00622a0 0x7fcbb0062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:42092/0 (socket says 172.21.15.181:42092) 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 -- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb015f5b0 msgr2=0x7fcbb00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 --2- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb015f5b0 0x7fcbb00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 -- 172.21.15.181:0/930444130 >> v1:172.21.15.175:6789/0 conn(0x7fcbb00622a0 legacy=0x7fcbb0062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 -- 172.21.15.181:0/930444130 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0060970 con 0x7fcbb00615f0 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.407+0000 7fcbb4ade700 1 --2- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb00615f0 0x7fcbb00d1830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fcba0006fd0 tx=0x7fcba0006eb0).ready entity=mon.1 client_cookie=15f18f9c7450f0e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcba7fff700 1 -- 172.21.15.181:0/930444130 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0009810 con 0x7fcbb00615f0 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcba7fff700 1 -- 172.21.15.181:0/930444130 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0009970 con 0x7fcbb00615f0 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcba7fff700 1 -- 172.21.15.181:0/930444130 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0009cf0 con 0x7fcbb00615f0 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb00615f0 msgr2=0x7fcbb00d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb00615f0 0x7fcbb00d1830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fcba0006fd0 tx=0x7fcba0006eb0).stop 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- 172.21.15.181:0/930444130 shutdown_connections 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb015f5b0 0x7fcbb00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- 172.21.15.181:0/930444130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb00615f0 0x7fcbb00d1830 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- 172.21.15.181:0/930444130 >> 172.21.15.181:0/930444130 conn(0x7fcbb00d69a0 msgr2=0x7fcbb00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- 172.21.15.181:0/930444130 shutdown_connections 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- 172.21.15.181:0/930444130 wait complete. 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 Processor -- start 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 -- start start 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb00622a0 0x7fcbb00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.411+0000 7fcbb7543700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb015f5b0 0x7fcbb0081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00cf710 con 0x7fcbb00622a0 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0170180 con 0x7fcbb015f5b0 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00cf8f0 con 0x7fcbb00615f0 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42522/0 (socket says 172.21.15.181:42522) 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 -- 172.21.15.181:0/2361148510 learned_addr learned my addr 172.21.15.181:0/2361148510 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb5ae0700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb015f5b0 0x7fcbb0081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb4ade700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb00622a0 0x7fcbb00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 -- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb015f5b0 msgr2=0x7fcbb0081170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb015f5b0 0x7fcbb0081170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 -- 172.21.15.181:0/2361148510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb00622a0 msgr2=0x7fcbb00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb00622a0 0x7fcbb00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 -- 172.21.15.181:0/2361148510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00cfb90 con 0x7fcbb00615f0 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb52df700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fcbac00eba0 tx=0x7fcbac008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00b310 con 0x7fcbb00615f0 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac00c8e0 con 0x7fcbb00615f0 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb00cfe30 con 0x7fcbb00615f0 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00cc80 con 0x7fcbb00615f0 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb00d05f0 con 0x7fcbb00615f0 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0170340 con 0x7fcbb00615f0 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcbac0123f0 con 0x7fcbb00615f0 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 0x7fcb88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcbac06df60 con 0x7fcbb00615f0 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb4ade700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 0x7fcb88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.415+0000 7fcbb4ade700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 0x7fcb88057520 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fcbb00a7d50 tx=0x7fcba0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.419+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcbb0170340 con 0x7fcbb00615f0 2020-11-16T22:33:41.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcbb0170340 con 0x7fcbb00615f0 2020-11-16T22:33:41.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcba57fa700 1 -- 172.21.15.181:0/2361148510 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcbb0170340 con 0x7fcbb00615f0 2020-11-16T22:33:41.564 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:33:41.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 msgr2=0x7fcb88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 0x7fcb88057520 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fcbb00a7d50 tx=0x7fcba0015000).stop 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 msgr2=0x7fcbb00cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fcbac00eba0 tx=0x7fcbac008040).stop 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 shutdown_connections 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcb88055060 0x7fcb88057520 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb015f5b0 0x7fcbb0081170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbb00615f0 0x7fcbb00cb900 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 --2- 172.21.15.181:0/2361148510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbb00622a0 0x7fcbb00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 >> 172.21.15.181:0/2361148510 conn(0x7fcbb00d69a0 msgr2=0x7fcbb0061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 shutdown_connections 2020-11-16T22:33:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.575+0000 7fcbb7543700 1 -- 172.21.15.181:0/2361148510 wait complete. 2020-11-16T22:33:41.579 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 Processor -- start 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 -- start start 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb870161860 0x7fb870062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700c8db0 con 0x7fb870161480 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb870165030 con 0x7fb870062740 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb875613700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8700c8330 con 0x7fb870161860 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb874e12700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb870062740 0x7fb8700644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48994/0 (socket says 172.21.15.181:48994) 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb874e12700 1 -- 172.21.15.181:0/261618906 learned_addr learned my addr 172.21.15.181:0/261618906 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.711+0000 7fb86f7fe700 1 --2- 172.21.15.181:0/261618906 >> v2:172.21.15.181:3300/0 conn(0x7fb870161860 0x7fb870062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544634894 0 0) 0x7fb870165030 con 0x7fb870062740 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb85c003240 con 0x7fb870062740 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872984636 0 0) 0x7fb8700c8db0 con 0x7fb870161480 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb870165030 con 0x7fb870161480 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86f7fe700 1 -- 172.21.15.181:0/261618906 >> v1:172.21.15.181:6789/0 conn(0x7fb870062740 legacy=0x7fb8700644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1636348410 0 0) 0x7fb85c003240 con 0x7fb870062740 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86f7fe700 1 -- 172.21.15.181:0/261618906 >> v1:172.21.15.175:6789/0 conn(0x7fb870161480 legacy=0x7fb870164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86f7fe700 1 -- 172.21.15.181:0/261618906 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700c85d0 con 0x7fb870161860 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86f7fe700 1 --2- 172.21.15.181:0/261618906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161860 0x7fb870062250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fb86400b920 tx=0x7fb864006df0).ready entity=mon.1 client_cookie=7b73b897e6edbd0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868001860 con 0x7fb870161860 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864011400 con 0x7fb870161860 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb86effd700 1 -- 172.21.15.181:0/261618906 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864011780 con 0x7fb870161860 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- 172.21.15.181:0/261618906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161860 msgr2=0x7fb870062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 --2- 172.21.15.181:0/261618906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161860 0x7fb870062250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fb86400b920 tx=0x7fb864006df0).stop 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- 172.21.15.181:0/261618906 shutdown_connections 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 --2- 172.21.15.181:0/261618906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161860 0x7fb870062250 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- 172.21.15.181:0/261618906 >> 172.21.15.181:0/261618906 conn(0x7fb8700d2dc0 msgr2=0x7fb8700ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- 172.21.15.181:0/261618906 shutdown_connections 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- 172.21.15.181:0/261618906 wait complete. 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 Processor -- start 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- start start 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 0x7fb870176bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 0x7fb870162e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 0x7fb870163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870177ad0 con 0x7fb870062740 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701770b0 con 0x7fb870161480 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.715+0000 7fb875613700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701772f0 con 0x7fb870161860 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 0x7fb870162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 0x7fb870163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 0x7fb870162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34694/0 (socket says 172.21.15.181:34694) 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86f7fe700 1 -- 172.21.15.181:0/1060054088 learned_addr learned my addr 172.21.15.181:0/1060054088 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86ffff700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 0x7fb870176bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 -- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 msgr2=0x7fb870162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 0x7fb870162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 -- 172.21.15.181:0/1060054088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 msgr2=0x7fb870176bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 0x7fb870176bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 -- 172.21.15.181:0/1060054088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870177590 con 0x7fb870161860 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86ffff700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 0x7fb870176bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb874e12700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 0x7fb870163380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb86800cbb0 tx=0x7fb8680018a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868004a90 con 0x7fb870161860 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868004c10 con 0x7fb870161860 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb870177830 con 0x7fb870161860 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868007170 con 0x7fb870161860 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8700d19a0 con 0x7fb870161860 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb870044f60 con 0x7fb870161860 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb868010070 con 0x7fb870161860 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 0x7fb848057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb86806ddd0 con 0x7fb870161860 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.719+0000 7fb86ffff700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 0x7fb848057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.723+0000 7fb86ffff700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 0x7fb848057520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb8700c91e0 tx=0x7fb860009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.727+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb868043fd0 con 0x7fb870161860 2020-11-16T22:33:41.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8700671e0 con 0x7fb870161860 2020-11-16T22:33:41.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb84ffff700 1 -- 172.21.15.181:0/1060054088 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4665 (secure 0 0 0) 0x7fb8680475f0 con 0x7fb870161860 2020-11-16T22:33:41.920 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:41.921 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":88,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6383,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/889543331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":889543331},{"type":"v1","addr":"172.21.15.181:6835","nonce":889543331}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/3237613721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":3237613721},{"type":"v1","addr":"172.21.15.181:6841","nonce":3237613721}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6777,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/1458688332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1458688332},{"type":"v1","addr":"172.21.15.175:6833","nonce":1458688332}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6789,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3657509637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3657509637},{"type":"v1","addr":"172.21.15.175:6835","nonce":3657509637}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77},{"gid":6792,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/4045940499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":4045940499},{"type":"v1","addr":"172.21.15.175:6837","nonce":4045940499}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":76},{"gid":6795,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/4055783109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4055783109},{"type":"v1","addr":"172.21.15.175:6839","nonce":4055783109}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:32:39.489360+0000","modified":"2020-11-16T22:33:24.911953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6946,"mds_1":6943,"mds_2":6798},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6798":{"gid":6798,"name":"h","rank":2,"incarnation":84,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6841/2661622280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":2661622280},{"type":"v1","addr":"172.21.15.175:6841","nonce":2661622280}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6943":{"gid":6943,"name":"d","rank":1,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/1009245241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1009245241},{"type":"v1","addr":"172.21.15.181:6837","nonce":1009245241}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6946":{"gid":6946,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6839/1430907052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1430907052},{"type":"v1","addr":"172.21.15.181:6839","nonce":1430907052}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:33:41.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 msgr2=0x7fb848057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 0x7fb848057520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb8700c91e0 tx=0x7fb860009040).stop 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 msgr2=0x7fb870163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.931+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 0x7fb870163380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb86800cbb0 tx=0x7fb8680018a0).stop 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 shutdown_connections 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb848055060 0x7fb848057520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb870161860 0x7fb870163380 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb870062740 0x7fb870176bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 --2- 172.21.15.181:0/1060054088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb870161480 0x7fb870162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:41.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 >> 172.21.15.181:0/1060054088 conn(0x7fb8700d2dc0 msgr2=0x7fb8701480b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:41.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 shutdown_connections 2020-11-16T22:33:41.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:41.935+0000 7fb875613700 1 -- 172.21.15.181:0/1060054088 wait complete. 2020-11-16T22:33:41.925 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 88 2020-11-16T22:33:41.939 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:33:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 Processor -- start 2020-11-16T22:33:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 -- start start 2020-11-16T22:33:42.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f425c15f5b0 0x7f425c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c160e40 con 0x7f425c0622a0 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c1609a0 con 0x7f425c15f5b0 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f426413f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c160ba0 con 0x7f425c0615f0 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42544/0 (socket says 172.21.15.181:42544) 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 -- 172.21.15.181:0/36288536 learned_addr learned my addr 172.21.15.181:0/36288536 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:42.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f4261edb700 1 --2- 172.21.15.181:0/36288536 >> v2:172.21.15.175:3300/0 conn(0x7f425c15f5b0 0x7f425c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 -- 172.21.15.181:0/36288536 >> v2:172.21.15.175:3300/0 conn(0x7f425c15f5b0 msgr2=0x7f425c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:42.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 --2- 172.21.15.181:0/36288536 >> v2:172.21.15.175:3300/0 conn(0x7f425c15f5b0 0x7f425c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 -- 172.21.15.181:0/36288536 >> v1:172.21.15.175:6789/0 conn(0x7f425c0622a0 legacy=0x7f425c062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.067+0000 7f42616da700 1 -- 172.21.15.181:0/36288536 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c060970 con 0x7f425c0615f0 2020-11-16T22:33:42.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f42616da700 1 --2- 172.21.15.181:0/36288536 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f4258006fd0 tx=0x7f4258006eb0).ready entity=mon.2 client_cookie=5ddd363f54b593bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f4260ed9700 1 -- 172.21.15.181:0/36288536 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258009810 con 0x7f425c0615f0 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f4260ed9700 1 -- 172.21.15.181:0/36288536 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258009970 con 0x7f425c0615f0 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f4260ed9700 1 -- 172.21.15.181:0/36288536 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258009cf0 con 0x7f425c0615f0 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- 172.21.15.181:0/36288536 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 msgr2=0x7f425c0d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 --2- 172.21.15.181:0/36288536 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f4258006fd0 tx=0x7f4258006eb0).stop 2020-11-16T22:33:42.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- 172.21.15.181:0/36288536 shutdown_connections 2020-11-16T22:33:42.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 --2- 172.21.15.181:0/36288536 >> v2:172.21.15.175:3300/0 conn(0x7f425c15f5b0 0x7f425c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 --2- 172.21.15.181:0/36288536 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0615f0 0x7f425c0d1830 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- 172.21.15.181:0/36288536 >> 172.21.15.181:0/36288536 conn(0x7f425c0d69a0 msgr2=0x7f425c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:42.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- 172.21.15.181:0/36288536 shutdown_connections 2020-11-16T22:33:42.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- 172.21.15.181:0/36288536 wait complete. 2020-11-16T22:33:42.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 Processor -- start 2020-11-16T22:33:42.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.071+0000 7f426413f700 1 -- start start 2020-11-16T22:33:42.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 0x7f425c162be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c15f5b0 0x7f425c163710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c16f6c0 con 0x7f425c15f5b0 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c16f960 con 0x7f425c0615f0 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c16fc00 con 0x7f425c0622a0 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f42616da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 0x7f425c162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34710/0 (socket says 172.21.15.181:34710) 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f42616da700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 0x7f425c162be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42550/0 (socket says 172.21.15.181:42550) 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 -- 172.21.15.181:0/29812405 learned_addr learned my addr 172.21.15.181:0/29812405 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f42626dc700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c15f5b0 0x7f425c163710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 -- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 msgr2=0x7f425c162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 0x7f425c162be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 -- 172.21.15.181:0/29812405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c15f5b0 msgr2=0x7f425c163710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c15f5b0 0x7f425c163710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 -- 172.21.15.181:0/29812405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c16fea0 con 0x7f425c0615f0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4261edb700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f424c00eba0 tx=0x7f424c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00b310 con 0x7f425c0615f0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00c8e0 con 0x7f425c0615f0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c170140 con 0x7f425c0615f0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00cc80 con 0x7f425c0615f0 2020-11-16T22:33:42.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c16f370 con 0x7f425c0615f0 2020-11-16T22:33:42.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f424c0123f0 con 0x7f425c0615f0 2020-11-16T22:33:42.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.075+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c1702d0 con 0x7f425c0615f0 2020-11-16T22:33:42.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.079+0000 7f4249ffb700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 0x7f4234057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.079+0000 7f42616da700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 0x7f4234057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.079+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f424c06df60 con 0x7f425c0615f0 2020-11-16T22:33:42.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.079+0000 7f42616da700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 0x7f4234057520 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f4258006ef0 tx=0x7f4258015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.083+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f424c044160 con 0x7f425c0615f0 2020-11-16T22:33:42.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.239+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f425c069eb0 con 0x7f425c0615f0 2020-11-16T22:33:42.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f4249ffb700 1 -- 172.21.15.181:0/29812405 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f424c016070 con 0x7f425c0615f0 2020-11-16T22:33:42.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:42.233 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:32:40.492654+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:32:37.187776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:32:38.352981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:33:42.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 msgr2=0x7f4234057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 0x7f4234057520 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f4258006ef0 tx=0x7f4258015000).stop 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 msgr2=0x7f425c14a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f424c00eba0 tx=0x7f424c008040).stop 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 shutdown_connections 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4234055060 0x7f4234057520 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f425c15f5b0 0x7f425c163710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0615f0 0x7f425c14a630 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 --2- 172.21.15.181:0/29812405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c0622a0 0x7f425c162be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 >> 172.21.15.181:0/29812405 conn(0x7f425c0d69a0 msgr2=0x7f425c062700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:42.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 shutdown_connections 2020-11-16T22:33:42.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.243+0000 7f426413f700 1 -- 172.21.15.181:0/29812405 wait complete. 2020-11-16T22:33:42.247 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:33:42.248 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 Processor -- start 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- start start 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff0240816e0 0x7ff0240cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff024061b50 con 0x7ff0241492c0 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0240618b0 con 0x7ff0240816e0 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240cb930 con 0x7ff024145a30 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff022ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff0241492c0 0x7ff024080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49022/0 (socket says 172.21.15.181:49022) 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42558/0 (socket says 172.21.15.181:42558) 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff022ffd700 1 -- 172.21.15.181:0/2820281110 learned_addr learned my addr 172.21.15.181:0/2820281110 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff023fff700 1 --2- 172.21.15.181:0/2820281110 >> v2:172.21.15.175:3300/0 conn(0x7ff0240816e0 0x7ff0240cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0227fc700 1 -- 172.21.15.181:0/2820281110 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434715764 0 0) 0x7ff024061b50 con 0x7ff0241492c0 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0227fc700 1 -- 172.21.15.181:0/2820281110 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff010003240 con 0x7ff0241492c0 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 -- 172.21.15.181:0/2820281110 >> v2:172.21.15.175:3300/0 conn(0x7ff0240816e0 msgr2=0x7ff0240cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 --2- 172.21.15.181:0/2820281110 >> v2:172.21.15.175:3300/0 conn(0x7ff0240816e0 0x7ff0240cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 -- 172.21.15.181:0/2820281110 >> v1:172.21.15.181:6789/0 conn(0x7ff0241492c0 legacy=0x7ff024080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 -- 172.21.15.181:0/2820281110 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0240cbbd0 con 0x7ff024145a30 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0237fe700 1 --2- 172.21.15.181:0/2820281110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7ff014009c90 tx=0x7ff0140091d0).ready entity=mon.2 client_cookie=8596a5f9347cb98c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0227fc700 1 -- 172.21.15.181:0/2820281110 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01400ba70 con 0x7ff024145a30 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0227fc700 1 -- 172.21.15.181:0/2820281110 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01400bbd0 con 0x7ff024145a30 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff0227fc700 1 -- 172.21.15.181:0/2820281110 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014011400 con 0x7ff024145a30 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- 172.21.15.181:0/2820281110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 msgr2=0x7ff024145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 --2- 172.21.15.181:0/2820281110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7ff014009c90 tx=0x7ff0140091d0).stop 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- 172.21.15.181:0/2820281110 shutdown_connections 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 --2- 172.21.15.181:0/2820281110 >> v2:172.21.15.175:3300/0 conn(0x7ff0240816e0 0x7ff0240cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 --2- 172.21.15.181:0/2820281110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff024145a30 0x7ff024145e10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.383+0000 7ff028f91700 1 -- 172.21.15.181:0/2820281110 >> 172.21.15.181:0/2820281110 conn(0x7ff0240d68c0 msgr2=0x7ff0240d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- 172.21.15.181:0/2820281110 shutdown_connections 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- 172.21.15.181:0/2820281110 wait complete. 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 Processor -- start 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- start start 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff024145a30 0x7ff024164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0241492c0 0x7ff0241653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241433a0 con 0x7ff0241492c0 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff024143640 con 0x7ff024145a30 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241438e0 con 0x7ff0240816e0 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff024145a30 0x7ff024164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42564/0 (socket says 172.21.15.181:42564) 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 -- 172.21.15.181:0/2772198040 learned_addr learned my addr 172.21.15.181:0/2772198040 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff023fff700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0241492c0 0x7ff0241653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 -- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff024145a30 msgr2=0x7ff024164ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff024145a30 0x7ff024164ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 -- 172.21.15.181:0/2772198040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0241492c0 msgr2=0x7ff0241653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0241492c0 0x7ff0241653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 -- 172.21.15.181:0/2772198040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0241708a0 con 0x7ff0240816e0 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0237fe700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7ff01400b770 tx=0x7ff014016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014011e70 con 0x7ff0240816e0 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01401c5f0 con 0x7ff0240816e0 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.387+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff024170b40 con 0x7ff0240816e0 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01401c9b0 con 0x7ff0240816e0 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff024172890 con 0x7ff0240816e0 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff024170d00 con 0x7ff0240816e0 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff014017070 con 0x7ff0240816e0 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff0037fe700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 0x7feffc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff01406ff70 con 0x7ff0240816e0 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff022ffd700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 0x7feffc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.391+0000 7ff022ffd700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 0x7feffc057520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7ff024141c30 tx=0x7ff018009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.399+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff014099ef0 con 0x7ff0240816e0 2020-11-16T22:33:42.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:42.595+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff0241706c0 con 0x7ff0240816e0 2020-11-16T22:33:42.988 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:33:42.994+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:33:43.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.015+0000 7ff0037fe700 1 -- 172.21.15.181:0/2772198040 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff014046170 con 0x7ff0240816e0 2020-11-16T22:33:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 msgr2=0x7feffc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 0x7feffc057520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7ff024141c30 tx=0x7ff018009040).stop 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 msgr2=0x7ff024141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7ff01400b770 tx=0x7ff014016000).stop 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 shutdown_connections 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feffc055060 0x7feffc057520 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0241492c0 0x7ff0241653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0240816e0 0x7ff024141740 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 --2- 172.21.15.181:0/2772198040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff024145a30 0x7ff024164ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 >> 172.21.15.181:0/2772198040 conn(0x7ff0240d68c0 msgr2=0x7ff02413ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 shutdown_connections 2020-11-16T22:33:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.019+0000 7ff028f91700 1 -- 172.21.15.181:0/2772198040 wait complete. 2020-11-16T22:33:43.009 INFO:tasks.ceph.mds.a.smithi181.stderr: -9326> 2020-11-16T22:32:39.957+0000 7fc697e19700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:33:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:33:43.023 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:33:43.026 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:33:43.038 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:33:43.038 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:33:43.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 Processor -- start 2020-11-16T22:33:43.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 -- start start 2020-11-16T22:33:43.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff65413e870 0x7ff65413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff65415fdc0 0x7ff654160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff654160730 con 0x7ff65415fdc0 2020-11-16T22:33:43.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65415d180 con 0x7ff65413e870 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff658923700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65413f140 con 0x7ff654061100 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff6537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff65413e870 0x7ff65413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42598/0 (socket says 172.21.15.181:42598) 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 -- 172.21.15.181:0/1198519149 learned_addr learned my addr 172.21.15.181:0/1198519149 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff652ffd700 1 --2- 172.21.15.181:0/1198519149 >> v2:172.21.15.175:3300/0 conn(0x7ff65415fdc0 0x7ff654160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 -- 172.21.15.181:0/1198519149 >> v2:172.21.15.181:3300/0 conn(0x7ff65413e870 msgr2=0x7ff65413ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 --2- 172.21.15.181:0/1198519149 >> v2:172.21.15.181:3300/0 conn(0x7ff65413e870 0x7ff65413ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 -- 172.21.15.181:0/1198519149 >> v2:172.21.15.175:3300/0 conn(0x7ff65415fdc0 msgr2=0x7ff654160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 --2- 172.21.15.181:0/1198519149 >> v2:172.21.15.175:3300/0 conn(0x7ff65415fdc0 0x7ff654160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.155+0000 7ff653fff700 1 -- 172.21.15.181:0/1198519149 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65415d810 con 0x7ff654061100 2020-11-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff653fff700 1 --2- 172.21.15.181:0/1198519149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ff6540675c0 tx=0x7ff64c006eb0).ready entity=mon.2 client_cookie=712bf79eb66d6ba5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:43.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff6527fc700 1 -- 172.21.15.181:0/1198519149 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c009810 con 0x7ff654061100 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff6527fc700 1 -- 172.21.15.181:0/1198519149 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c009970 con 0x7ff654061100 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff6527fc700 1 -- 172.21.15.181:0/1198519149 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c009cf0 con 0x7ff654061100 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- 172.21.15.181:0/1198519149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 msgr2=0x7ff654061570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 --2- 172.21.15.181:0/1198519149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ff6540675c0 tx=0x7ff64c006eb0).stop 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- 172.21.15.181:0/1198519149 shutdown_connections 2020-11-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 --2- 172.21.15.181:0/1198519149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff654061100 0x7ff654061570 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 --2- 172.21.15.181:0/1198519149 >> v2:172.21.15.181:3300/0 conn(0x7ff65413e870 0x7ff65413ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 --2- 172.21.15.181:0/1198519149 >> v2:172.21.15.175:3300/0 conn(0x7ff65415fdc0 0x7ff654160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- 172.21.15.181:0/1198519149 >> 172.21.15.181:0/1198519149 conn(0x7ff6540cf630 msgr2=0x7ff6540cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- 172.21.15.181:0/1198519149 shutdown_connections 2020-11-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- 172.21.15.181:0/1198519149 wait complete. 2020-11-16T22:33:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 Processor -- start 2020-11-16T22:33:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.159+0000 7ff658923700 1 -- start start 2020-11-16T22:33:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff65413e870 0x7ff6540ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff65415fdc0 0x7ff6540b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540b6010 con 0x7ff65415fdc0 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540c7b90 con 0x7ff654061100 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540c7dc0 con 0x7ff65413e870 2020-11-16T22:33:43.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34760/0 (socket says 172.21.15.181:34760) 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff65413e870 0x7ff6540ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 -- 172.21.15.181:0/2488047777 learned_addr learned my addr 172.21.15.181:0/2488047777 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff653fff700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff65415fdc0 0x7ff6540b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 -- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff65413e870 msgr2=0x7ff6540ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:43.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff65413e870 0x7ff6540ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 -- 172.21.15.181:0/2488047777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff65415fdc0 msgr2=0x7ff6540b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff65415fdc0 0x7ff6540b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 -- 172.21.15.181:0/2488047777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6540c8060 con 0x7ff654061100 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6537fe700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7ff64400ef80 tx=0x7ff6440091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400cd40 con 0x7ff654061100 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64400cea0 con 0x7ff654061100 2020-11-16T22:33:43.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6540c8300 con 0x7ff654061100 2020-11-16T22:33:43.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644012600 con 0x7ff654061100 2020-11-16T22:33:43.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6540cb5b0 con 0x7ff654061100 2020-11-16T22:33:43.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.163+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654044f60 con 0x7ff654061100 2020-11-16T22:33:43.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.167+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff644022070 con 0x7ff654061100 2020-11-16T22:33:43.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.167+0000 7ff6337fe700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 0x7ff62c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:43.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.167+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6331+0+0 (secure 0 0 0) 0x7ff64406e710 con 0x7ff654061100 2020-11-16T22:33:43.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.167+0000 7ff652ffd700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 0x7ff62c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:43.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.167+0000 7ff652ffd700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 0x7ff62c057550 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7ff6540cfda0 tx=0x7ff648008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:43.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.171+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff644044870 con 0x7ff654061100 2020-11-16T22:33:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:43.371+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff65405c310 con 0x7ff654061100 2020-11-16T22:33:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.015+0000 7ff6337fe700 1 -- 172.21.15.181:0/2488047777 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff644047e90 con 0x7ff654061100 2020-11-16T22:33:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 msgr2=0x7ff62c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 0x7ff62c057550 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7ff6540cfda0 tx=0x7ff648008040).stop 2020-11-16T22:33:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 msgr2=0x7ff6540caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7ff64400ef80 tx=0x7ff6440091d0).stop 2020-11-16T22:33:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 shutdown_connections 2020-11-16T22:33:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff62c055090 0x7ff62c057550 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff65415fdc0 0x7ff6540b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff654061100 0x7ff6540caf60 secure :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0x7ff64400ef80 tx=0x7ff6440091d0).stop 2020-11-16T22:33:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 --2- 172.21.15.181:0/2488047777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff65413e870 0x7ff6540ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 >> 172.21.15.181:0/2488047777 conn(0x7ff6540cf630 msgr2=0x7ff6541321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 shutdown_connections 2020-11-16T22:33:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.019+0000 7ff658923700 1 -- 172.21.15.181:0/2488047777 wait complete. 2020-11-16T22:33:44.023 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:33:44.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.215+0000 7feba4d91700 1 Processor -- start 2020-11-16T22:33:44.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.215+0000 7feba4d91700 1 -- start start 2020-11-16T22:33:44.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feba4d91700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feba00b4e00 0x7feba00b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:44.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feba4d91700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba00cd670 con 0x7feba00613a0 2020-11-16T22:33:44.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feba4d91700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba00cd8a0 con 0x7feba0061780 2020-11-16T22:33:44.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feba4d91700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba00cdb40 con 0x7feba00b4e00 2020-11-16T22:33:44.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7feba0061780 0x7feba00ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49078/0 (socket says 172.21.15.181:49078) 2020-11-16T22:33:44.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9dd9b700 1 -- 172.21.15.181:0/3863576497 learned_addr learned my addr 172.21.15.181:0/3863576497 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:44.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/3863576497 >> v2:172.21.15.181:3300/0 conn(0x7feba00b4e00 0x7feba00b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:44.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703694972 0 0) 0x7feba00cd8a0 con 0x7feba0061780 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb84003240 con 0x7feba0061780 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9ed9d700 1 -- 172.21.15.181:0/3863576497 >> v1:172.21.15.181:6789/0 conn(0x7feba0061780 legacy=0x7feba00ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9ed9d700 1 -- 172.21.15.181:0/3863576497 >> v1:172.21.15.175:6789/0 conn(0x7feba00613a0 legacy=0x7feba00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086557419 0 0) 0x7feba00cd670 con 0x7feba00613a0 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9ed9d700 1 -- 172.21.15.181:0/3863576497 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00cdde0 con 0x7feba00b4e00 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/3863576497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba00b4e00 0x7feba00b5220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7feb98009c90 tx=0x7feb980091d0).ready entity=mon.1 client_cookie=fc210bb2ca8dcf80 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:44.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9800ba70 con 0x7feba00b4e00 2020-11-16T22:33:44.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9800bbd0 con 0x7feba00b4e00 2020-11-16T22:33:44.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.219+0000 7feb9d59a700 1 -- 172.21.15.181:0/3863576497 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98011400 con 0x7feba00b4e00 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- 172.21.15.181:0/3863576497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba00b4e00 msgr2=0x7feba00b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 --2- 172.21.15.181:0/3863576497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba00b4e00 0x7feba00b5220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7feb98009c90 tx=0x7feb980091d0).stop 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- 172.21.15.181:0/3863576497 shutdown_connections 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 --2- 172.21.15.181:0/3863576497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba00b4e00 0x7feba00b5220 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- 172.21.15.181:0/3863576497 >> 172.21.15.181:0/3863576497 conn(0x7feba00cf770 msgr2=0x7feba00cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- 172.21.15.181:0/3863576497 shutdown_connections 2020-11-16T22:33:44.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- 172.21.15.181:0/3863576497 wait complete. 2020-11-16T22:33:44.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 Processor -- start 2020-11-16T22:33:44.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.223+0000 7feba4d91700 1 -- start start 2020-11-16T22:33:44.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 0x7feba0140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:44.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:44.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 0x7feba0158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba016c2b0 con 0x7feba00613a0 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01590d0 con 0x7feba0061780 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba0159370 con 0x7feba00b4e00 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34780/0 (socket says 172.21.15.181:34780) 2020-11-16T22:33:44.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9dd9b700 1 -- 172.21.15.181:0/659730384 learned_addr learned my addr 172.21.15.181:0/659730384 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 0x7feba0158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9e59c700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 0x7feba0140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 -- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 msgr2=0x7feba0158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 -- 172.21.15.181:0/659730384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 msgr2=0x7feba0140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 0x7feba0140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:44.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 -- 172.21.15.181:0/659730384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0159610 con 0x7feba00b4e00 2020-11-16T22:33:44.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9e59c700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 0x7feba0140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:44.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9dd9b700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:44.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb9ed9d700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 0x7feba0158a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7feba00b5710 tx=0x7feb98016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:44.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9801c6c0 con 0x7feba00b4e00 2020-11-16T22:33:44.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9801c820 con 0x7feba00b4e00 2020-11-16T22:33:44.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba01598b0 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9801cbc0 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba0141320 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.227+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba00d00c0 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.231+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feb98017070 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.231+0000 7feb8e7fc700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 0x7feb74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.231+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6331+0+0 (secure 0 0 0) 0x7feb980703b0 con 0x7feba00b4e00 2020-11-16T22:33:44.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.231+0000 7feb9e59c700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 0x7feb74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:44.222 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.231+0000 7feb9e59c700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 0x7feb74057520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7feb900011a0 tx=0x7feb90009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:44.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.235+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7feba00d00c0 con 0x7feba00b4e00 2020-11-16T22:33:44.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:44.391+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feba00d00c0 con 0x7feba00b4e00 2020-11-16T22:33:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.019+0000 7feb8e7fc700 1 -- 172.21.15.181:0/659730384 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7feba00d00c0 con 0x7feba00b4e00 2020-11-16T22:33:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 msgr2=0x7feb74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 0x7feb74057520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7feb900011a0 tx=0x7feb90009040).stop 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 msgr2=0x7feba0158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 0x7feba0158a30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7feba00b5710 tx=0x7feb98016000).stop 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 shutdown_connections 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb74055060 0x7feb74057520 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba00b4e00 0x7feba0158a30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba00613a0 0x7feba0140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 --2- 172.21.15.181:0/659730384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba0061780 0x7feba0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 >> 172.21.15.181:0/659730384 conn(0x7feba00cf770 msgr2=0x7feba00d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 shutdown_connections 2020-11-16T22:33:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.027+0000 7feba4d91700 1 -- 172.21.15.181:0/659730384 wait complete. 2020-11-16T22:33:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:33:45.033 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 Processor -- start 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- start start 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8b700cdf70 0x7f8b70065b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b70066060 con 0x7f8b70143a10 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700ce6a0 con 0x7f8b70060290 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b70143ec0 con 0x7f8b700cdf70 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b75863700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8b70060290 0x7f8b70143300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49090/0 (socket says 172.21.15.181:49090) 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b75863700 1 -- 172.21.15.181:0/1340754539 learned_addr learned my addr 172.21.15.181:0/1340754539 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b76064700 1 --2- 172.21.15.181:0/1340754539 >> v2:172.21.15.175:3300/0 conn(0x7f8b700cdf70 0x7f8b70065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189812126 0 0) 0x7f8b700ce6a0 con 0x7f8b70060290 2020-11-16T22:33:45.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b58003240 con 0x7f8b70060290 2020-11-16T22:33:45.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b76064700 1 -- 172.21.15.181:0/1340754539 >> v1:172.21.15.181:6789/0 conn(0x7f8b70060290 legacy=0x7f8b70143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b76064700 1 -- 172.21.15.181:0/1340754539 >> v1:172.21.15.175:6789/0 conn(0x7f8b70143a10 legacy=0x7f8b70065460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956582545 0 0) 0x7f8b70066060 con 0x7f8b70143a10 2020-11-16T22:33:45.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b76064700 1 -- 172.21.15.181:0/1340754539 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700ce940 con 0x7f8b700cdf70 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b76064700 1 --2- 172.21.15.181:0/1340754539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700cdf70 0x7f8b70065b70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f8b68009c90 tx=0x7f8b680091d0).ready entity=mon.0 client_cookie=7b88d4cb3a736c49 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800bb00 con 0x7f8b700cdf70 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800bc60 con 0x7f8b700cdf70 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b74861700 1 -- 172.21.15.181:0/1340754539 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68005aa0 con 0x7f8b700cdf70 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- 172.21.15.181:0/1340754539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700cdf70 msgr2=0x7f8b70065b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/1340754539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700cdf70 0x7f8b70065b70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f8b68009c90 tx=0x7f8b680091d0).stop 2020-11-16T22:33:45.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 -- 172.21.15.181:0/1340754539 shutdown_connections 2020-11-16T22:33:45.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.159+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/1340754539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700cdf70 0x7f8b70065b70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- 172.21.15.181:0/1340754539 >> 172.21.15.181:0/1340754539 conn(0x7f8b700d35d0 msgr2=0x7f8b700d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:45.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- 172.21.15.181:0/1340754539 shutdown_connections 2020-11-16T22:33:45.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- 172.21.15.181:0/1340754539 wait complete. 2020-11-16T22:33:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 Processor -- start 2020-11-16T22:33:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- start start 2020-11-16T22:33:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b700cdf70 0x7f8b7016e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b70143a10 0x7f8b7016cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7016d700 con 0x7f8b70143a10 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7016d9a0 con 0x7f8b700cdf70 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b77ac7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700d1580 con 0x7f8b70060290 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75062700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b700cdf70 0x7f8b7016e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42634/0 (socket says 172.21.15.181:42634) 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 -- 172.21.15.181:0/340874973 learned_addr learned my addr 172.21.15.181:0/340874973 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:45.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b76064700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b70143a10 0x7f8b7016cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 -- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b700cdf70 msgr2=0x7f8b7016e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b700cdf70 0x7f8b7016e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 -- 172.21.15.181:0/340874973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b70143a10 msgr2=0x7f8b7016cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b70143a10 0x7f8b7016cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.163+0000 7f8b75863700 1 -- 172.21.15.181:0/340874973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700d1820 con 0x7f8b70060290 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b75863700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f8b6c001e90 tx=0x7f8b6c007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:45.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0072e0 con 0x7f8b70060290 2020-11-16T22:33:45.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700d1ac0 con 0x7f8b70060290 2020-11-16T22:33:45.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7016e7b0 con 0x7f8b70060290 2020-11-16T22:33:45.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c007440 con 0x7f8b70060290 2020-11-16T22:33:45.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c005820 con 0x7f8b70060290 2020-11-16T22:33:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8b6c010070 con 0x7f8b70060290 2020-11-16T22:33:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7016db60 con 0x7f8b70060290 2020-11-16T22:33:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 0x7f8b48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f8b6c06d380 con 0x7f8b70060290 2020-11-16T22:33:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b75062700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 0x7f8b48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:45.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.167+0000 7f8b75062700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 0x7f8b48057520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f8b700d3360 tx=0x7f8b60007a00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:45.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.171+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8b6c043690 con 0x7f8b70060290 2020-11-16T22:33:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:45.323+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b70069eb0 con 0x7f8b70060290 2020-11-16T22:33:46.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.027+0000 7f8b5dffb700 1 -- 172.21.15.181:0/340874973 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b6c046cb0 con 0x7f8b70060290 2020-11-16T22:33:46.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.031+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 msgr2=0x7f8b48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:46.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.031+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 0x7f8b48057520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f8b700d3360 tx=0x7f8b60007a00).stop 2020-11-16T22:33:46.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.031+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 msgr2=0x7f8b700d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:46.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.031+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f8b6c001e90 tx=0x7f8b6c007ac0).stop 2020-11-16T22:33:46.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 shutdown_connections 2020-11-16T22:33:46.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b48055060 0x7f8b48057520 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b70143a10 0x7f8b7016cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b70060290 0x7f8b700d1f60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 --2- 172.21.15.181:0/340874973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b700cdf70 0x7f8b7016e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 >> 172.21.15.181:0/340874973 conn(0x7f8b700d35d0 msgr2=0x7f8b700607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:46.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 shutdown_connections 2020-11-16T22:33:46.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.035+0000 7f8b77ac7700 1 -- 172.21.15.181:0/340874973 wait complete. 2020-11-16T22:33:46.026 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:33:46.039 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:33:46.039 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:33:46.039 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:33:46.040 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.040 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:33:46.040 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:33:46.041 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:33:46.041 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.041 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:33:46.041 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:33:46.042 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:33:46.042 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.042 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:33:46.042 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:33:46.043 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:33:46.043 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.043 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:33:46.043 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:33:46.044 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:33:46.044 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.044 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:33:46.044 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:33:46.045 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:33:46.045 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.045 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:33:46.045 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:33:46.046 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:33:46.046 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.046 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:33:46.047 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:33:46.047 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:33:46.047 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.047 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:33:46.048 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:33:46.048 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:33:46.048 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:33:46.053 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:33:46.054 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:33:46.054 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:33:46.054 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:33:46.055 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:33:46.055 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:33:46.057 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:33:46.057 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:33:46.060 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:33:46.060 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:33:46.061 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:33:46.062 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:33:46.063 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:33:46.063 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:33:46.064 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:33:46.065 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:33:46.066 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:33:46.066 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:33:46.067 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:33:46.067 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:33:46.067 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.068 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.101 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:33:46.101 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:33:46.101 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:33:46.102 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.102 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.103 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.103 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:33:46.103 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.111 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:33:46.111 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:33:46.112 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.112 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.113 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:33:46.113 DEBUG:teuthology.parallel:result is None 2020-11-16T22:33:46.113 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:33:46.132 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:33:46.133 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:33:46.172 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:33:46.173 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:33:46.174 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:33:46.174 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:33:46.182 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:33:46.183 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:33:46.196 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:33:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 Processor -- start 2020-11-16T22:33:46.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 -- start start 2020-11-16T22:33:46.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4440cd270 0x7ff4440b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 0x7ff4440b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff444160cc0 con 0x7ff4440b7c00 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4440b6550 con 0x7ff4440cd270 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.263+0000 7ff44ac05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440b67f0 con 0x7ff4441603f0 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff443fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4440cd270 0x7ff4440b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42688/0 (socket says 172.21.15.181:42688) 2020-11-16T22:33:46.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 -- 172.21.15.181:0/3045836975 learned_addr learned my addr 172.21.15.181:0/3045836975 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:46.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4491a2700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 0x7ff4440b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 -- 172.21.15.181:0/3045836975 >> v2:172.21.15.181:3300/0 conn(0x7ff4440cd270 msgr2=0x7ff4440b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.181:3300/0 conn(0x7ff4440cd270 0x7ff4440b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 -- 172.21.15.181:0/3045836975 >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 msgr2=0x7ff4440b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 0x7ff4440b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 -- 172.21.15.181:0/3045836975 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440b6a90 con 0x7ff4441603f0 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4491a2700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 0x7ff4440b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4489a1700 1 --2- 172.21.15.181:0/3045836975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ff44405ce10 tx=0x7ff430006eb0).ready entity=mon.2 client_cookie=2b1657a5634fb5d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4437fe700 1 -- 172.21.15.181:0/3045836975 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430009810 con 0x7ff4441603f0 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4437fe700 1 -- 172.21.15.181:0/3045836975 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff430009970 con 0x7ff4441603f0 2020-11-16T22:33:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff4437fe700 1 -- 172.21.15.181:0/3045836975 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430009cf0 con 0x7ff4441603f0 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 -- 172.21.15.181:0/3045836975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 msgr2=0x7ff4441607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 --2- 172.21.15.181:0/3045836975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ff44405ce10 tx=0x7ff430006eb0).stop 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 -- 172.21.15.181:0/3045836975 shutdown_connections 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.175:3300/0 conn(0x7ff4440b7c00 0x7ff4440b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 --2- 172.21.15.181:0/3045836975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff4441607d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 --2- 172.21.15.181:0/3045836975 >> v2:172.21.15.181:3300/0 conn(0x7ff4440cd270 0x7ff4440b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 -- 172.21.15.181:0/3045836975 >> 172.21.15.181:0/3045836975 conn(0x7ff444060b70 msgr2=0x7ff4440d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 -- 172.21.15.181:0/3045836975 shutdown_connections 2020-11-16T22:33:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.267+0000 7ff44ac05700 1 -- 172.21.15.181:0/3045836975 wait complete. 2020-11-16T22:33:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 Processor -- start 2020-11-16T22:33:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 -- start start 2020-11-16T22:33:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 0x7ff4440c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 0x7ff44416f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff44416faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441705f0 con 0x7ff4440b7c00 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441700b0 con 0x7ff4440cd270 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff44ac05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff444170350 con 0x7ff4441603f0 2020-11-16T22:33:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff4491a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff44416faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 0x7ff44416f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff4491a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff44416faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42696/0 (socket says 172.21.15.181:42696) 2020-11-16T22:33:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff4491a2700 1 -- 172.21.15.181:0/2253177788 learned_addr learned my addr 172.21.15.181:0/2253177788 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff4489a1700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 0x7ff4440c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 -- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 msgr2=0x7ff44416faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:46.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff44416faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 -- 172.21.15.181:0/2253177788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 msgr2=0x7ff4440c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 0x7ff4440c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 -- 172.21.15.181:0/2253177788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44415e200 con 0x7ff4440cd270 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff4489a1700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 0x7ff4440c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.271+0000 7ff443fff700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 0x7ff44416f5b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7ff4440d0130 tx=0x7ff438006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800c820 con 0x7ff4440cd270 2020-11-16T22:33:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43800c9a0 con 0x7ff4440cd270 2020-11-16T22:33:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44415e4a0 con 0x7ff4440cd270 2020-11-16T22:33:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800cd60 con 0x7ff4440cd270 2020-11-16T22:33:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4440c8fe0 con 0x7ff4440cd270 2020-11-16T22:33:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff444069eb0 con 0x7ff4440cd270 2020-11-16T22:33:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff43801a030 con 0x7ff4440cd270 2020-11-16T22:33:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 0x7ff41c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff4489a1700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 0x7ff41c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5511+0+0 (secure 0 0 0) 0x7ff43806d7c0 con 0x7ff4440cd270 2020-11-16T22:33:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.275+0000 7ff4489a1700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 0x7ff41c057520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff430006ef0 tx=0x7ff430017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:46.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.283+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff438043c50 con 0x7ff4440cd270 2020-11-16T22:33:46.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:46.451+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff4440cfe60 con 0x7ff4440cd270 2020-11-16T22:33:47.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.239+0000 7ff440ff9700 1 -- 172.21.15.181:0/2253177788 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff438047270 con 0x7ff4440cd270 2020-11-16T22:33:47.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 msgr2=0x7ff41c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 0x7ff41c057520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff430006ef0 tx=0x7ff430017000).stop 2020-11-16T22:33:47.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 msgr2=0x7ff44416f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 0x7ff44416f5b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7ff4440d0130 tx=0x7ff438006df0).stop 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 shutdown_connections 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff41c055060 0x7ff41c057520 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4441603f0 0x7ff44416faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4440b7c00 0x7ff4440c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 --2- 172.21.15.181:0/2253177788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4440cd270 0x7ff44416f5b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 >> 172.21.15.181:0/2253177788 conn(0x7ff444060b70 msgr2=0x7ff44415d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 shutdown_connections 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.243+0000 7ff44ac05700 1 -- 172.21.15.181:0/2253177788 wait complete. 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:33:47.247 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:33:47.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 Processor -- start 2020-11-16T22:33:47.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- start start 2020-11-16T22:33:47.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc41415f5b0 0x7fc4140d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc414160e40 con 0x7fc4140622a0 2020-11-16T22:33:47.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4141609a0 con 0x7fc41415f5b0 2020-11-16T22:33:47.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc414160ba0 con 0x7fc4140615f0 2020-11-16T22:33:47.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42708/0 (socket says 172.21.15.181:42708) 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 -- 172.21.15.181:0/3345800570 learned_addr learned my addr 172.21.15.181:0/3345800570 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc418e64700 1 --2- 172.21.15.181:0/3345800570 >> v2:172.21.15.175:3300/0 conn(0x7fc41415f5b0 0x7fc4140d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 -- 172.21.15.181:0/3345800570 >> v2:172.21.15.175:3300/0 conn(0x7fc41415f5b0 msgr2=0x7fc4140d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 --2- 172.21.15.181:0/3345800570 >> v2:172.21.15.175:3300/0 conn(0x7fc41415f5b0 0x7fc4140d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 -- 172.21.15.181:0/3345800570 >> v1:172.21.15.175:6789/0 conn(0x7fc4140622a0 legacy=0x7fc414062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 -- 172.21.15.181:0/3345800570 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414060970 con 0x7fc4140615f0 2020-11-16T22:33:47.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc413fff700 1 --2- 172.21.15.181:0/3345800570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fc404006fd0 tx=0x7fc404006eb0).ready entity=mon.2 client_cookie=81c09df8928623f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:47.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc4137fe700 1 -- 172.21.15.181:0/3345800570 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404009810 con 0x7fc4140615f0 2020-11-16T22:33:47.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc4137fe700 1 -- 172.21.15.181:0/3345800570 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404009970 con 0x7fc4140615f0 2020-11-16T22:33:47.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc4137fe700 1 -- 172.21.15.181:0/3345800570 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404009cf0 con 0x7fc4140615f0 2020-11-16T22:33:47.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3345800570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 msgr2=0x7fc4140d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3345800570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fc404006fd0 tx=0x7fc404006eb0).stop 2020-11-16T22:33:47.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3345800570 shutdown_connections 2020-11-16T22:33:47.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3345800570 >> v2:172.21.15.175:3300/0 conn(0x7fc41415f5b0 0x7fc4140d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3345800570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4140615f0 0x7fc4140d1830 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3345800570 >> 172.21.15.181:0/3345800570 conn(0x7fc4140d69a0 msgr2=0x7fc4140d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.375+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3345800570 shutdown_connections 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3345800570 wait complete. 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 Processor -- start 2020-11-16T22:33:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- start start 2020-11-16T22:33:47.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 0x7fc4140d63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4141477f0 0x7fc414080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414170960 con 0x7fc4140622a0 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41415c630 con 0x7fc4141477f0 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41415c8d0 con 0x7fc41415f5b0 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42712/0 (socket says 172.21.15.181:42712) 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 -- 172.21.15.181:0/3592213019 learned_addr learned my addr 172.21.15.181:0/3592213019 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:47.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc418e64700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 0x7fc4140d63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc419665700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4141477f0 0x7fc414080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 -- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4141477f0 msgr2=0x7fc414080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4141477f0 0x7fc414080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 -- 172.21.15.181:0/3592213019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 msgr2=0x7fc4140d63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 0x7fc4140d63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 -- 172.21.15.181:0/3592213019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415cb70 con 0x7fc41415f5b0 2020-11-16T22:33:47.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc418e64700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 0x7fc4140d63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc413fff700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fc4140b92a0 tx=0x7fc404015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011d00 con 0x7fc41415f5b0 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404011e60 con 0x7fc41415f5b0 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41415ce10 con 0x7fc41415f5b0 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40401e670 con 0x7fc41415f5b0 2020-11-16T22:33:47.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.379+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41415c280 con 0x7fc41415f5b0 2020-11-16T22:33:47.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc404025030 con 0x7fc41415f5b0 2020-11-16T22:33:47.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc414069eb0 con 0x7fc41415f5b0 2020-11-16T22:33:47.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc410ff9700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 0x7fc3e4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:47.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc40406f0d0 con 0x7fc41415f5b0 2020-11-16T22:33:47.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc418e64700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 0x7fc3e4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:47.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.383+0000 7fc418e64700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 0x7fc3e4057550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fc40c006fd0 tx=0x7fc40c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:47.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.387+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc404045710 con 0x7fc41415f5b0 2020-11-16T22:33:47.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc41415c410 con 0x7fc41415f5b0 2020-11-16T22:33:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc410ff9700 1 -- 172.21.15.181:0/3592213019 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5978 (secure 0 0 0) 0x7fc404023070 con 0x7fc41415f5b0 2020-11-16T22:33:47.581 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:47.581 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.582 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.583 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.584 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.585 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.586 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.587 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.588 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:33:47.589 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.591 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.794 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.794 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.794 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.795 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.796 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.797 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.798 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:33:47.799 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:47.800 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:47.800 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:47.800 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:47.800 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:47.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:33:48.002 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.004 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:33:48.006 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:33:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:33:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.008 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.214 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.214 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.215 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.216 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:33:48.218 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:33:48.219 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.220 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:33:48.220 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:33:48.220 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:33:48.220 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.220 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.426 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.427 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:33:48.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:33:48.429 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:33:48.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:48.429 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:33:48.429 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:33:48.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 msgr2=0x7fc3e4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 0x7fc3e4057550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fc40c006fd0 tx=0x7fc40c008040).stop 2020-11-16T22:33:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 msgr2=0x7fc414147300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fc4140b92a0 tx=0x7fc404015000).stop 2020-11-16T22:33:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.591+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 shutdown_connections 2020-11-16T22:33:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc3e4055090 0x7fc3e4057550 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4141477f0 0x7fc414080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4140622a0 0x7fc4140d63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 --2- 172.21.15.181:0/3592213019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41415f5b0 0x7fc414147300 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 >> 172.21.15.181:0/3592213019 conn(0x7fc4140d69a0 msgr2=0x7fc414062700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 shutdown_connections 2020-11-16T22:33:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:47.595+0000 7fc41b0c8700 1 -- 172.21.15.181:0/3592213019 wait complete. 2020-11-16T22:33:48.635 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:33:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.779+0000 7f3398f2b700 1 Processor -- start 2020-11-16T22:33:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.779+0000 7f3398f2b700 1 -- start start 2020-11-16T22:33:48.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 0x7f339415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3394061b70 con 0x7f33940ce750 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339415cda0 con 0x7f339415fa10 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339415d040 con 0x7f33940612a0 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42718/0 (socket says 172.21.15.181:42718) 2020-11-16T22:33:48.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 -- 172.21.15.181:0/4218408992 learned_addr learned my addr 172.21.15.181:0/4218408992 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:48.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3392d9d700 1 --2- 172.21.15.181:0/4218408992 >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 0x7f339415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:48.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 -- 172.21.15.181:0/4218408992 >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 msgr2=0x7f339415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:48.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 --2- 172.21.15.181:0/4218408992 >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 0x7f339415fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 -- 172.21.15.181:0/4218408992 >> v1:172.21.15.175:6789/0 conn(0x7f33940ce750 legacy=0x7f33940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:48.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 -- 172.21.15.181:0/4218408992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339415d2e0 con 0x7f33940612a0 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3392d9d700 1 --2- 172.21.15.181:0/4218408992 >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 0x7f339415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339259c700 1 --2- 172.21.15.181:0/4218408992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f337c00dd20 tx=0x7f337c006e40).ready entity=mon.2 client_cookie=afa23c7349d7553c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339159a700 1 -- 172.21.15.181:0/4218408992 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00bdd0 con 0x7f33940612a0 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339159a700 1 -- 172.21.15.181:0/4218408992 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c0153f0 con 0x7f33940612a0 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f339159a700 1 -- 172.21.15.181:0/4218408992 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c015770 con 0x7f33940612a0 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- 172.21.15.181:0/4218408992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 msgr2=0x7f3394061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 --2- 172.21.15.181:0/4218408992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f337c00dd20 tx=0x7f337c006e40).stop 2020-11-16T22:33:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- 172.21.15.181:0/4218408992 shutdown_connections 2020-11-16T22:33:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 --2- 172.21.15.181:0/4218408992 >> v2:172.21.15.175:3300/0 conn(0x7f339415fa10 0x7f339415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 --2- 172.21.15.181:0/4218408992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f3394061680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- 172.21.15.181:0/4218408992 >> 172.21.15.181:0/4218408992 conn(0x7f33940cf890 msgr2=0x7f33940cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:48.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- 172.21.15.181:0/4218408992 shutdown_connections 2020-11-16T22:33:48.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.783+0000 7f3398f2b700 1 -- 172.21.15.181:0/4218408992 wait complete. 2020-11-16T22:33:48.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 Processor -- start 2020-11-16T22:33:48.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 -- start start 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33940ce750 0x7f339416b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 0x7f339416be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339416c2f0 con 0x7f339415fa10 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339416d6d0 con 0x7f33940ce750 2020-11-16T22:33:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339416d970 con 0x7f33940612a0 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42724/0 (socket says 172.21.15.181:42724) 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3391d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33940ce750 0x7f339416b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 -- 172.21.15.181:0/609390097 learned_addr learned my addr 172.21.15.181:0/609390097 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3392d9d700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 0x7f339416be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 -- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33940ce750 msgr2=0x7f339416b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33940ce750 0x7f339416b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 -- 172.21.15.181:0/609390097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 msgr2=0x7f339416be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 0x7f339416be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 -- 172.21.15.181:0/609390097 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339416dc10 con 0x7f33940612a0 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3392d9d700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 0x7f339416be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f339259c700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f337c00d4a0 tx=0x7f337c018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c022070 con 0x7f33940612a0 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c0157a0 con 0x7f33940612a0 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.787+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339416deb0 con 0x7f33940612a0 2020-11-16T22:33:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00c820 con 0x7f33940612a0 2020-11-16T22:33:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339416e3f0 con 0x7f33940612a0 2020-11-16T22:33:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3394142820 con 0x7f33940612a0 2020-11-16T22:33:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f337c00cb90 con 0x7f33940612a0 2020-11-16T22:33:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f338a7fc700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 0x7f3370057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f337c0714d0 con 0x7f33940612a0 2020-11-16T22:33:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f3391d9b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 0x7f3370057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.791+0000 7f3391d9b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 0x7f3370057550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f33940cdcf0 tx=0x7f3384007ae0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.795+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f337c047ae0 con 0x7f33940612a0 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.951+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3394142820 con 0x7f33940612a0 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f338a7fc700 1 -- 172.21.15.181:0/609390097 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13724 (secure 0 0 0) 0x7f337c04b100 con 0x7f33940612a0 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 msgr2=0x7f3370057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 0x7f3370057550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f33940cdcf0 tx=0x7f3384007ae0).stop 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 msgr2=0x7f33940a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f337c00d4a0 tx=0x7f337c018000).stop 2020-11-16T22:33:49.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 shutdown_connections 2020-11-16T22:33:49.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3370055090 0x7f3370057550 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339415fa10 0x7f339416be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33940612a0 0x7f33940a2d00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 --2- 172.21.15.181:0/609390097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33940ce750 0x7f339416b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:49.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 >> 172.21.15.181:0/609390097 conn(0x7f33940cf890 msgr2=0x7f3394160170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:49.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 shutdown_connections 2020-11-16T22:33:49.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:48.955+0000 7f3398f2b700 1 -- 172.21.15.181:0/609390097 wait complete. 2020-11-16T22:33:49.595 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:49.595 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":88,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:47.226231+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:33:50.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:33:50.414 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:33:50.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.563+0000 7f9389a01700 1 Processor -- start 2020-11-16T22:33:50.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.563+0000 7f9389a01700 1 -- start start 2020-11-16T22:33:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938415d4e0 0x7f938415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9384146e10 0x7f93840600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938415e090 con 0x7f9384147290 2020-11-16T22:33:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938415edb0 con 0x7f9384146e10 2020-11-16T22:33:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938415e330 con 0x7f938415d4e0 2020-11-16T22:33:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9384146e10 0x7f93840600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9384146e10 0x7f93840600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34894/0 (socket says 172.21.15.181:34894) 2020-11-16T22:33:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 -- 172.21.15.181:0/3805977503 learned_addr learned my addr 172.21.15.181:0/3805977503 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93837fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9384147290 0x7f93840605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49196/0 (socket says 172.21.15.181:49196) 2020-11-16T22:33:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9382ffd700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938415d4e0 0x7f938415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 -- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938415d4e0 msgr2=0x7f938415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938415d4e0 0x7f938415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 -- 172.21.15.181:0/3805977503 >> v1:172.21.15.181:6789/0 conn(0x7f9384147290 legacy=0x7f93840605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 -- 172.21.15.181:0/3805977503 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938415e5d0 con 0x7f9384146e10 2020-11-16T22:33:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9381ffb700 1 -- 172.21.15.181:0/3805977503 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411880546 0 0) 0x7f938415e090 con 0x7f9384147290 2020-11-16T22:33:50.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f93827fc700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f93840600f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f9374006fd0 tx=0x7f9374006eb0).ready entity=mon.1 client_cookie=2bd5c1ce8df76174 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:50.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9381ffb700 1 -- 172.21.15.181:0/3805977503 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374009810 con 0x7f9384146e10 2020-11-16T22:33:50.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9381ffb700 1 -- 172.21.15.181:0/3805977503 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374009970 con 0x7f9384146e10 2020-11-16T22:33:50.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9381ffb700 1 -- 172.21.15.181:0/3805977503 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374009cf0 con 0x7f9384146e10 2020-11-16T22:33:50.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 msgr2=0x7f93840600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f93840600f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f9374006fd0 tx=0x7f9374006eb0).stop 2020-11-16T22:33:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- 172.21.15.181:0/3805977503 shutdown_connections 2020-11-16T22:33:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938415d4e0 0x7f938415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 --2- 172.21.15.181:0/3805977503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f93840600f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:50.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- 172.21.15.181:0/3805977503 >> 172.21.15.181:0/3805977503 conn(0x7f93840d3680 msgr2=0x7f93840d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:50.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- 172.21.15.181:0/3805977503 shutdown_connections 2020-11-16T22:33:50.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 -- 172.21.15.181:0/3805977503 wait complete. 2020-11-16T22:33:50.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.567+0000 7f9389a01700 1 Processor -- start 2020-11-16T22:33:50.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- start start 2020-11-16T22:33:50.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f9384158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:50.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 0x7f9384141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384142da0 con 0x7f938415d4e0 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384141be0 con 0x7f9384146e10 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9384141e80 con 0x7f9384147290 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:50.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42738/0 (socket says 172.21.15.181:42738) 2020-11-16T22:33:50.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 -- 172.21.15.181:0/283737978 learned_addr learned my addr 172.21.15.181:0/283737978 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:50.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9382ffd700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f9384158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:50.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93837fe700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 0x7f9384141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 -- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 msgr2=0x7f9384158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f9384158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 -- 172.21.15.181:0/283737978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 msgr2=0x7f9384141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 0x7f9384141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 -- 172.21.15.181:0/283737978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384142120 con 0x7f9384147290 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93837fe700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 0x7f9384141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:33:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9382ffd700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f9384158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f93827fc700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f93840673b0 tx=0x7f9374015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374011d50 con 0x7f9384147290 2020-11-16T22:33:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937401e460 con 0x7f9384147290 2020-11-16T22:33:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937401e7e0 con 0x7f9384147290 2020-11-16T22:33:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93841423c0 con 0x7f9384147290 2020-11-16T22:33:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9384142900 con 0x7f9384147290 2020-11-16T22:33:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.571+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384067000 con 0x7f9384147290 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.575+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9374025030 con 0x7f9384147290 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.575+0000 7f937affd700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 0x7f9360057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.575+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f937406f170 con 0x7f9384147290 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.575+0000 7f9382ffd700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 0x7f9360057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.575+0000 7f9382ffd700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 0x7f9360057520 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f936c006fd0 tx=0x7f936c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.579+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9374045830 con 0x7f9384147290 2020-11-16T22:33:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.727+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9384142ac0 con 0x7f9384147290 2020-11-16T22:33:51.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f937affd700 1 -- 172.21.15.181:0/283737978 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v93) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9374023070 con 0x7f9384147290 2020-11-16T22:33:51.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 msgr2=0x7f9360057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 0x7f9360057520 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f936c006fd0 tx=0x7f936c008040).stop 2020-11-16T22:33:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 msgr2=0x7f93841594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f93840673b0 tx=0x7f9374015000).stop 2020-11-16T22:33:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 shutdown_connections 2020-11-16T22:33:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9360055060 0x7f9360057520 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938415d4e0 0x7f9384141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9384146e10 0x7f9384158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 --2- 172.21.15.181:0/283737978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9384147290 0x7f93841594e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 >> 172.21.15.181:0/283737978 conn(0x7f93840d3680 msgr2=0x7f93841445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 shutdown_connections 2020-11-16T22:33:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:50.731+0000 7f9389a01700 1 -- 172.21.15.181:0/283737978 wait complete. 2020-11-16T22:33:51.380 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:33:51.380 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:33:51.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.531+0000 7fa0733dd700 1 Processor -- start 2020-11-16T22:33:51.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.531+0000 7fa0733dd700 1 -- start start 2020-11-16T22:33:51.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 0x7fa06c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:51.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c15d180 con 0x7fa06c13ec50 2020-11-16T22:33:51.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c15d810 con 0x7fa06c13e870 2020-11-16T22:33:51.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c15dab0 con 0x7fa06c061810 2020-11-16T22:33:51.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0723db700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa06c13e870 0x7fa06c15fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49214/0 (socket says 172.21.15.181:49214) 2020-11-16T22:33:51.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0723db700 1 -- 172.21.15.181:0/2095805104 learned_addr learned my addr 172.21.15.181:0/2095805104 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:51.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa072bdc700 1 --2- 172.21.15.181:0/2095805104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 0x7fa06c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:51.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0713d9700 1 -- 172.21.15.181:0/2095805104 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216600781 0 0) 0x7fa06c15d810 con 0x7fa06c13e870 2020-11-16T22:33:51.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0713d9700 1 -- 172.21.15.181:0/2095805104 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa058003240 con 0x7fa06c13e870 2020-11-16T22:33:51.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa072bdc700 1 -- 172.21.15.181:0/2095805104 >> v1:172.21.15.181:6789/0 conn(0x7fa06c13e870 legacy=0x7fa06c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa072bdc700 1 -- 172.21.15.181:0/2095805104 >> v1:172.21.15.175:6789/0 conn(0x7fa06c13ec50 legacy=0x7fa06c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa072bdc700 1 -- 172.21.15.181:0/2095805104 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c15dd50 con 0x7fa06c061810 2020-11-16T22:33:51.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa072bdc700 1 --2- 172.21.15.181:0/2095805104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 0x7fa06c15e730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fa05c00b9d0 tx=0x7fa05c006df0).ready entity=mon.2 client_cookie=b446185f7abeb036 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:51.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0713d9700 1 -- 172.21.15.181:0/2095805104 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0097b0 con 0x7fa06c061810 2020-11-16T22:33:51.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0713d9700 1 -- 172.21.15.181:0/2095805104 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c009910 con 0x7fa06c061810 2020-11-16T22:33:51.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0713d9700 1 -- 172.21.15.181:0/2095805104 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c009c90 con 0x7fa06c061810 2020-11-16T22:33:51.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- 172.21.15.181:0/2095805104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 msgr2=0x7fa06c15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 --2- 172.21.15.181:0/2095805104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 0x7fa06c15e730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fa05c00b9d0 tx=0x7fa05c006df0).stop 2020-11-16T22:33:51.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- 172.21.15.181:0/2095805104 shutdown_connections 2020-11-16T22:33:51.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 --2- 172.21.15.181:0/2095805104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c061810 0x7fa06c15e730 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- 172.21.15.181:0/2095805104 >> 172.21.15.181:0/2095805104 conn(0x7fa06c0cf630 msgr2=0x7fa06c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:51.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- 172.21.15.181:0/2095805104 shutdown_connections 2020-11-16T22:33:51.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.535+0000 7fa0733dd700 1 -- 172.21.15.181:0/2095805104 wait complete. 2020-11-16T22:33:51.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 Processor -- start 2020-11-16T22:33:51.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 -- start start 2020-11-16T22:33:51.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa06c061810 0x7fa06c13acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:51.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c13e870 0x7fa06c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:51.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0caf60 con 0x7fa06c061810 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c16f530 con 0x7fa06c13ec50 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0733dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c16f7d0 con 0x7fa06c13e870 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34920/0 (socket says 172.21.15.181:34920) 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 -- 172.21.15.181:0/617274019 learned_addr learned my addr 172.21.15.181:0/617274019 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa071bda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c13e870 0x7fa06c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:51.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa0723db700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa06c061810 0x7fa06c13acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:51.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 -- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c13e870 msgr2=0x7fa06c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c13e870 0x7fa06c15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 -- 172.21.15.181:0/617274019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa06c061810 msgr2=0x7fa06c13acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa06c061810 0x7fa06c13acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 -- 172.21.15.181:0/617274019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c16fa70 con 0x7fa06c13ec50 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.539+0000 7fa072bdc700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fa05c00b920 tx=0x7fa05c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c01e560 con 0x7fa06c13ec50 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c01e6c0 con 0x7fa06c13ec50 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c01ea40 con 0x7fa06c13ec50 2020-11-16T22:33:51.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c16fd10 con 0x7fa06c13ec50 2020-11-16T22:33:51.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c13b1e0 con 0x7fa06c13ec50 2020-11-16T22:33:51.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa05c01a040 con 0x7fa06c13ec50 2020-11-16T22:33:52.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c15df30 con 0x7fa06c13ec50 2020-11-16T22:33:52.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 0x7fa048057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa05c071690 con 0x7fa06c13ec50 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa0723db700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 0x7fa048057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.543+0000 7fa0723db700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 0x7fa048057550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa064005920 tx=0x7fa064009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.551+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa05c047c20 con 0x7fa06c13ec50 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa06c044f60 con 0x7fa06c13ec50 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa06a7fc700 1 -- 172.21.15.181:0/617274019 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa05c04b240 con 0x7fa06c13ec50 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 msgr2=0x7fa048057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 0x7fa048057550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa064005920 tx=0x7fa064009040).stop 2020-11-16T22:33:52.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 msgr2=0x7fa06c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fa05c00b920 tx=0x7fa05c015000).stop 2020-11-16T22:33:52.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 shutdown_connections 2020-11-16T22:33:52.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa048055090 0x7fa048057550 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c13ec50 0x7fa06c0b8190 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa06c061810 0x7fa06c13acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 --2- 172.21.15.181:0/617274019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa06c13e870 0x7fa06c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 >> 172.21.15.181:0/617274019 conn(0x7fa06c0cf630 msgr2=0x7fa06c05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 shutdown_connections 2020-11-16T22:33:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:51.703+0000 7fa0733dd700 1 -- 172.21.15.181:0/617274019 wait complete. 2020-11-16T22:33:52.355 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:33:52.356 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:33:52.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.499+0000 7fe1ee2c1700 1 Processor -- start 2020-11-16T22:33:52.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.499+0000 7fe1ee2c1700 1 -- start start 2020-11-16T22:33:52.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8062700 con 0x7fe1e8063620 2020-11-16T22:33:52.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80629a0 con 0x7fe1e8160530 2020-11-16T22:33:52.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8062c40 con 0x7fe1e8142660 2020-11-16T22:33:52.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:52.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42766/0 (socket says 172.21.15.181:42766) 2020-11-16T22:33:52.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1edac0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe1e8160530 0x7fe1e8061810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49232/0 (socket says 172.21.15.181:49232) 2020-11-16T22:33:52.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 -- 172.21.15.181:0/181881540 learned_addr learned my addr 172.21.15.181:0/181881540 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:52.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 -- 172.21.15.181:0/181881540 >> v1:172.21.15.181:6789/0 conn(0x7fe1e8160530 legacy=0x7fe1e8061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 -- 172.21.15.181:0/181881540 >> v1:172.21.15.175:6789/0 conn(0x7fe1e8063620 legacy=0x7fe1e8063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1dffff700 1 -- 172.21.15.181:0/181881540 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729835936 0 0) 0x7fe1e80629a0 con 0x7fe1e8160530 2020-11-16T22:33:52.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 -- 172.21.15.181:0/181881540 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8062ee0 con 0x7fe1e8142660 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ed2bf700 1 --2- 172.21.15.181:0/181881540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fe1d800ba20 tx=0x7fe1d8006eb0).ready entity=mon.2 client_cookie=9ec129743c0f0790 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1dffff700 1 -- 172.21.15.181:0/181881540 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80097d0 con 0x7fe1e8142660 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1dffff700 1 -- 172.21.15.181:0/181881540 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d8009930 con 0x7fe1e8142660 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1dffff700 1 -- 172.21.15.181:0/181881540 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8009cb0 con 0x7fe1e8142660 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/181881540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 msgr2=0x7fe1e8142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/181881540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fe1d800ba20 tx=0x7fe1d8006eb0).stop 2020-11-16T22:33:52.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/181881540 shutdown_connections 2020-11-16T22:33:52.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/181881540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8142660 0x7fe1e8142a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/181881540 >> 172.21.15.181:0/181881540 conn(0x7fe1e80d2dc0 msgr2=0x7fe1e80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:52.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/181881540 shutdown_connections 2020-11-16T22:33:52.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.503+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/181881540 wait complete. 2020-11-16T22:33:52.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 Processor -- start 2020-11-16T22:33:52.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 -- start start 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 0x7fe1e8145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 0x7fe1e80b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 0x7fe1e815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815f660 con 0x7fe1e8142660 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815f900 con 0x7fe1e8063620 2020-11-16T22:33:52.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ee2c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8172af0 con 0x7fe1e8160530 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ed2bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 0x7fe1e8145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1edac0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 0x7fe1e815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ed2bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 0x7fe1e8145a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34930/0 (socket says 172.21.15.181:34930) 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ed2bf700 1 -- 172.21.15.181:0/213453199 learned_addr learned my addr 172.21.15.181:0/213453199 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.507+0000 7fe1ecabe700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 0x7fe1e80b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 -- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 msgr2=0x7fe1e8145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 0x7fe1e8145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 -- 172.21.15.181:0/213453199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 msgr2=0x7fe1e80b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 0x7fe1e80b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 -- 172.21.15.181:0/213453199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8172d90 con 0x7fe1e8160530 2020-11-16T22:33:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1ecabe700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 0x7fe1e80b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1edac0700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 0x7fe1e815ec20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fe1e000c730 tx=0x7fe1e00078c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00072e0 con 0x7fe1e8160530 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005530 con 0x7fe1e8160530 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00058b0 con 0x7fe1e8160530 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8173030 con 0x7fe1e8160530 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e80b5770 con 0x7fe1e8160530 2020-11-16T22:33:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e80b5990 con 0x7fe1e8160530 2020-11-16T22:33:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe1e0010070 con 0x7fe1e8160530 2020-11-16T22:33:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1ed2bf700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.511+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe1e006cf20 con 0x7fe1e8160530 2020-11-16T22:33:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.515+0000 7fe1ed2bf700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fe1d800b1b0 tx=0x7fe1d8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.519+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe1e00434b0 con 0x7fe1e8160530 2020-11-16T22:33:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.675+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe1e8172930 con 0x7fe1e8160530 2020-11-16T22:33:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.675+0000 7fe1dd7fa700 1 -- 172.21.15.181:0/213453199 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe1e0046ad0 con 0x7fe1e8160530 2020-11-16T22:33:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 msgr2=0x7fe1c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fe1d800b1b0 tx=0x7fe1d8015000).stop 2020-11-16T22:33:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 msgr2=0x7fe1e815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 0x7fe1e815ec20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fe1e000c730 tx=0x7fe1e00078c0).stop 2020-11-16T22:33:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 shutdown_connections 2020-11-16T22:33:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe1c0055060 0x7fe1c0057520 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1e8160530 0x7fe1e815ec20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8063620 0x7fe1e8145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 --2- 172.21.15.181:0/213453199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e8142660 0x7fe1e80b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 >> 172.21.15.181:0/213453199 conn(0x7fe1e80d2dc0 msgr2=0x7fe1e8063ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 shutdown_connections 2020-11-16T22:33:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:52.679+0000 7fe1ee2c1700 1 -- 172.21.15.181:0/213453199 wait complete. 2020-11-16T22:33:53.315 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:53.315 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:33:53.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:53.315 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:33:53.316 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:33:53.317 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:33:53.317 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:33:53.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.463+0000 7f20ca009700 1 Processor -- start 2020-11-16T22:33:53.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.463+0000 7f20ca009700 1 -- start start 2020-11-16T22:33:53.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:53.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40c8130 con 0x7f20c415d2e0 2020-11-16T22:33:53.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40c83d0 con 0x7f20c40b2900 2020-11-16T22:33:53.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40c8910 con 0x7f20c40b2ce0 2020-11-16T22:33:53.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:53.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42784/0 (socket says 172.21.15.181:42784) 2020-11-16T22:33:53.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 -- 172.21.15.181:0/940763405 learned_addr learned my addr 172.21.15.181:0/940763405 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:53.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 -- 172.21.15.181:0/940763405 >> v1:172.21.15.181:6789/0 conn(0x7f20c40b2900 legacy=0x7f20c40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20bbfff700 1 -- 172.21.15.181:0/940763405 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640612421 0 0) 0x7f20c40c83d0 con 0x7f20c40b2900 2020-11-16T22:33:53.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 -- 172.21.15.181:0/940763405 >> v1:172.21.15.175:6789/0 conn(0x7f20c415d2e0 legacy=0x7f20c40c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 -- 172.21.15.181:0/940763405 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40c8670 con 0x7f20c40b2ce0 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20c8806700 1 --2- 172.21.15.181:0/940763405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f20b400ba20 tx=0x7f20b4006eb0).ready entity=mon.2 client_cookie=2b298d54bd55c77f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20bbfff700 1 -- 172.21.15.181:0/940763405 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40098d0 con 0x7f20c40b2ce0 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20bbfff700 1 -- 172.21.15.181:0/940763405 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4009a30 con 0x7f20c40b2ce0 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20bbfff700 1 -- 172.21.15.181:0/940763405 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4009db0 con 0x7f20c40b2ce0 2020-11-16T22:33:53.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- 172.21.15.181:0/940763405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 msgr2=0x7f20c415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 --2- 172.21.15.181:0/940763405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f20b400ba20 tx=0x7f20b4006eb0).stop 2020-11-16T22:33:53.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- 172.21.15.181:0/940763405 shutdown_connections 2020-11-16T22:33:53.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 --2- 172.21.15.181:0/940763405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c40b2ce0 0x7f20c415cdf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- 172.21.15.181:0/940763405 >> 172.21.15.181:0/940763405 conn(0x7f20c40d2dc0 msgr2=0x7f20c40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:53.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- 172.21.15.181:0/940763405 shutdown_connections 2020-11-16T22:33:53.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.467+0000 7f20ca009700 1 -- 172.21.15.181:0/940763405 wait complete. 2020-11-16T22:33:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 Processor -- start 2020-11-16T22:33:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- start start 2020-11-16T22:33:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40b2900 0x7f20c41610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 0x7f20c40cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c416dfd0 con 0x7f20c40b2900 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc000 con 0x7f20c40b2ce0 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41712f0 con 0x7f20c415d2e0 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c8806700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 0x7f20c40cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c8806700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34946/0 (socket says 172.21.15.181:34946) 2020-11-16T22:33:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c8806700 1 -- 172.21.15.181:0/3172919049 learned_addr learned my addr 172.21.15.181:0/3172919049 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9007700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40b2900 0x7f20c41610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 -- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 msgr2=0x7f20c40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 -- 172.21.15.181:0/3172919049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40b2900 msgr2=0x7f20c41610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40b2900 0x7f20c41610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 -- 172.21.15.181:0/3172919049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c416d0b0 con 0x7f20c415d2e0 2020-11-16T22:33:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c8806700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20c9808700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 0x7f20c40cbb10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f20bc00ca90 tx=0x7f20bc007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00ce60 con 0x7f20c415d2e0 2020-11-16T22:33:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00a6d0 con 0x7f20c415d2e0 2020-11-16T22:33:53.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c416d350 con 0x7f20c415d2e0 2020-11-16T22:33:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00aa50 con 0x7f20c415d2e0 2020-11-16T22:33:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.471+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c416d890 con 0x7f20c415d2e0 2020-11-16T22:33:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4044f60 con 0x7f20c415d2e0 2020-11-16T22:33:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f20bc019030 con 0x7f20c415d2e0 2020-11-16T22:33:53.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20b97fa700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 0x7f209c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f20bc06d1a0 con 0x7f20c415d2e0 2020-11-16T22:33:54.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20c9007700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 0x7f209c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.475+0000 7f20c9007700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 0x7f209c057520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f20c00011a0 tx=0x7f20c0007a00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:54.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.479+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f20bc0436d0 con 0x7f20c415d2e0 2020-11-16T22:33:54.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.635+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f20c405d550 con 0x7f20c415d2e0 2020-11-16T22:33:54.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.799+0000 7f20b97fa700 1 -- 172.21.15.181:0/3172919049 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v94) v1 ==== 133+0+0 (secure 0 0 0) 0x7f20bc046cf0 con 0x7f20c415d2e0 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 msgr2=0x7f209c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 0x7f209c057520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f20c00011a0 tx=0x7f20c0007a00).stop 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 msgr2=0x7f20c40cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 0x7f20c40cbb10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f20bc00ca90 tx=0x7f20bc007e20).stop 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 shutdown_connections 2020-11-16T22:33:54.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f209c055060 0x7f209c057520 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c415d2e0 0x7f20c40cbb10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40b2900 0x7f20c41610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 --2- 172.21.15.181:0/3172919049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c40b2ce0 0x7f20c40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 >> 172.21.15.181:0/3172919049 conn(0x7f20c40d2dc0 msgr2=0x7f20c41595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 shutdown_connections 2020-11-16T22:33:54.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:53.803+0000 7f20ca009700 1 -- 172.21.15.181:0/3172919049 wait complete. 2020-11-16T22:33:54.283 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:33:54.283 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:33:54.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 Processor -- start 2020-11-16T22:33:54.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 -- start start 2020-11-16T22:33:54.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44540615f0 0x7f44540d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f44540622a0 0x7f4454062720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454160e40 con 0x7f44540622a0 2020-11-16T22:33:54.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454160ba0 con 0x7f44540615f0 2020-11-16T22:33:54.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f445b50e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4454060970 con 0x7f445415f5b0 2020-11-16T22:33:54.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f4458aa9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44540615f0 0x7f44540d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42794/0 (socket says 172.21.15.181:42794) 2020-11-16T22:33:54.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 -- 172.21.15.181:0/3984006129 learned_addr learned my addr 172.21.15.181:0/3984006129 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:54.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f4459aab700 1 --2- 172.21.15.181:0/3984006129 >> v2:172.21.15.175:3300/0 conn(0x7f44540622a0 0x7f4454062720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 -- 172.21.15.181:0/3984006129 >> v2:172.21.15.181:3300/0 conn(0x7f44540615f0 msgr2=0x7f44540d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:54.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 --2- 172.21.15.181:0/3984006129 >> v2:172.21.15.181:3300/0 conn(0x7f44540615f0 0x7f44540d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 -- 172.21.15.181:0/3984006129 >> v2:172.21.15.175:3300/0 conn(0x7f44540622a0 msgr2=0x7f4454062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 --2- 172.21.15.181:0/3984006129 >> v2:172.21.15.175:3300/0 conn(0x7f44540622a0 0x7f4454062720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.435+0000 7f44592aa700 1 -- 172.21.15.181:0/3984006129 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454060c10 con 0x7f445415f5b0 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f44592aa700 1 --2- 172.21.15.181:0/3984006129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f445405d470 tx=0x7f4450006e50).ready entity=mon.2 client_cookie=35a62e988be23be9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f444bfff700 1 -- 172.21.15.181:0/3984006129 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450006030 con 0x7f445415f5b0 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f444bfff700 1 -- 172.21.15.181:0/3984006129 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450006190 con 0x7f445415f5b0 2020-11-16T22:33:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f444bfff700 1 -- 172.21.15.181:0/3984006129 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450006510 con 0x7f445415f5b0 2020-11-16T22:33:54.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- 172.21.15.181:0/3984006129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 msgr2=0x7f44540d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:54.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- 172.21.15.181:0/3984006129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f445405d470 tx=0x7f4450006e50).stop 2020-11-16T22:33:54.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- 172.21.15.181:0/3984006129 shutdown_connections 2020-11-16T22:33:54.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- 172.21.15.181:0/3984006129 >> v2:172.21.15.175:3300/0 conn(0x7f44540622a0 0x7f4454062720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- 172.21.15.181:0/3984006129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445415f5b0 0x7f44540d1340 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- 172.21.15.181:0/3984006129 >> v2:172.21.15.181:3300/0 conn(0x7f44540615f0 0x7f44540d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:54.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- 172.21.15.181:0/3984006129 >> 172.21.15.181:0/3984006129 conn(0x7f44540d69a0 msgr2=0x7f44540d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:54.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- 172.21.15.181:0/3984006129 shutdown_connections 2020-11-16T22:33:54.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- 172.21.15.181:0/3984006129 wait complete. 2020-11-16T22:33:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 Processor -- start 2020-11-16T22:33:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- start start 2020-11-16T22:33:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44540622a0 0x7f44540b9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445415f5b0 0x7f44540bb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540baad0 con 0x7f445415f5b0 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ba290 con 0x7f44540622a0 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f445b50e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ba530 con 0x7f44540615f0 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f44592aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f4458aa9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44540622a0 0x7f44540b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f44592aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42800/0 (socket says 172.21.15.181:42800) 2020-11-16T22:33:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f44592aa700 1 -- 172.21.15.181:0/968750529 learned_addr learned my addr 172.21.15.181:0/968750529 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:55.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.439+0000 7f4459aab700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445415f5b0 0x7f44540bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 -- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44540622a0 msgr2=0x7f44540b9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44540622a0 0x7f44540b9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 -- 172.21.15.181:0/968750529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445415f5b0 msgr2=0x7f44540bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445415f5b0 0x7f44540bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 -- 172.21.15.181:0/968750529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540ba7d0 con 0x7f44540615f0 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44592aa700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f4450006e90 tx=0x7f4450015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500058e0 con 0x7f44540615f0 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450005a40 con 0x7f44540615f0 2020-11-16T22:33:55.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540bad00 con 0x7f44540615f0 2020-11-16T22:33:55.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450005de0 con 0x7f44540615f0 2020-11-16T22:33:55.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540b9e20 con 0x7f44540615f0 2020-11-16T22:33:55.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4454069eb0 con 0x7f44540615f0 2020-11-16T22:33:55.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.443+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4450025030 con 0x7f44540615f0 2020-11-16T22:33:55.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.447+0000 7f44497fa700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 0x7f442c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.447+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f445006f210 con 0x7f44540615f0 2020-11-16T22:33:55.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.447+0000 7f4458aa9700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 0x7f442c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.447+0000 7f4458aa9700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 0x7f442c057520 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f445415d590 tx=0x7f4444006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.451+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4450045570 con 0x7f44540615f0 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.603+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f44540ba070 con 0x7f44540615f0 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.803+0000 7f44497fa700 1 -- 172.21.15.181:0/968750529 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v95) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4450048b50 con 0x7f44540615f0 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 msgr2=0x7f442c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 0x7f442c057520 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f445415d590 tx=0x7f4444006df0).stop 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 msgr2=0x7f4454170b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f4450006e90 tx=0x7f4450015000).stop 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 shutdown_connections 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f442c055060 0x7f442c057520 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f445415f5b0 0x7f44540bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44540615f0 0x7f4454170b90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 --2- 172.21.15.181:0/968750529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44540622a0 0x7f44540b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 >> 172.21.15.181:0/968750529 conn(0x7f44540d69a0 msgr2=0x7f44540b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:55.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 shutdown_connections 2020-11-16T22:33:55.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:54.811+0000 7f445b50e700 1 -- 172.21.15.181:0/968750529 wait complete. 2020-11-16T22:33:55.253 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:33:55.254 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:33:55.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.403+0000 7fb4f4d24700 1 Processor -- start 2020-11-16T22:33:55.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.403+0000 7fb4f4d24700 1 -- start start 2020-11-16T22:33:55.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb4f0145a60 0x7fb4f0145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4f00615b0 0x7fb4f0080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f00cb950 con 0x7fb4f0061a30 2020-11-16T22:33:55.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00cbbf0 con 0x7fb4f0145a60 2020-11-16T22:33:55.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00cbe90 con 0x7fb4f00615b0 2020-11-16T22:33:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4f00615b0 0x7fb4f0080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4f00615b0 0x7fb4f0080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34966/0 (socket says 172.21.15.181:34966) 2020-11-16T22:33:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 -- 172.21.15.181:0/1709299595 learned_addr learned my addr 172.21.15.181:0/1709299595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4ee59c700 1 --2- 172.21.15.181:0/1709299595 >> v2:172.21.15.175:3300/0 conn(0x7fb4f0145a60 0x7fb4f0145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 -- 172.21.15.181:0/1709299595 >> v2:172.21.15.175:3300/0 conn(0x7fb4f0145a60 msgr2=0x7fb4f0145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:55.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 --2- 172.21.15.181:0/1709299595 >> v2:172.21.15.175:3300/0 conn(0x7fb4f0145a60 0x7fb4f0145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 -- 172.21.15.181:0/1709299595 >> v1:172.21.15.175:6789/0 conn(0x7fb4f0061a30 legacy=0x7fb4f00814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 -- 172.21.15.181:0/1709299595 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00cc130 con 0x7fb4f00615b0 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4ed59a700 1 -- 172.21.15.181:0/1709299595 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929963245 0 0) 0x7fb4f00cb950 con 0x7fb4f0061a30 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4edd9b700 1 --2- 172.21.15.181:0/1709299595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f0080fd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fb4d8006fd0 tx=0x7fb4d8006eb0).ready entity=mon.1 client_cookie=be3821f1b59b7643 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4ed59a700 1 -- 172.21.15.181:0/1709299595 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80097d0 con 0x7fb4f00615b0 2020-11-16T22:33:55.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4ed59a700 1 -- 172.21.15.181:0/1709299595 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d8009930 con 0x7fb4f00615b0 2020-11-16T22:33:55.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4ed59a700 1 -- 172.21.15.181:0/1709299595 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8009cb0 con 0x7fb4f00615b0 2020-11-16T22:33:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1709299595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 msgr2=0x7fb4f0080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1709299595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f0080fd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fb4d8006fd0 tx=0x7fb4d8006eb0).stop 2020-11-16T22:33:55.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1709299595 shutdown_connections 2020-11-16T22:33:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1709299595 >> v2:172.21.15.175:3300/0 conn(0x7fb4f0145a60 0x7fb4f0145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1709299595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f0080fd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1709299595 >> 172.21.15.181:0/1709299595 conn(0x7fb4f00d68e0 msgr2=0x7fb4f00d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1709299595 shutdown_connections 2020-11-16T22:33:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.407+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1709299595 wait complete. 2020-11-16T22:33:55.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.411+0000 7fb4f4d24700 1 Processor -- start 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.411+0000 7fb4f4d24700 1 -- start start 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0061a30 0x7fb4f00d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 0x7fb4f00d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ba1d0 con 0x7fb4f0061a30 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ba470 con 0x7fb4f00615b0 2020-11-16T22:33:55.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ba710 con 0x7fb4f0145a60 2020-11-16T22:33:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34972/0 (socket says 172.21.15.181:34972) 2020-11-16T22:33:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 -- 172.21.15.181:0/1351498007 learned_addr learned my addr 172.21.15.181:0/1351498007 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4eed9d700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 0x7fb4f00d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:55.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4edd9b700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0061a30 0x7fb4f00d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 -- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 msgr2=0x7fb4f00d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 0x7fb4f00d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 -- 172.21.15.181:0/1351498007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0061a30 msgr2=0x7fb4f00d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0061a30 0x7fb4f00d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 -- 172.21.15.181:0/1351498007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00ba9b0 con 0x7fb4f00615b0 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4eed9d700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 0x7fb4f00d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4ee59c700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb4e000eba0 tx=0x7fb4e0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000c8e0 con 0x7fb4f00615b0 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000ca40 con 0x7fb4f00615b0 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000cdc0 con 0x7fb4f00615b0 2020-11-16T22:33:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f00bac50 con 0x7fb4f00615b0 2020-11-16T22:33:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f00b9f00 con 0x7fb4f00615b0 2020-11-16T22:33:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.415+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f0069eb0 con 0x7fb4f00615b0 2020-11-16T22:33:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.419+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb4e0012600 con 0x7fb4f00615b0 2020-11-16T22:33:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.419+0000 7fb4e67fc700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 0x7fb4cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.419+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fb4e006e0a0 con 0x7fb4f00615b0 2020-11-16T22:33:56.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.419+0000 7fb4edd9b700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 0x7fb4cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.419+0000 7fb4edd9b700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 0x7fb4cc057520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb4f00bcb30 tx=0x7fb4d8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:56.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.423+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb4e0044310 con 0x7fb4f00615b0 2020-11-16T22:33:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.619+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fb4f005d590 con 0x7fb4f00615b0 2020-11-16T22:33:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.635+0000 7fb4e67fc700 1 -- 172.21.15.181:0/1351498007 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v93) v1 ==== 189+0+0 (secure 0 0 0) 0x7fb4e0047930 con 0x7fb4f00615b0 2020-11-16T22:33:56.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 msgr2=0x7fb4cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 0x7fb4cc057520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb4f00bcb30 tx=0x7fb4d8015000).stop 2020-11-16T22:33:56.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 msgr2=0x7fb4f00b9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb4e000eba0 tx=0x7fb4e0006df0).stop 2020-11-16T22:33:56.221 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 shutdown_connections 2020-11-16T22:33:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb4cc055060 0x7fb4cc057520 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4f0145a60 0x7fb4f00d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4f00615b0 0x7fb4f00b9870 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 --2- 172.21.15.181:0/1351498007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4f0061a30 0x7fb4f00d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 >> 172.21.15.181:0/1351498007 conn(0x7fb4f00d68e0 msgr2=0x7fb4f0163ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 shutdown_connections 2020-11-16T22:33:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:55.643+0000 7fb4f4d24700 1 -- 172.21.15.181:0/1351498007 wait complete. 2020-11-16T22:33:56.423 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 13 and data pool 14 2020-11-16T22:33:56.424 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:33:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 Processor -- start 2020-11-16T22:33:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 -- start start 2020-11-16T22:33:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd3f0161480 0x7fd3f0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0164e10 0x7fd3f0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00c8db0 con 0x7fd3f0161860 2020-11-16T22:33:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00c8330 con 0x7fd3f0161480 2020-11-16T22:33:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00c85d0 con 0x7fd3f0164e10 2020-11-16T22:33:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd3f0161480 0x7fd3f0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0164e10 0x7fd3f0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f4df7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd3f0161860 0x7fd3f0062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49296/0 (socket says 172.21.15.181:49296) 2020-11-16T22:33:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f4df7700 1 -- 172.21.15.181:0/2568026527 learned_addr learned my addr 172.21.15.181:0/2568026527 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:33:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd3f0161480 0x7fd3f0164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34988/0 (socket says 172.21.15.181:34988) 2020-11-16T22:33:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 -- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0164e10 msgr2=0x7fd3f0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 --2- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0164e10 0x7fd3f0062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 -- 172.21.15.181:0/2568026527 >> v1:172.21.15.181:6789/0 conn(0x7fd3f0161860 legacy=0x7fd3f0062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3eeffd700 1 -- 172.21.15.181:0/2568026527 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269891442 0 0) 0x7fd3f00c8db0 con 0x7fd3f0161860 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 -- 172.21.15.181:0/2568026527 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00c8870 con 0x7fd3f0161480 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3effff700 1 --2- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161480 0x7fd3f0164920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fd3f00c8260 tx=0x7fd3e00091d0).ready entity=mon.1 client_cookie=1b10f6898113118 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3eeffd700 1 -- 172.21.15.181:0/2568026527 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000ba70 con 0x7fd3f0161480 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3eeffd700 1 -- 172.21.15.181:0/2568026527 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000bbd0 con 0x7fd3f0161480 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3eeffd700 1 -- 172.21.15.181:0/2568026527 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0011400 con 0x7fd3f0161480 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.567+0000 7fd3f55f8700 1 -- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161480 msgr2=0x7fd3f0164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161480 0x7fd3f0164920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fd3f00c8260 tx=0x7fd3e00091d0).stop 2020-11-16T22:33:56.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- 172.21.15.181:0/2568026527 shutdown_connections 2020-11-16T22:33:56.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161480 0x7fd3f0164920 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/2568026527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0164e10 0x7fd3f0062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- 172.21.15.181:0/2568026527 >> 172.21.15.181:0/2568026527 conn(0x7fd3f00d2dc0 msgr2=0x7fd3f00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:56.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- 172.21.15.181:0/2568026527 shutdown_connections 2020-11-16T22:33:56.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- 172.21.15.181:0/2568026527 wait complete. 2020-11-16T22:33:56.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 Processor -- start 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- start start 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 0x7fd3f01464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0162af0 con 0x7fd3f0161480 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01770a0 con 0x7fd3f0161860 2020-11-16T22:33:56.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f55f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0177340 con 0x7fd3f0062320 2020-11-16T22:33:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f4df7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f4df7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42840/0 (socket says 172.21.15.181:42840) 2020-11-16T22:33:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34996/0 (socket says 172.21.15.181:34996) 2020-11-16T22:33:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f4df7700 1 -- 172.21.15.181:0/584036806 learned_addr learned my addr 172.21.15.181:0/584036806 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3effff700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 0x7fd3f01464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 -- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 msgr2=0x7fd3f00627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 -- 172.21.15.181:0/584036806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 msgr2=0x7fd3f01464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 0x7fd3f01464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 -- 172.21.15.181:0/584036806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f01775e0 con 0x7fd3f0161860 2020-11-16T22:33:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3effff700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 0x7fd3f01464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3f4df7700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.571+0000 7fd3ef7fe700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fd3d800eba0 tx=0x7fd3d8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800b2c0 con 0x7fd3f0161860 2020-11-16T22:33:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f0177880 con 0x7fd3f0161860 2020-11-16T22:33:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d800c8e0 con 0x7fd3f0161860 2020-11-16T22:33:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f0176d60 con 0x7fd3f0161860 2020-11-16T22:33:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800cc60 con 0x7fd3f0161860 2020-11-16T22:33:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f00687a0 con 0x7fd3f0161860 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd3d801c460 con 0x7fd3f0161860 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 0x7fd3c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3d806df10 con 0x7fd3f0161860 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3effff700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 0x7fd3c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.575+0000 7fd3effff700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 0x7fd3c0057520 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fd3e0009fd0 tx=0x7fd3e0016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.583+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd3f00687a0 con 0x7fd3f0161860 2020-11-16T22:33:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f0065bb0 con 0x7fd3f0161860 2020-11-16T22:33:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3e7fff700 1 -- 172.21.15.181:0/584036806 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fd3f0065bb0 con 0x7fd3f0161860 2020-11-16T22:33:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 msgr2=0x7fd3c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:57.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 0x7fd3c0057520 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fd3e0009fd0 tx=0x7fd3e0016000).stop 2020-11-16T22:33:57.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 msgr2=0x7fd3f01469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:57.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.739+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fd3d800eba0 tx=0x7fd3d8006df0).stop 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 shutdown_connections 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd3c0055060 0x7fd3c0057520 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f0062320 0x7fd3f00627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f0161480 0x7fd3f01464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 --2- 172.21.15.181:0/584036806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0161860 0x7fd3f01469c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 >> 172.21.15.181:0/584036806 conn(0x7fd3f00d2dc0 msgr2=0x7fd3f00652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:57.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 shutdown_connections 2020-11-16T22:33:57.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:56.743+0000 7fd3f55f8700 1 -- 172.21.15.181:0/584036806 wait complete. 2020-11-16T22:33:57.384 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:57.384 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:33:58.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.339+0000 7efdee6ad700 1 Processor -- start 2020-11-16T22:33:58.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- start start 2020-11-16T22:33:58.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 0x7efde80622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde8160e40 con 0x7efde815f5b0 2020-11-16T22:33:58.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde81609a0 con 0x7efde80615f0 2020-11-16T22:33:58.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde8160ba0 con 0x7efde8062790 2020-11-16T22:33:58.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42848/0 (socket says 172.21.15.181:42848) 2020-11-16T22:33:58.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 -- 172.21.15.181:0/624932593 learned_addr learned my addr 172.21.15.181:0/624932593 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:58.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efde77fe700 1 --2- 172.21.15.181:0/624932593 >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 0x7efde80622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 -- 172.21.15.181:0/624932593 >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 msgr2=0x7efde80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:58.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 --2- 172.21.15.181:0/624932593 >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 0x7efde80622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:58.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 -- 172.21.15.181:0/624932593 >> v1:172.21.15.175:6789/0 conn(0x7efde815f5b0 legacy=0x7efde80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:58.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 -- 172.21.15.181:0/624932593 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8060970 con 0x7efde8062790 2020-11-16T22:33:58.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efde77fe700 1 --2- 172.21.15.181:0/624932593 >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 0x7efde80622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:33:58.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdecc4a700 1 --2- 172.21.15.181:0/624932593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7efde0009d50 tx=0x7efde0009290).ready entity=mon.2 client_cookie=341805c859e7fe31 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:58.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efde6ffd700 1 -- 172.21.15.181:0/624932593 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000bc80 con 0x7efde8062790 2020-11-16T22:33:58.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efde6ffd700 1 -- 172.21.15.181:0/624932593 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde000bde0 con 0x7efde8062790 2020-11-16T22:33:58.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efde6ffd700 1 -- 172.21.15.181:0/624932593 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0011610 con 0x7efde8062790 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- 172.21.15.181:0/624932593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 msgr2=0x7efde8080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 --2- 172.21.15.181:0/624932593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7efde0009d50 tx=0x7efde0009290).stop 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- 172.21.15.181:0/624932593 shutdown_connections 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 --2- 172.21.15.181:0/624932593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde8080fd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 --2- 172.21.15.181:0/624932593 >> v2:172.21.15.175:3300/0 conn(0x7efde80615f0 0x7efde80622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- 172.21.15.181:0/624932593 >> 172.21.15.181:0/624932593 conn(0x7efde80d69a0 msgr2=0x7efde80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:58.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- 172.21.15.181:0/624932593 shutdown_connections 2020-11-16T22:33:58.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.343+0000 7efdee6ad700 1 -- 172.21.15.181:0/624932593 wait complete. 2020-11-16T22:33:58.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 Processor -- start 2020-11-16T22:33:58.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 -- start start 2020-11-16T22:33:58.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde80615f0 0x7efde816f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde80d5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 0x7efde80d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80bc9b0 con 0x7efde80615f0 2020-11-16T22:33:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80cb900 con 0x7efde815f5b0 2020-11-16T22:33:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdee6ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde80baab0 con 0x7efde8062790 2020-11-16T22:33:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efde77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde80d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efde77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde80d5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42852/0 (socket says 172.21.15.181:42852) 2020-11-16T22:33:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efde77fe700 1 -- 172.21.15.181:0/3662464771 learned_addr learned my addr 172.21.15.181:0/3662464771 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 0x7efde80d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efde7fff700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde80615f0 0x7efde816f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 -- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 msgr2=0x7efde80d5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde80d5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 -- 172.21.15.181:0/3662464771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde80615f0 msgr2=0x7efde816f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde80615f0 0x7efde816f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 -- 172.21.15.181:0/3662464771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8170000 con 0x7efde815f5b0 2020-11-16T22:33:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdecc4a700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 0x7efde80d5d30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7efde0009d50 tx=0x7efde0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:58.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.347+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0009e90 con 0x7efde815f5b0 2020-11-16T22:33:58.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde0003290 con 0x7efde815f5b0 2020-11-16T22:33:58.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde81702a0 con 0x7efde815f5b0 2020-11-16T22:33:58.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0010cf0 con 0x7efde815f5b0 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde80cbb30 con 0x7efde815f5b0 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde8069eb0 con 0x7efde815f5b0 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efde0017070 con 0x7efde815f5b0 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdc7fff700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 0x7efdc0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efde006fdd0 con 0x7efde815f5b0 2020-11-16T22:33:58.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efde7fff700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 0x7efdc0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:58.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.351+0000 7efde7fff700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 0x7efdc0057520 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7efdd8003350 tx=0x7efdd8007af0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:58.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.359+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efde00460b0 con 0x7efde815f5b0 2020-11-16T22:33:58.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.511+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efde81642f0 con 0x7efde815f5b0 2020-11-16T22:33:58.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdc7fff700 1 -- 172.21.15.181:0/3662464771 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7efde0015070 con 0x7efde815f5b0 2020-11-16T22:33:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 msgr2=0x7efdc0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 0x7efdc0057520 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7efdd8003350 tx=0x7efdd8007af0).stop 2020-11-16T22:33:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 msgr2=0x7efde80d5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 0x7efde80d5d30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7efde0009d50 tx=0x7efde0016000).stop 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 shutdown_connections 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efdc0055060 0x7efdc0057520 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde815f5b0 0x7efde80d5d30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde80615f0 0x7efde816f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 --2- 172.21.15.181:0/3662464771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efde8062790 0x7efde80d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 >> 172.21.15.181:0/3662464771 conn(0x7efde80d69a0 msgr2=0x7efde80a75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 shutdown_connections 2020-11-16T22:33:59.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:58.515+0000 7efdee6ad700 1 -- 172.21.15.181:0/3662464771 wait complete. 2020-11-16T22:33:59.156 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:33:59.156 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:33:59.359 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.503+0000 7f301c976700 1 Processor -- start 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.503+0000 7f301c976700 1 -- start start 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 0x7f301815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f301815f0c0 0x7f301815f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f30180600f0 0x7f30180605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301815fa30 con 0x7f30180600f0 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301815e940 con 0x7f301815f0c0 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f301c976700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30180cdf10 con 0x7f301815aed0 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 0x7f301815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 0x7f301815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42858/0 (socket says 172.21.15.181:42858) 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f301815f0c0 0x7f301815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30177fe700 1 -- 172.21.15.181:0/257187521 learned_addr learned my addr 172.21.15.181:0/257187521 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3017fff700 1 --2- 172.21.15.181:0/257187521 >> v2:172.21.15.175:3300/0 conn(0x7f30180600f0 0x7f30180605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 -- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 msgr2=0x7f301815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 --2- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 0x7f301815b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 -- 172.21.15.181:0/257187521 >> v2:172.21.15.175:3300/0 conn(0x7f30180600f0 msgr2=0x7f30180605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 --2- 172.21.15.181:0/257187521 >> v2:172.21.15.175:3300/0 conn(0x7f30180600f0 0x7f30180605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 -- 172.21.15.181:0/257187521 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180ce1b0 con 0x7f301815f0c0 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f3016ffd700 1 --2- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815f0c0 0x7f301815f540 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f3018067c80 tx=0x7f300c006df0).ready entity=mon.1 client_cookie=ac304ed97e52d52f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30167fc700 1 -- 172.21.15.181:0/257187521 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c009840 con 0x7f301815f0c0 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30167fc700 1 -- 172.21.15.181:0/257187521 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c0099a0 con 0x7f301815f0c0 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.507+0000 7f30167fc700 1 -- 172.21.15.181:0/257187521 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c009d20 con 0x7f301815f0c0 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815f0c0 msgr2=0x7f301815f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 --2- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815f0c0 0x7f301815f540 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f3018067c80 tx=0x7f300c006df0).stop 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- 172.21.15.181:0/257187521 shutdown_connections 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 --2- 172.21.15.181:0/257187521 >> v2:172.21.15.175:3300/0 conn(0x7f30180600f0 0x7f30180605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 --2- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815aed0 0x7f301815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 --2- 172.21.15.181:0/257187521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815f0c0 0x7f301815f540 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- 172.21.15.181:0/257187521 >> 172.21.15.181:0/257187521 conn(0x7f30180d33f0 msgr2=0x7f30180d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- 172.21.15.181:0/257187521 shutdown_connections 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- 172.21.15.181:0/257187521 wait complete. 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 Processor -- start 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.511+0000 7f301c976700 1 -- start start 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30180600f0 0x7f30180b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018157e90 con 0x7f30180600f0 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018158900 con 0x7f301815aed0 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3018158ba0 con 0x7f301815f0c0 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3017fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3017fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42868/0 (socket says 172.21.15.181:42868) 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35024/0 (socket says 172.21.15.181:35024) 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3017fff700 1 -- 172.21.15.181:0/2391638916 learned_addr learned my addr 172.21.15.181:0/2391638916 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f30177fe700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30180600f0 0x7f30180b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 -- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 msgr2=0x7f3018162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 -- 172.21.15.181:0/2391638916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30180600f0 msgr2=0x7f30180b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30180600f0 0x7f30180b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 -- 172.21.15.181:0/2391638916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3018158e40 con 0x7f301815aed0 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3017fff700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f3016ffd700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f300c012000 tx=0x7f300c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c011d50 con 0x7f301815aed0 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c01e460 con 0x7f301815aed0 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c01e7e0 con 0x7f301815aed0 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30181590e0 con 0x7f301815aed0 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.515+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3018162770 con 0x7f301815aed0 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30181592c0 con 0x7f301815aed0 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f300c01a040 con 0x7f301815aed0 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f2ff77fe700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 0x7f2ff0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f300c06f030 con 0x7f301815aed0 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f30177fe700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 0x7f2ff0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.519+0000 7f30177fe700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 0x7f2ff0057520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f3008006a10 tx=0x7f3008008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.527+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f30181592c0 con 0x7f301815aed0 2020-11-16T22:33:59.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:33:59.735+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f30181592c0 con 0x7f301815aed0 2020-11-16T22:34:00.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.667+0000 7f2ff77fe700 1 -- 172.21.15.181:0/2391638916 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30181592c0 con 0x7f301815aed0 2020-11-16T22:34:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 msgr2=0x7f2ff0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 0x7f2ff0057520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f3008006a10 tx=0x7f3008008040).stop 2020-11-16T22:34:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 msgr2=0x7f30180b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f300c012000 tx=0x7f300c015000).stop 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 shutdown_connections 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2ff0055060 0x7f2ff0057520 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f301815f0c0 0x7f3018162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30180600f0 0x7f30180b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 --2- 172.21.15.181:0/2391638916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f301815aed0 0x7f30180b6a10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 >> 172.21.15.181:0/2391638916 conn(0x7f30180d33f0 msgr2=0x7f3018159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 shutdown_connections 2020-11-16T22:34:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.671+0000 7f301c976700 1 -- 172.21.15.181:0/2391638916 wait complete. 2020-11-16T22:34:00.675 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:00.768 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:34:00.779+0000 7f5755002700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:34:00.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.803+0000 7faa53572700 1 Processor -- start 2020-11-16T22:34:00.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.803+0000 7faa53572700 1 -- start start 2020-11-16T22:34:00.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa4c15f5b0 0x7faa4c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:00.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 0x7faa4c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:00.796 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4c160e40 con 0x7faa4c0622a0 2020-11-16T22:34:00.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c1609a0 con 0x7faa4c0615f0 2020-11-16T22:34:00.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c160ba0 con 0x7faa4c15f5b0 2020-11-16T22:34:00.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa4c15f5b0 0x7faa4c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:00.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa4c15f5b0 0x7faa4c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35030/0 (socket says 172.21.15.181:35030) 2020-11-16T22:34:00.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa51b0f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faa4c0622a0 0x7faa4c062730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49338/0 (socket says 172.21.15.181:49338) 2020-11-16T22:34:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 -- 172.21.15.181:0/2766909597 learned_addr learned my addr 172.21.15.181:0/2766909597 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:01.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa50b0d700 1 --2- 172.21.15.181:0/2766909597 >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 0x7faa4c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 -- 172.21.15.181:0/2766909597 >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 msgr2=0x7faa4c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 --2- 172.21.15.181:0/2766909597 >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 0x7faa4c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 -- 172.21.15.181:0/2766909597 >> v1:172.21.15.181:6789/0 conn(0x7faa4c0622a0 legacy=0x7faa4c062730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 -- 172.21.15.181:0/2766909597 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c060970 con 0x7faa4c15f5b0 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa3ffff700 1 -- 172.21.15.181:0/2766909597 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821379402 0 0) 0x7faa4c160e40 con 0x7faa4c0622a0 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa50b0d700 1 --2- 172.21.15.181:0/2766909597 >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 0x7faa4c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa5130e700 1 --2- 172.21.15.181:0/2766909597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c15f5b0 0x7faa4c0d1340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7faa4c083ad0 tx=0x7faa40006e50).ready entity=mon.1 client_cookie=c68bfbca8652c42f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa3ffff700 1 -- 172.21.15.181:0/2766909597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40006030 con 0x7faa4c15f5b0 2020-11-16T22:34:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa3ffff700 1 -- 172.21.15.181:0/2766909597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa40006190 con 0x7faa4c15f5b0 2020-11-16T22:34:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa3ffff700 1 -- 172.21.15.181:0/2766909597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40006510 con 0x7faa4c15f5b0 2020-11-16T22:34:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- 172.21.15.181:0/2766909597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c15f5b0 msgr2=0x7faa4c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 --2- 172.21.15.181:0/2766909597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c15f5b0 0x7faa4c0d1340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7faa4c083ad0 tx=0x7faa40006e50).stop 2020-11-16T22:34:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- 172.21.15.181:0/2766909597 shutdown_connections 2020-11-16T22:34:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 --2- 172.21.15.181:0/2766909597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c15f5b0 0x7faa4c0d1340 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 --2- 172.21.15.181:0/2766909597 >> v2:172.21.15.175:3300/0 conn(0x7faa4c0615f0 0x7faa4c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- 172.21.15.181:0/2766909597 >> 172.21.15.181:0/2766909597 conn(0x7faa4c0d69a0 msgr2=0x7faa4c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:01.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- 172.21.15.181:0/2766909597 shutdown_connections 2020-11-16T22:34:01.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.807+0000 7faa53572700 1 -- 172.21.15.181:0/2766909597 wait complete. 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 Processor -- start 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- start start 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 0x7faa4c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 0x7faa4c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c164310 con 0x7faa4c15f5b0 2020-11-16T22:34:01.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c161740 con 0x7faa4c0622a0 2020-11-16T22:34:01.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c1619e0 con 0x7faa4c0615f0 2020-11-16T22:34:01.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:01.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa50b0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 0x7faa4c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:01.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa50b0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 0x7faa4c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35040/0 (socket says 172.21.15.181:35040) 2020-11-16T22:34:01.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa50b0d700 1 -- 172.21.15.181:0/3083849628 learned_addr learned my addr 172.21.15.181:0/3083849628 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:01.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42878/0 (socket says 172.21.15.181:42878) 2020-11-16T22:34:01.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa51b0f700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 0x7faa4c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:01.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 -- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 msgr2=0x7faa4c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 0x7faa4c15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 -- 172.21.15.181:0/3083849628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 msgr2=0x7faa4c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 0x7faa4c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 -- 172.21.15.181:0/3083849628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c161c80 con 0x7faa4c0615f0 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa51b0f700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 0x7faa4c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa5130e700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7faa40012000 tx=0x7faa40015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:01.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400059a0 con 0x7faa4c0615f0 2020-11-16T22:34:01.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa40005b00 con 0x7faa4c0615f0 2020-11-16T22:34:01.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c161f20 con 0x7faa4c0615f0 2020-11-16T22:34:01.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40005ea0 con 0x7faa4c0615f0 2020-11-16T22:34:01.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.811+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c161400 con 0x7faa4c0615f0 2020-11-16T22:34:01.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faa40025030 con 0x7faa4c0615f0 2020-11-16T22:34:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa3d7fa700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 0x7faa24057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c161590 con 0x7faa4c0615f0 2020-11-16T22:34:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa4006f1d0 con 0x7faa4c0615f0 2020-11-16T22:34:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa50b0d700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 0x7faa24057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.815+0000 7faa50b0d700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 0x7faa24057520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7faa4c0b7050 tx=0x7faa48006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:00.819+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faa40023020 con 0x7faa4c0615f0 2020-11-16T22:34:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.019+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa4c069eb0 con 0x7faa4c0615f0 2020-11-16T22:34:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa3d7fa700 1 -- 172.21.15.181:0/3083849628 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4631 (secure 0 0 0) 0x7faa40045430 con 0x7faa4c0615f0 2020-11-16T22:34:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 msgr2=0x7faa24057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 0x7faa24057520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7faa4c0b7050 tx=0x7faa48006df0).stop 2020-11-16T22:34:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 msgr2=0x7faa4c163e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7faa40012000 tx=0x7faa40015000).stop 2020-11-16T22:34:01.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 shutdown_connections 2020-11-16T22:34:01.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa24055060 0x7faa24057520 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4c15f5b0 0x7faa4c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4c0615f0 0x7faa4c163e20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 --2- 172.21.15.181:0/3083849628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4c0622a0 0x7faa4c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 >> 172.21.15.181:0/3083849628 conn(0x7faa4c0d69a0 msgr2=0x7faa4c062730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 shutdown_connections 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:01.023+0000 7faa53572700 1 -- 172.21.15.181:0/3083849628 wait complete. 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 96 2020-11-16T22:34:01.979 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:01.980 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7396,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:00.652468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:02.182 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:34:02.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.323+0000 7f5c30b76700 1 Processor -- start 2020-11-16T22:34:02.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.323+0000 7f5c30b76700 1 -- start start 2020-11-16T22:34:02.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 0x7f5c2c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:02.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2c160e40 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2c160ba0 con 0x7f5c2c0615f0 2020-11-16T22:34:02.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1609a0 con 0x7f5c2c0629b0 2020-11-16T22:34:02.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c29d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5c2c0615f0 0x7f5c2c0622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49348/0 (socket says 172.21.15.181:49348) 2020-11-16T22:34:02.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c29d9b700 1 -- 172.21.15.181:0/303624925 learned_addr learned my addr 172.21.15.181:0/303624925 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:02.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2ad9d700 1 --2- 172.21.15.181:0/303624925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 0x7f5c2c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:02.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2959a700 1 -- 172.21.15.181:0/303624925 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827841176 0 0) 0x7f5c2c160ba0 con 0x7f5c2c0615f0 2020-11-16T22:34:02.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2959a700 1 -- 172.21.15.181:0/303624925 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c10003240 con 0x7f5c2c0615f0 2020-11-16T22:34:02.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/303624925 >> v1:172.21.15.181:6789/0 conn(0x7f5c2c0615f0 legacy=0x7f5c2c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/303624925 >> v1:172.21.15.175:6789/0 conn(0x7f5c2c15f5b0 legacy=0x7f5c2c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/303624925 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c060970 con 0x7f5c2c0629b0 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2ad9d700 1 --2- 172.21.15.181:0/303624925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 0x7f5c2c080fd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5c24009cc0 tx=0x7f5c240091d0).ready entity=mon.2 client_cookie=44afb2a6c33624ae server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2959a700 1 -- 172.21.15.181:0/303624925 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400bad0 con 0x7f5c2c0629b0 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2959a700 1 -- 172.21.15.181:0/303624925 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2400bc30 con 0x7f5c2c0629b0 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c2959a700 1 -- 172.21.15.181:0/303624925 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24011400 con 0x7f5c2c0629b0 2020-11-16T22:34:02.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- 172.21.15.181:0/303624925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 msgr2=0x7f5c2c080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 --2- 172.21.15.181:0/303624925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 0x7f5c2c080fd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5c24009cc0 tx=0x7f5c240091d0).stop 2020-11-16T22:34:02.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- 172.21.15.181:0/303624925 shutdown_connections 2020-11-16T22:34:02.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 --2- 172.21.15.181:0/303624925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c0629b0 0x7f5c2c080fd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- 172.21.15.181:0/303624925 >> 172.21.15.181:0/303624925 conn(0x7f5c2c0d69a0 msgr2=0x7f5c2c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:02.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- 172.21.15.181:0/303624925 shutdown_connections 2020-11-16T22:34:02.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- 172.21.15.181:0/303624925 wait complete. 2020-11-16T22:34:02.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 Processor -- start 2020-11-16T22:34:02.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.327+0000 7f5c30b76700 1 -- start start 2020-11-16T22:34:02.526 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 0x7f5c2c0d5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c2c0629b0 0x7f5c2c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c16f540 con 0x7f5c2c0629b0 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c16f7e0 con 0x7f5c2c0615f0 2020-11-16T22:34:02.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c16fa80 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 0x7f5c2c0d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:02.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:02.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42896/0 (socket says 172.21.15.181:42896) 2020-11-16T22:34:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/3752589952 learned_addr learned my addr 172.21.15.181:0/3752589952 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c29d9b700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c2c0629b0 0x7f5c2c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 msgr2=0x7f5c2c0d5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 0x7f5c2c0d5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/3752589952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c2c0629b0 msgr2=0x7f5c2c14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c2c0629b0 0x7f5c2c14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 -- 172.21.15.181:0/3752589952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c16fd20 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2a59c700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 0x7f5c2c0d5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c2ad9d700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f5c24009210 tx=0x7f5c24016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401c6c0 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2401c820 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401cba0 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c16ffc0 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c0d5550 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.331+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c170460 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.335+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5c24024030 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.335+0000 7f5c227fc700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 0x7f5bfc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.335+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c2406f900 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.335+0000 7f5c2a59c700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 0x7f5bfc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.335+0000 7f5c2a59c700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 0x7f5bfc057520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f5c2c0bcae0 tx=0x7f5c1c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.339+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5c24045c90 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.491+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c2c0d27a0 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c227fc700 1 -- 172.21.15.181:0/3752589952 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f5c24030200 con 0x7f5c2c15f5b0 2020-11-16T22:34:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 msgr2=0x7f5bfc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 0x7f5bfc057520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f5c2c0bcae0 tx=0x7f5c1c009040).stop 2020-11-16T22:34:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 msgr2=0x7f5c2c14b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f5c24009210 tx=0x7f5c24016000).stop 2020-11-16T22:34:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 shutdown_connections 2020-11-16T22:34:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5bfc055060 0x7f5bfc057520 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c2c15f5b0 0x7f5c2c14b830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2c0615f0 0x7f5c2c0d5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 --2- 172.21.15.181:0/3752589952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c2c0629b0 0x7f5c2c14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 >> 172.21.15.181:0/3752589952 conn(0x7f5c2c0d69a0 msgr2=0x7f5c2c15c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 shutdown_connections 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:02.495+0000 7f5c30b76700 1 -- 172.21.15.181:0/3752589952 wait complete. 2020-11-16T22:34:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:02.940 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:34:03.551 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:34:03.557 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:03.559 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:03.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.707+0000 7fc005531700 1 Processor -- start 2020-11-16T22:34:03.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.707+0000 7fc005531700 1 -- start start 2020-11-16T22:34:03.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc000081550 0x7fc0001612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:03.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000d1bc0 con 0x7fc0001617b0 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000d1e60 con 0x7fc000081170 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000d2100 con 0x7fc000081550 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc000081170 0x7fc0000d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49362/0 (socket says 172.21.15.181:49362) 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffffff700 1 -- 172.21.15.181:0/2981882461 learned_addr learned my addr 172.21.15.181:0/2981882461 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/2981882461 >> v2:172.21.15.175:3300/0 conn(0x7fc000081550 0x7fc0001612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:03.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190898839 0 0) 0x7fc0000d1e60 con 0x7fc000081170 2020-11-16T22:34:03.701 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfec003240 con 0x7fc000081170 2020-11-16T22:34:03.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1120877363 0 0) 0x7fbfec003240 con 0x7fc000081170 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 >> v2:172.21.15.175:3300/0 conn(0x7fc000081550 msgr2=0x7fc0001612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 --2- 172.21.15.181:0/2981882461 >> v2:172.21.15.175:3300/0 conn(0x7fc000081550 0x7fc0001612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 >> v1:172.21.15.175:6789/0 conn(0x7fc0001617b0 legacy=0x7fc0001455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000d23a0 con 0x7fc000081170 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7fbff00014e0 con 0x7fc000081170 2020-11-16T22:34:03.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 >> v1:172.21.15.181:6789/0 conn(0x7fc000081170 legacy=0x7fc0000d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:03.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfec003730 0x7fbfec003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:03.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fbfec004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:03.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec004e50 0x7fbfec00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:03.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000d2100 con 0x7fbfec003730 2020-11-16T22:34:03.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0042c0 con 0x7fbfec004e50 2020-11-16T22:34:03.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec00bc20 con 0x7fbfec0044e0 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc004d30700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec004e50 0x7fbfec00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffffff700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfec003730 0x7fbfec003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fbfec004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 -- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec004e50 msgr2=0x7fbfec00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec004e50 0x7fbfec00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 -- 172.21.15.181:0/2981882461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfec003730 msgr2=0x7fbfec003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:03.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfec003730 0x7fbfec003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:03.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 -- 172.21.15.181:0/2981882461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec003240 con 0x7fbfec0044e0 2020-11-16T22:34:03.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc004d30700 1 -- 172.21.15.181:0/2981882461 reap_dead start 2020-11-16T22:34:03.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fbfec004960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fbff400bc20 tx=0x7fbff4006df0).ready entity=mon.2 client_cookie=a2b004c91956f12b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:04.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400b300 con 0x7fbfec0044e0 2020-11-16T22:34:04.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff400e9d0 con 0x7fbfec0044e0 2020-11-16T22:34:04.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fbffeffd700 1 -- 172.21.15.181:0/2981882461 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400ed50 con 0x7fbfec0044e0 2020-11-16T22:34:04.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 msgr2=0x7fbfec004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fbfec004960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fbff400bc20 tx=0x7fbff4006df0).stop 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- 172.21.15.181:0/2981882461 shutdown_connections 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 --2- 172.21.15.181:0/2981882461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fbfec004960 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- 172.21.15.181:0/2981882461 >> 172.21.15.181:0/2981882461 conn(0x7fc0000d67e0 msgr2=0x7fc0000d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- 172.21.15.181:0/2981882461 shutdown_connections 2020-11-16T22:34:04.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.711+0000 7fc005531700 1 -- 172.21.15.181:0/2981882461 wait complete. 2020-11-16T22:34:04.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 Processor -- start 2020-11-16T22:34:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- start start 2020-11-16T22:34:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 0x7fc0000d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000d70b0 0x7fc0000bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00017b9c0 con 0x7fc0000bad10 2020-11-16T22:34:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00017bc30 con 0x7fc0000d70b0 2020-11-16T22:34:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00017bed0 con 0x7fbfec0044e0 2020-11-16T22:34:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42912/0 (socket says 172.21.15.181:42912) 2020-11-16T22:34:04.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 -- 172.21.15.181:0/3536435325 learned_addr learned my addr 172.21.15.181:0/3536435325 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc004d30700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000d70b0 0x7fc0000bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 0x7fc0000d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 -- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000d70b0 msgr2=0x7fc0000bb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000d70b0 0x7fc0000bb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 -- 172.21.15.181:0/3536435325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 msgr2=0x7fc0000d6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 0x7fc0000d6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 -- 172.21.15.181:0/3536435325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00017c170 con 0x7fbfec0044e0 2020-11-16T22:34:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 0x7fc0000d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:04.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbffffff700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbff0004dd0 tx=0x7fbff0008b20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:04.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000fb60 con 0x7fbfec0044e0 2020-11-16T22:34:04.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00017c410 con 0x7fbfec0044e0 2020-11-16T22:34:04.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff000fcc0 con 0x7fbfec0044e0 2020-11-16T22:34:04.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0016440 con 0x7fbfec0044e0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.715+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00017c950 con 0x7fbfec0044e0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbff00167b0 con 0x7fbfec0044e0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000069eb0 con 0x7fbfec0044e0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fbfdffff700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 0x7fbfd8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 0x7fbfd8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbff006df20 con 0x7fbfec0044e0 2020-11-16T22:34:04.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.719+0000 7fbfff7fe700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 0x7fbfd8057520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fbff4006d00 tx=0x7fbff4019000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.727+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbff0044010 con 0x7fbfec0044e0 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.959+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fc0000b5bd0 con 0x7fbfec0044e0 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.971+0000 7fbfdffff700 1 -- 172.21.15.181:0/3536435325 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbff0047630 con 0x7fbfec0044e0 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.971+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 msgr2=0x7fbfd8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.971+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 0x7fbfd8057520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fbff4006d00 tx=0x7fbff4019000).stop 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.971+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 msgr2=0x7fc0000d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.971+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbff0004dd0 tx=0x7fbff0008b20).stop 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 shutdown_connections 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbfd8055060 0x7fbfd8057520 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000d70b0 0x7fc0000bb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfec0044e0 0x7fc0000d04c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 --2- 172.21.15.181:0/3536435325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0000bad10 0x7fc0000d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 >> 172.21.15.181:0/3536435325 conn(0x7fc0000d67e0 msgr2=0x7fc0000d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:04.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 shutdown_connections 2020-11-16T22:34:04.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:03.975+0000 7fc005531700 1 -- 172.21.15.181:0/3536435325 wait complete. 2020-11-16T22:34:04.526 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:34:04.526 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:04.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 Processor -- start 2020-11-16T22:34:04.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- start start 2020-11-16T22:34:04.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 0x7f5c4c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c4c0d1830 0x7f5c4c0629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c160e40 con 0x7f5c4c0615f0 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c1609a0 con 0x7f5c4c15f5b0 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c160ba0 con 0x7f5c4c0d1830 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c4c0d1830 0x7f5c4c0629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c4c0d1830 0x7f5c4c0629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35078/0 (socket says 172.21.15.181:35078) 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 -- 172.21.15.181:0/1160808649 learned_addr learned my addr 172.21.15.181:0/1160808649 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c50e1d700 1 --2- 172.21.15.181:0/1160808649 >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 0x7f5c4c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 -- 172.21.15.181:0/1160808649 >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 msgr2=0x7f5c4c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 --2- 172.21.15.181:0/1160808649 >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 0x7f5c4c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 -- 172.21.15.181:0/1160808649 >> v1:172.21.15.175:6789/0 conn(0x7f5c4c0615f0 legacy=0x7f5c4c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 -- 172.21.15.181:0/1160808649 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c060970 con 0x7f5c4c0d1830 2020-11-16T22:34:04.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c50e1d700 1 --2- 172.21.15.181:0/1160808649 >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 0x7f5c4c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c5161e700 1 --2- 172.21.15.181:0/1160808649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0629b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f5c4c0d2ea0 tx=0x7f5c3c006eb0).ready entity=mon.1 client_cookie=f128a49f0200bc77 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c4b7fe700 1 -- 172.21.15.181:0/1160808649 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009890 con 0x7f5c4c0d1830 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c4b7fe700 1 -- 172.21.15.181:0/1160808649 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c0099f0 con 0x7f5c4c0d1830 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c4b7fe700 1 -- 172.21.15.181:0/1160808649 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009d70 con 0x7f5c4c0d1830 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- 172.21.15.181:0/1160808649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c0d1830 msgr2=0x7f5c4c0629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:04.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 --2- 172.21.15.181:0/1160808649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0629b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f5c4c0d2ea0 tx=0x7f5c3c006eb0).stop 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- 172.21.15.181:0/1160808649 shutdown_connections 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 --2- 172.21.15.181:0/1160808649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0629b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 --2- 172.21.15.181:0/1160808649 >> v2:172.21.15.175:3300/0 conn(0x7f5c4c15f5b0 0x7f5c4c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- 172.21.15.181:0/1160808649 >> 172.21.15.181:0/1160808649 conn(0x7f5c4c0d69a0 msgr2=0x7f5c4c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- 172.21.15.181:0/1160808649 shutdown_connections 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.667+0000 7f5c53081700 1 -- 172.21.15.181:0/1160808649 wait complete. 2020-11-16T22:34:04.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 Processor -- start 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 -- start start 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c4c0615f0 0x7f5c4c0b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:05.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0ba2c0 con 0x7f5c4c0d1830 2020-11-16T22:34:05.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0bc070 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0ba9c0 con 0x7f5c4c0615f0 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c50e1d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c4c0615f0 0x7f5c4c0b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35088/0 (socket says 172.21.15.181:35088) 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 -- 172.21.15.181:0/56581759 learned_addr learned my addr 172.21.15.181:0/56581759 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c4bfff700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 -- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c4c0615f0 msgr2=0x7f5c4c0b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c4c0615f0 0x7f5c4c0b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 -- 172.21.15.181:0/56581759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0d1830 msgr2=0x7f5c4c0bbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0bbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 -- 172.21.15.181:0/56581759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0bac60 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c5161e700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f5c4c0d6970 tx=0x7f5c3c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:05.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c002ef0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c01d5d0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c0baf00 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.671+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c01d970 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c0bb6e0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c0bb520 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5c3c025030 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c48ff9700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 0x7f5c1c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c3c06f6b0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c50e1d700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 0x7f5c1c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:05.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.675+0000 7f5c50e1d700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 0x7f5c1c057520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5c44009870 tx=0x7f5c44008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:05.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.679+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5c3c0459c0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.883+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c4c069eb0 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.883+0000 7f5c48ff9700 1 -- 172.21.15.181:0/56581759 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4631 (secure 0 0 0) 0x7f5c3c023070 con 0x7f5c4c15f5b0 2020-11-16T22:34:05.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 msgr2=0x7f5c1c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:05.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 0x7f5c1c057520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5c44009870 tx=0x7f5c44008040).stop 2020-11-16T22:34:05.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 msgr2=0x7f5c4c170670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:05.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f5c4c0d6970 tx=0x7f5c3c015000).stop 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 shutdown_connections 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5c1c055060 0x7f5c1c057520 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c4c15f5b0 0x7f5c4c170670 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c4c0615f0 0x7f5c4c0b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 --2- 172.21.15.181:0/56581759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0d1830 0x7f5c4c0bbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:05.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 >> 172.21.15.181:0/56581759 conn(0x7f5c4c0d69a0 msgr2=0x7f5c4c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:05.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 shutdown_connections 2020-11-16T22:34:05.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:04.887+0000 7f5c53081700 1 -- 172.21.15.181:0/56581759 wait complete. 2020-11-16T22:34:05.484 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 96 2020-11-16T22:34:05.484 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:05.484 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7396,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:00.652468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:06.796 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:33:55.623227+0000', 'modified': '2020-11-16T22:34:00.652468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7405}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7405': {'gid': 7405, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6839/1296416553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1296416553}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1296416553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:34:06.796 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:34:06.797 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:34:06.931 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":7405,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.303586942000001,"mdsmap_epoch":96,"osdmap_epoch":97,"osdmap_epoch_barrier":97,"uptime":20.735542679000002} 2020-11-16T22:34:06.932 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 7405, "mdsmap_epoch": 96, "osdmap_epoch": 97, "osdmap_epoch_barrier": 97, "rank_uptime": 11.303586942, "state": "up:active", "uptime": 20.735542679, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:34:06.932 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:34:06.932 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:34:06.933 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:34:06.933 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:34:06.933 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:34:06.933 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:34:06.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:34:06.934 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:34:06.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:06.935 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:34:06.956 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:34:06.957 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:34:06.958 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:34:06.959 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.959 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:34:06.959 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:34:06.959 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:34:06.959 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:34:06.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:06.960 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:34:06.995 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:34:06.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:07.000 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:34:07.015 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:34:07.016 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:34:07.016 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:34:07.016 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:34:07.016 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:34:07.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:07.017 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:34:07.017 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:34:07.017 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:34:07.018 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:34:07.018 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:34:07.018 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:34:07.179 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:34:07.179 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:34:07.180 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:34:07.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:07.181 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:34:07.208 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:34:07.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.346+0000 7f8b55828700 1 Processor -- start 2020-11-16T22:34:07.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.346+0000 7f8b55828700 1 -- start start 2020-11-16T22:34:07.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8b500cf920 0x7f8b500656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500c85e0 con 0x7f8b500cf4a0 2020-11-16T22:34:07.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5015ff70 con 0x7f8b500cf920 2020-11-16T22:34:07.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500c8840 con 0x7f8b5015d140 2020-11-16T22:34:07.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42936/0 (socket says 172.21.15.181:42936) 2020-11-16T22:34:07.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 -- 172.21.15.181:0/4195416926 learned_addr learned my addr 172.21.15.181:0/4195416926 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:07.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55027700 1 --2- 172.21.15.181:0/4195416926 >> v2:172.21.15.175:3300/0 conn(0x7f8b500cf920 0x7f8b500656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 -- 172.21.15.181:0/4195416926 >> v2:172.21.15.175:3300/0 conn(0x7f8b500cf920 msgr2=0x7f8b500656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 --2- 172.21.15.181:0/4195416926 >> v2:172.21.15.175:3300/0 conn(0x7f8b500cf920 0x7f8b500656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 -- 172.21.15.181:0/4195416926 >> v1:172.21.15.175:6789/0 conn(0x7f8b500cf4a0 legacy=0x7f8b50064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 -- 172.21.15.181:0/4195416926 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500c8ae0 con 0x7f8b5015d140 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b54826700 1 --2- 172.21.15.181:0/4195416926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f8b40006fd0 tx=0x7f8b40006eb0).ready entity=mon.2 client_cookie=4a66b1680e7be6d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b4f7fe700 1 -- 172.21.15.181:0/4195416926 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40009810 con 0x7f8b5015d140 2020-11-16T22:34:07.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b4f7fe700 1 -- 172.21.15.181:0/4195416926 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40009970 con 0x7f8b5015d140 2020-11-16T22:34:07.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b4f7fe700 1 -- 172.21.15.181:0/4195416926 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40009cf0 con 0x7f8b5015d140 2020-11-16T22:34:07.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- 172.21.15.181:0/4195416926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 msgr2=0x7f8b5015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 --2- 172.21.15.181:0/4195416926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f8b40006fd0 tx=0x7f8b40006eb0).stop 2020-11-16T22:34:07.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- 172.21.15.181:0/4195416926 shutdown_connections 2020-11-16T22:34:07.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 --2- 172.21.15.181:0/4195416926 >> v2:172.21.15.175:3300/0 conn(0x7f8b500cf920 0x7f8b500656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 --2- 172.21.15.181:0/4195416926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5015d140 0x7f8b5015d520 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- 172.21.15.181:0/4195416926 >> 172.21.15.181:0/4195416926 conn(0x7f8b500d2dc0 msgr2=0x7f8b500ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:07.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- 172.21.15.181:0/4195416926 shutdown_connections 2020-11-16T22:34:07.547 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.350+0000 7f8b55828700 1 -- 172.21.15.181:0/4195416926 wait complete. 2020-11-16T22:34:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.354+0000 7f8b55828700 1 Processor -- start 2020-11-16T22:34:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.354+0000 7f8b55828700 1 -- start start 2020-11-16T22:34:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b500cf4a0 0x7f8b501439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 0x7f8b500656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5016f400 con 0x7f8b500cf4a0 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50142db0 con 0x7f8b5015d140 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50065ba0 con 0x7f8b500cf920 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55027700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 0x7f8b500656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42942/0 (socket says 172.21.15.181:42942) 2020-11-16T22:34:07.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 -- 172.21.15.181:0/97628933 learned_addr learned my addr 172.21.15.181:0/97628933 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55027700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 0x7f8b500656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35102/0 (socket says 172.21.15.181:35102) 2020-11-16T22:34:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b54826700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b500cf4a0 0x7f8b501439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 -- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 msgr2=0x7f8b500656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 0x7f8b500656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 -- 172.21.15.181:0/97628933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b500cf4a0 msgr2=0x7f8b501439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b500cf4a0 0x7f8b501439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 -- 172.21.15.181:0/97628933 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50160630 con 0x7f8b500cf920 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4ffff700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8b4400c740 tx=0x7f8b44007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400e490 con 0x7f8b500cf920 2020-11-16T22:34:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400e5f0 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b501608d0 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400e9b0 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b50143520 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b50143360 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.358+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8b44019030 con 0x7f8b500cf920 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.362+0000 7f8b4cff9700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 0x7f8b28057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.362+0000 7f8b54826700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 0x7f8b28057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.362+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b4406d140 con 0x7f8b500cf920 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.362+0000 7f8b54826700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 0x7f8b28057520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f8b4000b1b0 tx=0x7f8b40017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.366+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8b440433a0 con 0x7f8b500cf920 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.518+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b50044f60 con 0x7f8b500cf920 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.518+0000 7f8b4cff9700 1 -- 172.21.15.181:0/97628933 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f8b44017020 con 0x7f8b500cf920 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 msgr2=0x7f8b28057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 0x7f8b28057520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f8b4000b1b0 tx=0x7f8b40017000).stop 2020-11-16T22:34:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 msgr2=0x7f8b5016ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8b4400c740 tx=0x7f8b44007b80).stop 2020-11-16T22:34:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 shutdown_connections 2020-11-16T22:34:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8b28055060 0x7f8b28057520 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5015d140 0x7f8b500656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b500cf4a0 0x7f8b501439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 --2- 172.21.15.181:0/97628933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b500cf920 0x7f8b5016ef10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 >> 172.21.15.181:0/97628933 conn(0x7f8b500d2dc0 msgr2=0x7f8b500b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 shutdown_connections 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:07.522+0000 7f8b55828700 1 -- 172.21.15.181:0/97628933 wait complete. 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:07.963 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:34:08.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:33:53.695421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:33:54.659875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:34:08.371 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:34:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 Processor -- start 2020-11-16T22:34:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 -- start start 2020-11-16T22:34:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feb5815fbe0 0x7feb5815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb580cfbb0 con 0x7feb5815cd00 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb580659d0 con 0x7feb5815fbe0 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5e4b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5815d770 con 0x7feb5815b810 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42950/0 (socket says 172.21.15.181:42950) 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 -- 172.21.15.181:0/927146447 learned_addr learned my addr 172.21.15.181:0/927146447 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5ccb3700 1 --2- 172.21.15.181:0/927146447 >> v2:172.21.15.175:3300/0 conn(0x7feb5815fbe0 0x7feb5815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 -- 172.21.15.181:0/927146447 >> v2:172.21.15.175:3300/0 conn(0x7feb5815fbe0 msgr2=0x7feb5815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 --2- 172.21.15.181:0/927146447 >> v2:172.21.15.175:3300/0 conn(0x7feb5815fbe0 0x7feb5815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 -- 172.21.15.181:0/927146447 >> v1:172.21.15.175:6789/0 conn(0x7feb5815cd00 legacy=0x7feb580cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.522+0000 7feb5d4b4700 1 -- 172.21.15.181:0/927146447 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5815da10 con 0x7feb5815b810 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5d4b4700 1 --2- 172.21.15.181:0/927146447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7feb50009d80 tx=0x7feb50009290).ready entity=mon.2 client_cookie=3c37ef6207b9fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb57fff700 1 -- 172.21.15.181:0/927146447 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000bb80 con 0x7feb5815b810 2020-11-16T22:34:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb57fff700 1 -- 172.21.15.181:0/927146447 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5000bce0 con 0x7feb5815b810 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb57fff700 1 -- 172.21.15.181:0/927146447 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50011610 con 0x7feb5815b810 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- 172.21.15.181:0/927146447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 msgr2=0x7feb5815f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/927146447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7feb50009d80 tx=0x7feb50009290).stop 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- 172.21.15.181:0/927146447 shutdown_connections 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/927146447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815b810 0x7feb5815f6f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/927146447 >> v2:172.21.15.175:3300/0 conn(0x7feb5815fbe0 0x7feb5815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- 172.21.15.181:0/927146447 >> 172.21.15.181:0/927146447 conn(0x7feb580d2dc0 msgr2=0x7feb580ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- 172.21.15.181:0/927146447 shutdown_connections 2020-11-16T22:34:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- 172.21.15.181:0/927146447 wait complete. 2020-11-16T22:34:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 Processor -- start 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.526+0000 7feb5e4b6700 1 -- start start 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb5815b810 0x7feb581443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cd00 0x7feb580cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580ccba0 con 0x7feb5815b810 2020-11-16T22:34:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58170850 con 0x7feb5815cd00 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb58170af0 con 0x7feb5815fbe0 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42956/0 (socket says 172.21.15.181:42956) 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 -- 172.21.15.181:0/491646602 learned_addr learned my addr 172.21.15.181:0/491646602 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5d4b4700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb5815b810 0x7feb581443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5ccb3700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cd00 0x7feb580cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 -- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cd00 msgr2=0x7feb580cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cd00 0x7feb580cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 -- 172.21.15.181:0/491646602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb5815b810 msgr2=0x7feb581443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb5815b810 0x7feb581443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 -- 172.21.15.181:0/491646602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58170d90 con 0x7feb5815fbe0 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5dcb5700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7feb4800c740 tx=0x7feb48007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800e490 con 0x7feb5815fbe0 2020-11-16T22:34:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4800e5f0 con 0x7feb5815fbe0 2020-11-16T22:34:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb58171030 con 0x7feb5815fbe0 2020-11-16T22:34:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800e990 con 0x7feb5815fbe0 2020-11-16T22:34:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5814e490 con 0x7feb5815fbe0 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5814e710 con 0x7feb5815fbe0 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feb48010030 con 0x7feb5815fbe0 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.530+0000 7feb557fa700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 0x7feb34057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.534+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb4806da70 con 0x7feb5815fbe0 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.534+0000 7feb5d4b4700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 0x7feb34057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.534+0000 7feb5d4b4700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 0x7feb34057520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7feb50009eb0 tx=0x7feb50016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.538+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7feb48043cd0 con 0x7feb5815fbe0 2020-11-16T22:34:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7feb58044f60 con 0x7feb5815fbe0 2020-11-16T22:34:08.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb557fa700 1 -- 172.21.15.181:0/491646602 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7feb4806de60 con 0x7feb5815fbe0 2020-11-16T22:34:08.677 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:34:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 msgr2=0x7feb34057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 0x7feb34057520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7feb50009eb0 tx=0x7feb50016000).stop 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 msgr2=0x7feb58150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7feb4800c740 tx=0x7feb48007ac0).stop 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 shutdown_connections 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feb34055060 0x7feb34057520 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5815fbe0 0x7feb58150f00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb5815b810 0x7feb581443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 --2- 172.21.15.181:0/491646602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cd00 0x7feb580cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 >> 172.21.15.181:0/491646602 conn(0x7feb580d2dc0 msgr2=0x7feb580d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 shutdown_connections 2020-11-16T22:34:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.686+0000 7feb5e4b6700 1 -- 172.21.15.181:0/491646602 wait complete. 2020-11-16T22:34:08.691 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:34:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.826+0000 7f9fa1101700 1 Processor -- start 2020-11-16T22:34:08.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.826+0000 7f9fa1101700 1 -- start start 2020-11-16T22:34:08.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9f9c145a30 0x7f9f9c145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9f9c1492c0 0x7f9f9c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c061b50 con 0x7f9f9c0814c0 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0618b0 con 0x7f9f9c145a30 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0cb930 con 0x7f9f9c1492c0 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9f9c1492c0 0x7f9f9c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa0900700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9f9c0814c0 0x7f9f9c0cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49426/0 (socket says 172.21.15.181:49426) 2020-11-16T22:34:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa0900700 1 -- 172.21.15.181:0/2601408259 learned_addr learned my addr 172.21.15.181:0/2601408259 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9bfff700 1 --2- 172.21.15.181:0/2601408259 >> v2:172.21.15.175:3300/0 conn(0x7f9f9c145a30 0x7f9f9c145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/2601408259 >> v2:172.21.15.175:3300/0 conn(0x7f9f9c145a30 msgr2=0x7f9f9c145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 --2- 172.21.15.181:0/2601408259 >> v2:172.21.15.175:3300/0 conn(0x7f9f9c145a30 0x7f9f9c145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/2601408259 >> v1:172.21.15.181:6789/0 conn(0x7f9f9c0814c0 legacy=0x7f9f9c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/2601408259 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0cbbd0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9affd700 1 -- 172.21.15.181:0/2601408259 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414865510 0 0) 0x7f9f9c061b50 con 0x7f9f9c0814c0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9b7fe700 1 --2- 172.21.15.181:0/2601408259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c080fd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f9f9c0cd910 tx=0x7f9f90006df0).ready entity=mon.1 client_cookie=a405fb54f959db02 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9affd700 1 -- 172.21.15.181:0/2601408259 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f900097b0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9affd700 1 -- 172.21.15.181:0/2601408259 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90009910 con 0x7f9f9c1492c0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9f9affd700 1 -- 172.21.15.181:0/2601408259 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90009c90 con 0x7f9f9c1492c0 2020-11-16T22:34:08.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- 172.21.15.181:0/2601408259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 msgr2=0x7f9f9c080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 --2- 172.21.15.181:0/2601408259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c080fd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f9f9c0cd910 tx=0x7f9f90006df0).stop 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- 172.21.15.181:0/2601408259 shutdown_connections 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 --2- 172.21.15.181:0/2601408259 >> v2:172.21.15.175:3300/0 conn(0x7f9f9c145a30 0x7f9f9c145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 --2- 172.21.15.181:0/2601408259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c080fd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- 172.21.15.181:0/2601408259 >> 172.21.15.181:0/2601408259 conn(0x7f9f9c0d68c0 msgr2=0x7f9f9c0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:08.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- 172.21.15.181:0/2601408259 shutdown_connections 2020-11-16T22:34:08.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.830+0000 7f9fa1101700 1 -- 172.21.15.181:0/2601408259 wait complete. 2020-11-16T22:34:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 Processor -- start 2020-11-16T22:34:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 -- start start 2020-11-16T22:34:08.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 0x7f9f9c16fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0cdbd0 con 0x7f9f9c0814c0 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0cf1f0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa1101700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0bc2d0 con 0x7f9f9c145a30 2020-11-16T22:34:08.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42970/0 (socket says 172.21.15.181:42970) 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35130/0 (socket says 172.21.15.181:35130) 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3034991809 learned_addr learned my addr 172.21.15.181:0/3034991809 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9bfff700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 0x7f9f9c16fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 -- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 msgr2=0x7f9f9c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 -- 172.21.15.181:0/3034991809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 msgr2=0x7f9f9c16fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 0x7f9f9c16fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9fa0900700 1 -- 172.21.15.181:0/3034991809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0bc570 con 0x7f9f9c1492c0 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9bfff700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 0x7f9f9c16fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.834+0000 7f9f9b7fe700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9fa0900700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f9f9400cae0 tx=0x7f9f94007c60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400ceb0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94005b80 con 0x7f9f9c1492c0 2020-11-16T22:34:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94007110 con 0x7f9f9c1492c0 2020-11-16T22:34:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c0bc810 con 0x7f9f9c1492c0 2020-11-16T22:34:08.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c0ce0a0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c0ce2a0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9f94010030 con 0x7f9f9c1492c0 2020-11-16T22:34:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 0x7f9f74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f9406da90 con 0x7f9f9c1492c0 2020-11-16T22:34:08.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.838+0000 7f9f9bfff700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 0x7f9f74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:08.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.842+0000 7f9f9bfff700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 0x7f9f74057520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f9f9c082160 tx=0x7f9f8c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.846+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9f9c0ce2a0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.998+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9f9c0ce2a0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:08.998+0000 7f9f7bfff700 1 -- 172.21.15.181:0/3034991809 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f9c0ce2a0 con 0x7f9f9c1492c0 2020-11-16T22:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:34:08.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 msgr2=0x7f9f74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 0x7f9f74057520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f9f9c082160 tx=0x7f9f8c006df0).stop 2020-11-16T22:34:08.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 msgr2=0x7f9f9c0bb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f9f9400cae0 tx=0x7f9f94007c60).stop 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 shutdown_connections 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f74055060 0x7f9f74057520 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9c1492c0 0x7f9f9c0bb870 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c0814c0 0x7f9f9c16fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 --2- 172.21.15.181:0/3034991809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f9c145a30 0x7f9f9c060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:08.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 >> 172.21.15.181:0/3034991809 conn(0x7f9f9c0d68c0 msgr2=0x7f9f9c0ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:08.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 shutdown_connections 2020-11-16T22:34:08.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.002+0000 7f9fa1101700 1 -- 172.21.15.181:0/3034991809 wait complete. 2020-11-16T22:34:09.005 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:34:09.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.138+0000 7f2c6cb06700 1 Processor -- start 2020-11-16T22:34:09.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.138+0000 7f2c6cb06700 1 -- start start 2020-11-16T22:34:09.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c68161480 0x7f2c68164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680c8db0 con 0x7f2c68164e10 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680c8330 con 0x7f2c68161480 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c85d0 con 0x7f2c68161860 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42976/0 (socket says 172.21.15.181:42976) 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 -- 172.21.15.181:0/1272318229 learned_addr learned my addr 172.21.15.181:0/1272318229 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:09.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c677fe700 1 --2- 172.21.15.181:0/1272318229 >> v2:172.21.15.175:3300/0 conn(0x7f2c68161480 0x7f2c68164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 -- 172.21.15.181:0/1272318229 >> v2:172.21.15.175:3300/0 conn(0x7f2c68161480 msgr2=0x7f2c68164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 --2- 172.21.15.181:0/1272318229 >> v2:172.21.15.175:3300/0 conn(0x7f2c68161480 0x7f2c68164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 -- 172.21.15.181:0/1272318229 >> v1:172.21.15.181:6789/0 conn(0x7f2c68164e10 legacy=0x7f2c68062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c667fc700 1 -- 172.21.15.181:0/1272318229 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361947317 0 0) 0x7f2c680c8db0 con 0x7f2c68164e10 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 -- 172.21.15.181:0/1272318229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680c8870 con 0x7f2c68161860 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c67fff700 1 --2- 172.21.15.181:0/1272318229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f2c50009d50 tx=0x7f2c50009290).ready entity=mon.2 client_cookie=ecb5df617de53ba0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c667fc700 1 -- 172.21.15.181:0/1272318229 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000bbc0 con 0x7f2c68161860 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c667fc700 1 -- 172.21.15.181:0/1272318229 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5000bd20 con 0x7f2c68161860 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c667fc700 1 -- 172.21.15.181:0/1272318229 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011610 con 0x7f2c68161860 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- 172.21.15.181:0/1272318229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 msgr2=0x7f2c68062960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/1272318229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f2c50009d50 tx=0x7f2c50009290).stop 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- 172.21.15.181:0/1272318229 shutdown_connections 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/1272318229 >> v2:172.21.15.175:3300/0 conn(0x7f2c68161480 0x7f2c68164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/1272318229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c68161860 0x7f2c68062960 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- 172.21.15.181:0/1272318229 >> 172.21.15.181:0/1272318229 conn(0x7f2c680d2dc0 msgr2=0x7f2c680ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- 172.21.15.181:0/1272318229 shutdown_connections 2020-11-16T22:34:09.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.142+0000 7f2c6cb06700 1 -- 172.21.15.181:0/1272318229 wait complete. 2020-11-16T22:34:09.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 Processor -- start 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- start start 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c68161860 0x7f2c68145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68163460 con 0x7f2c68161860 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68163700 con 0x7f2c68161480 2020-11-16T22:34:09.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681639a0 con 0x7f2c680d0120 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c67fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35142/0 (socket says 172.21.15.181:35142) 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c67fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42986/0 (socket says 172.21.15.181:42986) 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 -- 172.21.15.181:0/196157818 learned_addr learned my addr 172.21.15.181:0/196157818 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c66ffd700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c68161860 0x7f2c68145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 -- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 msgr2=0x7f2c681491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 -- 172.21.15.181:0/196157818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c68161860 msgr2=0x7f2c68145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c68161860 0x7f2c68145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 -- 172.21.15.181:0/196157818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68163c40 con 0x7f2c68161480 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c67fff700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c677fe700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f2c6000eba0 tx=0x7f2c60006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000c970 con 0x7f2c68161480 2020-11-16T22:34:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6000cad0 con 0x7f2c68161480 2020-11-16T22:34:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c68144650 con 0x7f2c68161480 2020-11-16T22:34:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000ce50 con 0x7f2c68161480 2020-11-16T22:34:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.146+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68144b90 con 0x7f2c68161480 2020-11-16T22:34:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68044f60 con 0x7f2c68161480 2020-11-16T22:34:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2c60012600 con 0x7f2c68161480 2020-11-16T22:34:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c5f7fe700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 0x7f2c38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c6006e190 con 0x7f2c68161480 2020-11-16T22:34:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c66ffd700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 0x7f2c38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.150+0000 7f2c66ffd700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 0x7f2c38057520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f2c680b8550 tx=0x7f2c58007970).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.154+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2c60044390 con 0x7f2c68161480 2020-11-16T22:34:09.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.310+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c680671e0 con 0x7f2c68161480 2020-11-16T22:34:09.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.310+0000 7f2c5f7fe700 1 -- 172.21.15.181:0/196157818 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2c600479b0 con 0x7f2c68161480 2020-11-16T22:34:09.302 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:34:09.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 msgr2=0x7f2c38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 0x7f2c38057520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f2c680b8550 tx=0x7f2c58007970).stop 2020-11-16T22:34:09.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 msgr2=0x7f2c68162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f2c6000eba0 tx=0x7f2c60006df0).stop 2020-11-16T22:34:09.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 shutdown_connections 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2c38055060 0x7f2c38057520 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680d0120 0x7f2c681491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68161480 0x7f2c68162e40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 --2- 172.21.15.181:0/196157818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c68161860 0x7f2c68145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 >> 172.21.15.181:0/196157818 conn(0x7f2c680d2dc0 msgr2=0x7f2c68063220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 shutdown_connections 2020-11-16T22:34:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.314+0000 7f2c6cb06700 1 -- 172.21.15.181:0/196157818 wait complete. 2020-11-16T22:34:09.317 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:34:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 Processor -- start 2020-11-16T22:34:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- start start 2020-11-16T22:34:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb65c15f5b0 0x7fb65c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c160e40 con 0x7fb65c0615f0 2020-11-16T22:34:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c160ba0 con 0x7fb65c0d1830 2020-11-16T22:34:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c1609a0 con 0x7fb65c15f5b0 2020-11-16T22:34:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb65c15f5b0 0x7fb65c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb65c15f5b0 0x7fb65c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35148/0 (socket says 172.21.15.181:35148) 2020-11-16T22:34:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 -- 172.21.15.181:0/2854203772 learned_addr learned my addr 172.21.15.181:0/2854203772 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 -- 172.21.15.181:0/2854203772 >> v1:172.21.15.181:6789/0 conn(0x7fb65c0d1830 legacy=0x7fb65c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 -- 172.21.15.181:0/2854203772 >> v1:172.21.15.175:6789/0 conn(0x7fb65c0615f0 legacy=0x7fb65c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 -- 172.21.15.181:0/2854203772 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c060970 con 0x7fb65c15f5b0 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65a59c700 1 --2- 172.21.15.181:0/2854203772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0d1340 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fb650006fd0 tx=0x7fb650006e50).ready entity=mon.1 client_cookie=98b7710e83e381e2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65959a700 1 -- 172.21.15.181:0/2854203772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650006030 con 0x7fb65c15f5b0 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65959a700 1 -- 172.21.15.181:0/2854203772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650006190 con 0x7fb65c15f5b0 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb65959a700 1 -- 172.21.15.181:0/2854203772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650006510 con 0x7fb65c15f5b0 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- 172.21.15.181:0/2854203772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c15f5b0 msgr2=0x7fb65c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 --2- 172.21.15.181:0/2854203772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0d1340 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fb650006fd0 tx=0x7fb650006e50).stop 2020-11-16T22:34:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- 172.21.15.181:0/2854203772 shutdown_connections 2020-11-16T22:34:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 --2- 172.21.15.181:0/2854203772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0d1340 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.458+0000 7fb660c70700 1 -- 172.21.15.181:0/2854203772 >> 172.21.15.181:0/2854203772 conn(0x7fb65c0d69a0 msgr2=0x7fb65c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 -- 172.21.15.181:0/2854203772 shutdown_connections 2020-11-16T22:34:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 -- 172.21.15.181:0/2854203772 wait complete. 2020-11-16T22:34:09.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 Processor -- start 2020-11-16T22:34:09.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 -- start start 2020-11-16T22:34:09.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.462+0000 7fb660c70700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 0x7fb65c081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c170580 con 0x7fb65c15f5b0 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c170bd0 con 0x7fb65c0615f0 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c170e70 con 0x7fb65c0d1830 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb659d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 0x7fb65c081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35154/0 (socket says 172.21.15.181:35154) 2020-11-16T22:34:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb659d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 0x7fb65c081660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42998/0 (socket says 172.21.15.181:42998) 2020-11-16T22:34:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 -- 172.21.15.181:0/2665303198 learned_addr learned my addr 172.21.15.181:0/2665303198 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65ad9d700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 -- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 msgr2=0x7fb65c081660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 0x7fb65c081660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 -- 172.21.15.181:0/2665303198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb65c15f5b0 msgr2=0x7fb65c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 -- 172.21.15.181:0/2665303198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c171110 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb65a59c700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fb65c05d520 tx=0x7fb650015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650006830 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650005a30 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c1713b0 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650005dd0 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c0cfd50 con 0x7fb65c0615f0 2020-11-16T22:34:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.466+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c069eb0 con 0x7fb65c0615f0 2020-11-16T22:34:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.470+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb650025030 con 0x7fb65c0615f0 2020-11-16T22:34:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.470+0000 7fb64a7fc700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 0x7fb630057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.470+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb65006f350 con 0x7fb65c0615f0 2020-11-16T22:34:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.470+0000 7fb659d9b700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 0x7fb630057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.470+0000 7fb659d9b700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 0x7fb630057520 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fb65c0b6810 tx=0x7fb64c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.474+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb650045660 con 0x7fb65c0615f0 2020-11-16T22:34:09.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb65c0600f0 con 0x7fb65c0615f0 2020-11-16T22:34:09.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb64a7fc700 1 -- 172.21.15.181:0/2665303198 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb650023070 con 0x7fb65c0615f0 2020-11-16T22:34:09.617 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:34:09.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 msgr2=0x7fb630057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 0x7fb630057520 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fb65c0b6810 tx=0x7fb64c009040).stop 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 msgr2=0x7fb65c081170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fb65c05d520 tx=0x7fb650015000).stop 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 shutdown_connections 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb630055060 0x7fb630057520 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb65c15f5b0 0x7fb65c0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb65c0615f0 0x7fb65c081170 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 --2- 172.21.15.181:0/2665303198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65c0d1830 0x7fb65c081660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 >> 172.21.15.181:0/2665303198 conn(0x7fb65c0d69a0 msgr2=0x7fb65c061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:09.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 shutdown_connections 2020-11-16T22:34:09.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.626+0000 7fb660c70700 1 -- 172.21.15.181:0/2665303198 wait complete. 2020-11-16T22:34:09.631 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:34:09.631 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.762+0000 7f6ca86bb700 1 Processor -- start 2020-11-16T22:34:09.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.762+0000 7f6ca86bb700 1 -- start start 2020-11-16T22:34:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca86bb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ca0160430 0x7f6ca0161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca86bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 0x7f6ca0160810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca86bb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca0161a10 con 0x7f6ca0162bd0 2020-11-16T22:34:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca86bb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca00c8a40 con 0x7f6ca0160430 2020-11-16T22:34:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca86bb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0160d00 con 0x7f6ca00c85e0 2020-11-16T22:34:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca6c58700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6ca0162bd0 0x7f6ca00600f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49470/0 (socket says 172.21.15.181:49470) 2020-11-16T22:34:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca6c58700 1 -- 172.21.15.181:0/2284620083 learned_addr learned my addr 172.21.15.181:0/2284620083 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca6457700 1 --2- 172.21.15.181:0/2284620083 >> v2:172.21.15.175:3300/0 conn(0x7f6ca0160430 0x7f6ca0161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 --2- 172.21.15.181:0/2284620083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 0x7f6ca0160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:09.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 -- 172.21.15.181:0/2284620083 >> v2:172.21.15.175:3300/0 conn(0x7f6ca0160430 msgr2=0x7f6ca0161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:09.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 --2- 172.21.15.181:0/2284620083 >> v2:172.21.15.175:3300/0 conn(0x7f6ca0160430 0x7f6ca0161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 -- 172.21.15.181:0/2284620083 >> v1:172.21.15.181:6789/0 conn(0x7f6ca0162bd0 legacy=0x7f6ca00600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 -- 172.21.15.181:0/2284620083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00cf030 con 0x7f6ca00c85e0 2020-11-16T22:34:09.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.766+0000 7f6ca5c56700 1 --2- 172.21.15.181:0/2284620083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 0x7f6ca0160810 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f6c9000b9a0 tx=0x7f6c90006df0).ready entity=mon.2 client_cookie=9a857d7ef9d0adc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:09.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca5455700 1 -- 172.21.15.181:0/2284620083 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900097b0 con 0x7f6ca00c85e0 2020-11-16T22:34:09.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca5455700 1 -- 172.21.15.181:0/2284620083 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c90009910 con 0x7f6ca00c85e0 2020-11-16T22:34:09.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca5455700 1 -- 172.21.15.181:0/2284620083 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90009c90 con 0x7f6ca00c85e0 2020-11-16T22:34:09.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- 172.21.15.181:0/2284620083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 msgr2=0x7f6ca0160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:09.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/2284620083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 0x7f6ca0160810 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f6c9000b9a0 tx=0x7f6c90006df0).stop 2020-11-16T22:34:09.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- 172.21.15.181:0/2284620083 shutdown_connections 2020-11-16T22:34:09.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/2284620083 >> v2:172.21.15.175:3300/0 conn(0x7f6ca0160430 0x7f6ca0161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/2284620083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca00c85e0 0x7f6ca0160810 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:09.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- 172.21.15.181:0/2284620083 >> 172.21.15.181:0/2284620083 conn(0x7f6ca00d3680 msgr2=0x7f6ca00d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:09.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- 172.21.15.181:0/2284620083 shutdown_connections 2020-11-16T22:34:09.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- 172.21.15.181:0/2284620083 wait complete. 2020-11-16T22:34:09.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 Processor -- start 2020-11-16T22:34:09.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.770+0000 7f6ca86bb700 1 -- start start 2020-11-16T22:34:09.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca0160430 0x7f6ca0142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:10.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 0x7f6ca0157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:10.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:10.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca015d3c0 con 0x7f6ca0160430 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca016ff30 con 0x7f6ca0162bd0 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01701d0 con 0x7f6ca016fa60 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca5c56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 0x7f6ca0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43014/0 (socket says 172.21.15.181:43014) 2020-11-16T22:34:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca5c56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 0x7f6ca0157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35170/0 (socket says 172.21.15.181:35170) 2020-11-16T22:34:10.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 -- 172.21.15.181:0/1329767625 learned_addr learned my addr 172.21.15.181:0/1329767625 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:10.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6457700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca0160430 0x7f6ca0142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:10.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 -- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 msgr2=0x7f6ca0157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:10.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 0x7f6ca0157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 -- 172.21.15.181:0/1329767625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca0160430 msgr2=0x7f6ca0142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:10.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca0160430 0x7f6ca0142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 -- 172.21.15.181:0/1329767625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0170470 con 0x7f6ca016fa60 2020-11-16T22:34:10.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6c58700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f6c98005920 tx=0x7f6c98007f20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005b80 con 0x7f6ca016fa60 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98005ce0 con 0x7f6ca016fa60 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0170710 con 0x7f6ca016fa60 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98007340 con 0x7f6ca016fa60 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0170f90 con 0x7f6ca016fa60 2020-11-16T22:34:10.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0069eb0 con 0x7f6ca016fa60 2020-11-16T22:34:10.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6c98010030 con 0x7f6ca016fa60 2020-11-16T22:34:10.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 0x7f6c78057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:10.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6c9806dba0 con 0x7f6ca016fa60 2020-11-16T22:34:10.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.774+0000 7f6ca6457700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 0x7f6c78057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:10.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.778+0000 7f6ca6457700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 0x7f6c78057550 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f6c9c006fd0 tx=0x7f6c9c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:10.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.782+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6c98043e00 con 0x7f6ca016fa60 2020-11-16T22:34:10.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ca00b4810 con 0x7f6ca016fa60 2020-11-16T22:34:10.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6c8e7fc700 1 -- 172.21.15.181:0/1329767625 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4631 (secure 0 0 0) 0x7f6c98047420 con 0x7f6ca016fa60 2020-11-16T22:34:10.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 msgr2=0x7f6c78057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:10.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 0x7f6c78057550 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f6c9c006fd0 tx=0x7f6c9c006df0).stop 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 msgr2=0x7f6ca00d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f6c98005920 tx=0x7f6c98007f20).stop 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 shutdown_connections 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6c78055090 0x7f6c78057550 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ca016fa60 0x7f6ca00d00b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca0160430 0x7f6ca0142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 --2- 172.21.15.181:0/1329767625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ca0162bd0 0x7f6ca0157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:10.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.982+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 >> 172.21.15.181:0/1329767625 conn(0x7f6ca00d3680 msgr2=0x7f6ca015e010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.986+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 shutdown_connections 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:09.986+0000 7f6ca86bb700 1 -- 172.21.15.181:0/1329767625 wait complete. 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 96 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7396,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:00.652468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:11.895 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:33:55.623227+0000', 'modified': '2020-11-16T22:34:00.652468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7405}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7405': {'gid': 7405, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6839/1296416553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1296416553}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1296416553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:34:11.896 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:34:11.897 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:34:12.035 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":7405,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.407198299000001,"mdsmap_epoch":96,"osdmap_epoch":97,"osdmap_epoch_barrier":97,"uptime":25.839147062999999} 2020-11-16T22:34:12.035 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 7405, "mdsmap_epoch": 96, "osdmap_epoch": 97, "osdmap_epoch_barrier": 97, "rank_uptime": 16.407198299, "state": "up:active", "uptime": 25.839147063, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:34:12.036 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:34:12.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:12.037 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:34:12.053 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:34:12.054 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:34:12.162 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:34:12.162 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:34:12.188 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:34:12.188 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:34:12.194 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:34:12.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:12.195 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:34:12.221 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.366+0000 7fb917fff700 1 Processor -- start 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.366+0000 7fb917fff700 1 -- start start 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb918161860 0x7fb918062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb918062740 0x7fb9180644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9180c8db0 con 0x7fb918161480 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb918165030 con 0x7fb918161860 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180c8330 con 0x7fb918062740 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb918062740 0x7fb9180644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb918062740 0x7fb9180644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35180/0 (socket says 172.21.15.181:35180) 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 -- 172.21.15.181:0/2378820445 learned_addr learned my addr 172.21.15.181:0/2378820445 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9167fc700 1 --2- 172.21.15.181:0/2378820445 >> v2:172.21.15.175:3300/0 conn(0x7fb918161860 0x7fb918062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 -- 172.21.15.181:0/2378820445 >> v2:172.21.15.175:3300/0 conn(0x7fb918161860 msgr2=0x7fb918062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 --2- 172.21.15.181:0/2378820445 >> v2:172.21.15.175:3300/0 conn(0x7fb918161860 0x7fb918062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 -- 172.21.15.181:0/2378820445 >> v1:172.21.15.175:6789/0 conn(0x7fb918161480 legacy=0x7fb918164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 -- 172.21.15.181:0/2378820445 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180c85d0 con 0x7fb918062740 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb9177fe700 1 --2- 172.21.15.181:0/2378820445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9180644e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fb9180b8580 tx=0x7fb910006eb0).ready entity=mon.1 client_cookie=cf83f336819c969a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb915ffb700 1 -- 172.21.15.181:0/2378820445 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100098d0 con 0x7fb918062740 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb915ffb700 1 -- 172.21.15.181:0/2378820445 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910009a30 con 0x7fb918062740 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb915ffb700 1 -- 172.21.15.181:0/2378820445 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910009db0 con 0x7fb918062740 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- 172.21.15.181:0/2378820445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 msgr2=0x7fb9180644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 --2- 172.21.15.181:0/2378820445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9180644e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fb9180b8580 tx=0x7fb910006eb0).stop 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- 172.21.15.181:0/2378820445 shutdown_connections 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 --2- 172.21.15.181:0/2378820445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9180644e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 --2- 172.21.15.181:0/2378820445 >> v2:172.21.15.175:3300/0 conn(0x7fb918161860 0x7fb918062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- 172.21.15.181:0/2378820445 >> 172.21.15.181:0/2378820445 conn(0x7fb9180d2dc0 msgr2=0x7fb9180ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- 172.21.15.181:0/2378820445 shutdown_connections 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.370+0000 7fb917fff700 1 -- 172.21.15.181:0/2378820445 wait complete. 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 Processor -- start 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- start start 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9181654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 0x7fb9180cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918145a10 con 0x7fb918161480 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918174be0 con 0x7fb918062740 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb918167c50 con 0x7fb918161860 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43024/0 (socket says 172.21.15.181:43024) 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 -- 172.21.15.181:0/3267485174 learned_addr learned my addr 172.21.15.181:0/3267485174 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb916ffd700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9181654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9167fc700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 0x7fb9180cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 -- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 msgr2=0x7fb9181654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9181654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 -- 172.21.15.181:0/3267485174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 msgr2=0x7fb9180cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 0x7fb9180cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 -- 172.21.15.181:0/3267485174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918167ef0 con 0x7fb918161860 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9167fc700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 0x7fb9180cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb9177fe700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fb910006fd0 tx=0x7fb910015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91001d5d0 con 0x7fb918161860 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91001d730 con 0x7fb918161860 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91001dad0 con 0x7fb918161860 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb918168190 con 0x7fb918161860 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9181659e0 con 0x7fb918161860 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.374+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9181684e0 con 0x7fb918161860 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.378+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb910025030 con 0x7fb918161860 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.378+0000 7fb90effd700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 0x7fb8e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.378+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb91006fd20 con 0x7fb918161860 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.378+0000 7fb916ffd700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 0x7fb8e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.378+0000 7fb916ffd700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 0x7fb8e8057520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fb9180c91e0 tx=0x7fb908007b50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.382+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb9100460b0 con 0x7fb918161860 2020-11-16T22:34:12.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.582+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fb918044f60 con 0x7fb918161860 2020-11-16T22:34:12.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.914+0000 7fb90effd700 1 -- 172.21.15.181:0/3267485174 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v97) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb910023030 con 0x7fb918161860 2020-11-16T22:34:12.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 msgr2=0x7fb8e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 0x7fb8e8057520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fb9180c91e0 tx=0x7fb908007b50).stop 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 msgr2=0x7fb9180600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fb910006fd0 tx=0x7fb910015000).stop 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 shutdown_connections 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8e8055060 0x7fb8e8057520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb918161860 0x7fb9180600f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb918062740 0x7fb9181654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 --2- 172.21.15.181:0/3267485174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918161480 0x7fb9180cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:12.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 >> 172.21.15.181:0/3267485174 conn(0x7fb9180d2dc0 msgr2=0x7fb918147400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:12.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 shutdown_connections 2020-11-16T22:34:12.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:12.918+0000 7fb917fff700 1 -- 172.21.15.181:0/3267485174 wait complete. 2020-11-16T22:34:12.936 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 Processor -- start 2020-11-16T22:34:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 -- start start 2020-11-16T22:34:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f638c15f3c0 0x7f638c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c0cccd0 con 0x7f638c061810 2020-11-16T22:34:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c0ccf70 con 0x7f638c156a90 2020-11-16T22:34:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638b59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c15e030 con 0x7f638c15f3c0 2020-11-16T22:34:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f638c156a90 0x7f638c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49502/0 (socket says 172.21.15.181:49502) 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638a59c700 1 -- 172.21.15.181:0/3103039277 learned_addr learned my addr 172.21.15.181:0/3103039277 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.062+0000 7f638ad9d700 1 --2- 172.21.15.181:0/3103039277 >> v2:172.21.15.175:3300/0 conn(0x7f638c15f3c0 0x7f638c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749236952 0 0) 0x7f638c0ccf70 con 0x7f638c156a90 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6370003240 con 0x7f638c156a90 2020-11-16T22:34:13.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3958824583 0 0) 0x7f6370003240 con 0x7f638c156a90 2020-11-16T22:34:13.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 >> v2:172.21.15.175:3300/0 conn(0x7f638c15f3c0 msgr2=0x7f638c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 --2- 172.21.15.181:0/3103039277 >> v2:172.21.15.175:3300/0 conn(0x7f638c15f3c0 0x7f638c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 >> v1:172.21.15.175:6789/0 conn(0x7f638c061810 legacy=0x7f638c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c15e2d0 con 0x7f638c156a90 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f637c0014e0 con 0x7f638c156a90 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 >> v1:172.21.15.181:6789/0 conn(0x7f638c156a90 legacy=0x7f638c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 0x7f6370003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63700044e0 0x7f6370004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370004e50 0x7f637000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c15e030 con 0x7f63700044e0 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370004260 con 0x7f6370003730 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637000bc20 con 0x7f6370004e50 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 0x7f6370003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638ad9d700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370004e50 0x7f637000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f6389d9b700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63700044e0 0x7f6370004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 -- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370004e50 msgr2=0x7f637000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370004e50 0x7f637000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 -- 172.21.15.181:0/3103039277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63700044e0 msgr2=0x7f6370004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63700044e0 0x7f6370004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 -- 172.21.15.181:0/3103039277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370003240 con 0x7f6370003730 2020-11-16T22:34:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638ad9d700 1 -- 172.21.15.181:0/3103039277 reap_dead start 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f6389d9b700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63700044e0 0x7f6370004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638a59c700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 0x7f6370003b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f637c00c470 tx=0x7f637c0077d0).ready entity=mon.1 client_cookie=11e5ac8133aa344c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0070d0 con 0x7f6370003730 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c007230 con 0x7f6370003730 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638959a700 1 -- 172.21.15.181:0/3103039277 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0055f0 con 0x7f6370003730 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 msgr2=0x7f6370003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 0x7f6370003b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f637c00c470 tx=0x7f637c0077d0).stop 2020-11-16T22:34:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- 172.21.15.181:0/3103039277 shutdown_connections 2020-11-16T22:34:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 --2- 172.21.15.181:0/3103039277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6370003730 0x7f6370003b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- 172.21.15.181:0/3103039277 >> 172.21.15.181:0/3103039277 conn(0x7f638c0cf630 msgr2=0x7f638c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- 172.21.15.181:0/3103039277 shutdown_connections 2020-11-16T22:34:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- 172.21.15.181:0/3103039277 wait complete. 2020-11-16T22:34:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 Processor -- start 2020-11-16T22:34:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.066+0000 7f638b59e700 1 -- start start 2020-11-16T22:34:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c0cc640 0x7f638c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f638c1606b0 0x7f638c172b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c172700 con 0x7f638c1606b0 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0cd6b0 con 0x7f638c0cc640 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0cd950 con 0x7f6370003730 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43052/0 (socket says 172.21.15.181:43052) 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 -- 172.21.15.181:0/1267967155 learned_addr learned my addr 172.21.15.181:0/1267967155 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f6389d9b700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c0cc640 0x7f638c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638ad9d700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f638c1606b0 0x7f638c172b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 -- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c0cc640 msgr2=0x7f638c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c0cc640 0x7f638c1601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 -- 172.21.15.181:0/1267967155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f638c1606b0 msgr2=0x7f638c172b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f638c1606b0 0x7f638c172b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 -- 172.21.15.181:0/1267967155 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0cdbf0 con 0x7f6370003730 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638a59c700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f637c000bc0 tx=0x7f637c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0016c0 con 0x7f6370003730 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c001820 con 0x7f6370003730 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c0cde90 con 0x7f6370003730 2020-11-16T22:34:13.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c013660 con 0x7f6370003730 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.070+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c14eae0 con 0x7f6370003730 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c160a90 con 0x7f6370003730 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f637c01d070 con 0x7f6370003730 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f637a7fc700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 0x7f6360057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f6389d9b700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 0x7f6360057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f637c06ecd0 con 0x7f6370003730 2020-11-16T22:34:13.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.074+0000 7f6389d9b700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 0x7f6360057520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f638c0cb120 tx=0x7f6380009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:13.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.082+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f637c048e60 con 0x7f6370003730 2020-11-16T22:34:13.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.282+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f638c044f60 con 0x7f6370003730 2020-11-16T22:34:13.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.282+0000 7f637a7fc700 1 -- 172.21.15.181:0/1267967155 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4647 (secure 0 0 0) 0x7f637c01a030 con 0x7f6370003730 2020-11-16T22:34:13.275 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:13.275 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7396,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:12.912600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7405,"mds_1":7399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:creating","state_seq":2,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:13.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 msgr2=0x7f6360057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 0x7f6360057520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f638c0cb120 tx=0x7f6380009040).stop 2020-11-16T22:34:13.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 msgr2=0x7f638c0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:13.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f637c000bc0 tx=0x7f637c015000).stop 2020-11-16T22:34:13.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 shutdown_connections 2020-11-16T22:34:13.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6360055060 0x7f6360057520 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f638c1606b0 0x7f638c172b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6370003730 0x7f638c0cc150 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 --2- 172.21.15.181:0/1267967155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c0cc640 0x7f638c1601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 >> 172.21.15.181:0/1267967155 conn(0x7f638c0cf630 msgr2=0x7f638c15ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 shutdown_connections 2020-11-16T22:34:13.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:13.286+0000 7f638b59e700 1 -- 172.21.15.181:0/1267967155 wait complete. 2020-11-16T22:34:13.279 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 98 2020-11-16T22:34:14.196 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:33:55.623227+0000', 'modified': '2020-11-16T22:34:12.912600+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7405, 'mds_1': 7399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7399': {'gid': 7399, 'name': 'b', 'rank': 1, 'incarnation': 98, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.15.181:6837/1406664890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1406664890}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1406664890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7405': {'gid': 7405, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6839/1296416553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1296416553}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1296416553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:34:14.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7399:up:creating 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:15.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.338+0000 7f4197748700 1 Processor -- start 2020-11-16T22:34:15.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.338+0000 7f4197748700 1 -- start start 2020-11-16T22:34:15.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.338+0000 7f4197748700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4190162f20 0x7f4190163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4190163a40 0x7f419015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41901633a0 con 0x7f4190163a40 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4190161870 con 0x7f4190162f20 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4190161b10 con 0x7f41901428e0 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4190162f20 0x7f4190163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f41954e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4190162f20 0x7f4190163550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35224/0 (socket says 172.21.15.181:35224) 2020-11-16T22:34:15.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f41954e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43070/0 (socket says 172.21.15.181:43070) 2020-11-16T22:34:15.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 -- 172.21.15.181:0/1507178863 learned_addr learned my addr 172.21.15.181:0/1507178863 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:15.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4195ce5700 1 --2- 172.21.15.181:0/1507178863 >> v2:172.21.15.175:3300/0 conn(0x7f4190163a40 0x7f419015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 -- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 msgr2=0x7f4190162a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:15.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 -- 172.21.15.181:0/1507178863 >> v2:172.21.15.175:3300/0 conn(0x7f4190163a40 msgr2=0x7f419015ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1507178863 >> v2:172.21.15.175:3300/0 conn(0x7f4190163a40 0x7f419015ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 -- 172.21.15.181:0/1507178863 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190161db0 con 0x7f4190162f20 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f41954e4700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190162f20 0x7f4190163550 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f419005ce10 tx=0x7f4184006eb0).ready entity=mon.1 client_cookie=15053049bb362a9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4183fff700 1 -- 172.21.15.181:0/1507178863 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840097b0 con 0x7f4190162f20 2020-11-16T22:34:15.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4183fff700 1 -- 172.21.15.181:0/1507178863 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4184009910 con 0x7f4190162f20 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4183fff700 1 -- 172.21.15.181:0/1507178863 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184009c90 con 0x7f4190162f20 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190162f20 msgr2=0x7f4190163550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190162f20 0x7f4190163550 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f419005ce10 tx=0x7f4184006eb0).stop 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- 172.21.15.181:0/1507178863 shutdown_connections 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- 172.21.15.181:0/1507178863 >> v2:172.21.15.175:3300/0 conn(0x7f4190163a40 0x7f419015ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41901428e0 0x7f4190162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 --2- 172.21.15.181:0/1507178863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190162f20 0x7f4190163550 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- 172.21.15.181:0/1507178863 >> 172.21.15.181:0/1507178863 conn(0x7f4190060bc0 msgr2=0x7f41900d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- 172.21.15.181:0/1507178863 shutdown_connections 2020-11-16T22:34:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.342+0000 7f4197748700 1 -- 172.21.15.181:0/1507178863 wait complete. 2020-11-16T22:34:15.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 Processor -- start 2020-11-16T22:34:15.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- start start 2020-11-16T22:34:15.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41901428e0 0x7f41900cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 0x7f41900b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419013ea60 con 0x7f41901428e0 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900cd820 con 0x7f4190163a40 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900cdac0 con 0x7f4190162f20 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4195ce5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 0x7f41900b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43074/0 (socket says 172.21.15.181:43074) 2020-11-16T22:34:15.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 -- 172.21.15.181:0/1298437842 learned_addr learned my addr 172.21.15.181:0/1298437842 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:15.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f41954e4700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41901428e0 0x7f41900cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 -- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 msgr2=0x7f41900b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 0x7f41900b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 -- 172.21.15.181:0/1298437842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41901428e0 msgr2=0x7f41900cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41901428e0 0x7f41900cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 -- 172.21.15.181:0/1298437842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900cdd60 con 0x7f4190162f20 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4195ce5700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 0x7f41900b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4194ce3700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f41840094b0 tx=0x7f4184015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184011c80 con 0x7f4190162f20 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4184011de0 con 0x7f4190162f20 2020-11-16T22:34:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419015b920 con 0x7f4190162f20 2020-11-16T22:34:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418401e670 con 0x7f4190162f20 2020-11-16T22:34:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.346+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419015a100 con 0x7f4190162f20 2020-11-16T22:34:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419005d2e0 con 0x7f4190162f20 2020-11-16T22:34:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f418401a040 con 0x7f4190162f20 2020-11-16T22:34:15.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f41817fa700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 0x7f4168057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:15.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f418406ef60 con 0x7f4190162f20 2020-11-16T22:34:15.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f41954e4700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 0x7f4168057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:15.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.350+0000 7f41954e4700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 0x7f4168057520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f418c006fd0 tx=0x7f418c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:15.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.358+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f419005d2e0 con 0x7f4190162f20 2020-11-16T22:34:15.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.558+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f419005d2e0 con 0x7f4190162f20 2020-11-16T22:34:15.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.558+0000 7f41817fa700 1 -- 172.21.15.181:0/1298437842 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4662 (secure 0 0 0) 0x7f419005d2e0 con 0x7f4190162f20 2020-11-16T22:34:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 msgr2=0x7f4168057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 0x7f4168057520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f418c006fd0 tx=0x7f418c006df0).stop 2020-11-16T22:34:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 msgr2=0x7f41900b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f41840094b0 tx=0x7f4184015000).stop 2020-11-16T22:34:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 shutdown_connections 2020-11-16T22:34:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4168055060 0x7f4168057520 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4190163a40 0x7f41900b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:16.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41901428e0 0x7f41900cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:16.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 --2- 172.21.15.181:0/1298437842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4190162f20 0x7f41900b5c60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:16.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 >> 172.21.15.181:0/1298437842 conn(0x7f4190060bc0 msgr2=0x7f4190142150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:16.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 shutdown_connections 2020-11-16T22:34:16.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:15.562+0000 7f4197748700 1 -- 172.21.15.181:0/1298437842 wait complete. 2020-11-16T22:34:16.160 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 101 2020-11-16T22:34:16.160 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:16.160 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:14.931501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:16.161 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:34:15.778+0000 7f5755002700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:34:16.162 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:34:15.778+0000 7f5755002700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:34:17.068 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:33:55.623227+0000', 'modified': '2020-11-16T22:34:14.931501+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7405, 'mds_1': 7399, 'mds_2': 7396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7396': {'gid': 7396, 'name': 'i', 'rank': 2, 'incarnation': 100, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6839/3389989681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6837', 'nonce': 3389989681}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3389989681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7399': {'gid': 7399, 'name': 'b', 'rank': 1, 'incarnation': 98, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6837/1406664890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1406664890}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1406664890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7405': {'gid': 7405, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6839/1296416553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1296416553}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1296416553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:34:17.068 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:34:17.069 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:34:17.210 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":7396,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.2775722200000001,"mdsmap_epoch":101,"osdmap_epoch":97,"osdmap_epoch_barrier":97,"uptime":31.024284022} 2020-11-16T22:34:17.211 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 7396, "mdsmap_epoch": 101, "osdmap_epoch": 97, "osdmap_epoch_barrier": 97, "rank_uptime": 3.27757222, "state": "up:active", "uptime": 31.024284022, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:34:17.211 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:34:17.347 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":7399,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.4256800370000002,"mdsmap_epoch":101,"osdmap_epoch":97,"osdmap_epoch_barrier":97,"uptime":31.160654569999998} 2020-11-16T22:34:17.348 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 7399, "mdsmap_epoch": 101, "osdmap_epoch": 97, "osdmap_epoch_barrier": 97, "rank_uptime": 4.425680037, "state": "up:active", "uptime": 31.16065457, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:34:17.349 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:34:17.473 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":7405,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.844313352,"mdsmap_epoch":101,"osdmap_epoch":97,"osdmap_epoch_barrier":97,"uptime":31.276260842999999} 2020-11-16T22:34:17.473 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 7405, "mdsmap_epoch": 101, "osdmap_epoch": 97, "osdmap_epoch_barrier": 97, "rank_uptime": 21.844313352, "state": "up:active", "uptime": 31.276260843, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:34:17.474 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:17.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 Processor -- start 2020-11-16T22:34:17.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 -- start start 2020-11-16T22:34:17.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:17.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac15d8d0 con 0x7fc8ac15fec0 2020-11-16T22:34:17.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac0ce070 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b4081700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac0ce310 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:17.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35240/0 (socket says 172.21.15.181:35240) 2020-11-16T22:34:17.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b261e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc8ac0c85a0 0x7fc8ac15dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49546/0 (socket says 172.21.15.181:49546) 2020-11-16T22:34:17.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 -- 172.21.15.181:0/353550981 learned_addr learned my addr 172.21.15.181:0/353550981 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:17.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 -- 172.21.15.181:0/353550981 >> v1:172.21.15.181:6789/0 conn(0x7fc8ac0c85a0 legacy=0x7fc8ac15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 -- 172.21.15.181:0/353550981 >> v1:172.21.15.175:6789/0 conn(0x7fc8ac15fec0 legacy=0x7fc8ac0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.606+0000 7fc8b1e1d700 1 -- 172.21.15.181:0/353550981 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac0ce5b0 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b1e1d700 1 --2- 172.21.15.181:0/353550981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc89c009d50 tx=0x7fc89c009290).ready entity=mon.1 client_cookie=fe843e253e0afe8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b0e1b700 1 -- 172.21.15.181:0/353550981 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00bd10 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b0e1b700 1 -- 172.21.15.181:0/353550981 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c00be70 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b0e1b700 1 -- 172.21.15.181:0/353550981 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c010620 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- 172.21.15.181:0/353550981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac0c7ab0 msgr2=0x7fc8ac15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 --2- 172.21.15.181:0/353550981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc89c009d50 tx=0x7fc89c009290).stop 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- 172.21.15.181:0/353550981 shutdown_connections 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 --2- 172.21.15.181:0/353550981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac15f9d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- 172.21.15.181:0/353550981 >> 172.21.15.181:0/353550981 conn(0x7fc8ac060bc0 msgr2=0x7fc8ac0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- 172.21.15.181:0/353550981 shutdown_connections 2020-11-16T22:34:17.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- 172.21.15.181:0/353550981 wait complete. 2020-11-16T22:34:17.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 Processor -- start 2020-11-16T22:34:17.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.610+0000 7fc8b4081700 1 -- start start 2020-11-16T22:34:17.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:17.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:17.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac15fec0 0x7fc8ac0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac16faf0 con 0x7fc8ac0c7ab0 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac16fd90 con 0x7fc8ac15fec0 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac170030 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43088/0 (socket says 172.21.15.181:43088) 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 -- 172.21.15.181:0/810123475 learned_addr learned my addr 172.21.15.181:0/810123475 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:17.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b261e700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac15fec0 0x7fc8ac0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:17.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b1e1d700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 -- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac15fec0 msgr2=0x7fc8ac0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac15fec0 0x7fc8ac0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 -- 172.21.15.181:0/810123475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ac0c7ab0 msgr2=0x7fc8ac0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac0cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 -- 172.21.15.181:0/810123475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac1702d0 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b161c700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc8a000c6e0 tx=0x7fc8a0007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0007020 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0007180 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00064f0 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac170570 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac0cca20 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.614+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac069eb0 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.618+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc8a0010070 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.618+0000 7fc8a9ffb700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 0x7fc884057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:17.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.618+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc8a006cfe0 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.618+0000 7fc8b1e1d700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 0x7fc884057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:17.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.618+0000 7fc8b1e1d700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 0x7fc884057550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fc89c0092d0 tx=0x7fc89c017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:17.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.622+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc8a006aef0 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.822+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8ac061100 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.822+0000 7fc8a9ffb700 1 -- 172.21.15.181:0/810123475 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4662 (secure 0 0 0) 0x7fc8a0021250 con 0x7fc8ac0c85a0 2020-11-16T22:34:17.814 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:17.815 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:14.931501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 msgr2=0x7fc884057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 0x7fc884057550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fc89c0092d0 tx=0x7fc89c017000).stop 2020-11-16T22:34:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 msgr2=0x7fc8ac0d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc8a000c6e0 tx=0x7fc8a0007c50).stop 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 shutdown_connections 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc884055090 0x7fc884057550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ac15fec0 0x7fc8ac0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ac0c7ab0 0x7fc8ac0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 --2- 172.21.15.181:0/810123475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8ac0c85a0 0x7fc8ac0d1f20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 >> 172.21.15.181:0/810123475 conn(0x7fc8ac060bc0 msgr2=0x7fc8ac0c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 shutdown_connections 2020-11-16T22:34:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:17.826+0000 7fc8b4081700 1 -- 172.21.15.181:0/810123475 wait complete. 2020-11-16T22:34:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 101 2020-11-16T22:34:17.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:17.830 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:34:17.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:17.893 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:34:21.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:21.210 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:34:21.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:21.251 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:34:21.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:21.289 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:34:21.370 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:21.503 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:34:21.504 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:21.504 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:23.639 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:34:23.640 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:23.641 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:25.642 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:25.783 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:34:25.784 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:25.784 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:27.785 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:27.925 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:34:27.925 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:27.926 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:29.927 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:30.065 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:34:30.066 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:30.067 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:32.068 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:34:32.204 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:34:32.204 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:34:32.333 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:34:32.334 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:32.335 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:34:33.635 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:33.641 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:33.644 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:34:34.336 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:34:34.473 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:34:34.474 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:34.475 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:34:36.476 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:34:36.609 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}}] 2020-11-16T22:34:36.635 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:36.636 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:34:38.637 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:34:38.774 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:34:38.775 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:34:38.775 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:34:38.776 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"15","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:34:38.912 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:34:38.912 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:39.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 Processor -- start 2020-11-16T22:34:39.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 -- start start 2020-11-16T22:34:39.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f17901672c0 0x7f1790167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1790168390 con 0x7f1790167ac0 2020-11-16T22:34:39.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900b2060 con 0x7f17900b3930 2020-11-16T22:34:39.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f1794c60700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900b5f00 con 0x7f17901672c0 2020-11-16T22:34:39.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f17900b3930 0x7f1790166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49556/0 (socket says 172.21.15.181:49556) 2020-11-16T22:34:39.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178dd9b700 1 -- 172.21.15.181:0/2016512183 learned_addr learned my addr 172.21.15.181:0/2016512183 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:39.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178ed9d700 1 --2- 172.21.15.181:0/2016512183 >> v2:172.21.15.175:3300/0 conn(0x7f17901672c0 0x7f1790167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785641536 0 0) 0x7f17900b2060 con 0x7f17900b3930 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1774003240 con 0x7f17900b3930 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959821278 0 0) 0x7f1790168390 con 0x7f1790167ac0 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17900b2060 con 0x7f1790167ac0 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178ed9d700 1 -- 172.21.15.181:0/2016512183 >> v1:172.21.15.181:6789/0 conn(0x7f17900b3930 legacy=0x7f1790166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178ed9d700 1 -- 172.21.15.181:0/2016512183 >> v1:172.21.15.175:6789/0 conn(0x7f1790167ac0 legacy=0x7f17900b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178ed9d700 1 -- 172.21.15.181:0/2016512183 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900b2300 con 0x7f17901672c0 2020-11-16T22:34:39.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178ed9d700 1 --2- 172.21.15.181:0/2016512183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901672c0 0x7f1790167ea0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f178000b9a0 tx=0x7f1780006df0).ready entity=mon.0 client_cookie=dd1ed9c2345ced07 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.058+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780009680 con 0x7f17901672c0 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800097e0 con 0x7f17901672c0 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178d59a700 1 -- 172.21.15.181:0/2016512183 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780009b60 con 0x7f17901672c0 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/2016512183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901672c0 msgr2=0x7f1790167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 --2- 172.21.15.181:0/2016512183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901672c0 0x7f1790167ea0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f178000b9a0 tx=0x7f1780006df0).stop 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/2016512183 shutdown_connections 2020-11-16T22:34:39.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 --2- 172.21.15.181:0/2016512183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17901672c0 0x7f1790167ea0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/2016512183 >> 172.21.15.181:0/2016512183 conn(0x7f1790060bc0 msgr2=0x7f17900d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/2016512183 shutdown_connections 2020-11-16T22:34:39.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/2016512183 wait complete. 2020-11-16T22:34:39.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 Processor -- start 2020-11-16T22:34:39.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- start start 2020-11-16T22:34:39.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 0x7f1790162d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 0x7f1790175e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790174aa0 con 0x7f1790167ac0 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900d1090 con 0x7f17900b3930 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1790174cd0 con 0x7f17901672c0 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43100/0 (socket says 172.21.15.181:43100) 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 -- 172.21.15.181:0/3233782954 learned_addr learned my addr 172.21.15.181:0/3233782954 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178e59c700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 0x7f1790162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178ed9d700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 0x7f1790175e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 -- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 msgr2=0x7f1790162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 0x7f1790162d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 -- 172.21.15.181:0/3233782954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 msgr2=0x7f1790175e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 0x7f1790175e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 -- 172.21.15.181:0/3233782954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790174f70 con 0x7f17901672c0 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178ed9d700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 0x7f1790175e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:39.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178e59c700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 0x7f1790162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f178dd9b700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f1778007fd0 tx=0x7f1778007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778007320 con 0x7f17901672c0 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1778007480 con 0x7f17901672c0 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790175210 con 0x7f17901672c0 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778005800 con 0x7f17901672c0 2020-11-16T22:34:39.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.062+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1790175750 con 0x7f17901672c0 2020-11-16T22:34:39.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1778010030 con 0x7f17901672c0 2020-11-16T22:34:39.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f17867fc700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 0x7f1760057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179005d2e0 con 0x7f17901672c0 2020-11-16T22:34:39.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f178e59c700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 0x7f1760057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f177806dae0 con 0x7f17901672c0 2020-11-16T22:34:39.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.066+0000 7f178e59c700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 0x7f1760057550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f1788000f40 tx=0x7f1788009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.074+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f179005d2e0 con 0x7f17901672c0 2020-11-16T22:34:39.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.266+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f179005d2e0 con 0x7f17901672c0 2020-11-16T22:34:39.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f17867fc700 1 -- 172.21.15.181:0/3233782954 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4666 (secure 0 0 0) 0x7f179005d2e0 con 0x7f17901672c0 2020-11-16T22:34:39.259 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:39.260 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:36.781890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:39.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 msgr2=0x7f1760057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 0x7f1760057550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f1788000f40 tx=0x7f1788009040).stop 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 msgr2=0x7f1790163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f1778007fd0 tx=0x7f1778007b70).stop 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 shutdown_connections 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1760055090 0x7f1760057550 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790167ac0 0x7f1790175e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900b3930 0x7f1790162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 --2- 172.21.15.181:0/3233782954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17901672c0 0x7f1790163280 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 >> 172.21.15.181:0/3233782954 conn(0x7f1790060bc0 msgr2=0x7f1790164520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 shutdown_connections 2020-11-16T22:34:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.270+0000 7f1794c60700 1 -- 172.21.15.181:0/3233782954 wait complete. 2020-11-16T22:34:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 103 2020-11-16T22:34:39.274 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d flush journal 2020-11-16T22:34:39.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.406+0000 7f75afddc700 1 Processor -- start 2020-11-16T22:34:39.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.406+0000 7f75afddc700 1 -- start start 2020-11-16T22:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 0x7f75a80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75a80615f0 0x7f75a80d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a8160e40 con 0x7f75a80622a0 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a81609a0 con 0x7f75a80615f0 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8160ba0 con 0x7f75a815f5b0 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 0x7f75a80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75ae379700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f75a80622a0 0x7f75a8062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49570/0 (socket says 172.21.15.181:49570) 2020-11-16T22:34:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75ae379700 1 -- 172.21.15.181:0/3365332077 learned_addr learned my addr 172.21.15.181:0/3365332077 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75ad377700 1 --2- 172.21.15.181:0/3365332077 >> v2:172.21.15.175:3300/0 conn(0x7f75a80615f0 0x7f75a80d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75acb76700 1 -- 172.21.15.181:0/3365332077 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889786010 0 0) 0x7f75a8160e40 con 0x7f75a80622a0 2020-11-16T22:34:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75acb76700 1 -- 172.21.15.181:0/3365332077 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7590003240 con 0x7f75a80622a0 2020-11-16T22:34:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 -- 172.21.15.181:0/3365332077 >> v2:172.21.15.175:3300/0 conn(0x7f75a80615f0 msgr2=0x7f75a80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 --2- 172.21.15.181:0/3365332077 >> v2:172.21.15.175:3300/0 conn(0x7f75a80615f0 0x7f75a80d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 -- 172.21.15.181:0/3365332077 >> v1:172.21.15.181:6789/0 conn(0x7f75a80622a0 legacy=0x7f75a8062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 -- 172.21.15.181:0/3365332077 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8060970 con 0x7f75a815f5b0 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75adb78700 1 --2- 172.21.15.181:0/3365332077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 0x7f75a80d1340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f759800b990 tx=0x7f7598006df0).ready entity=mon.2 client_cookie=f7a66618bc5c1e64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75acb76700 1 -- 172.21.15.181:0/3365332077 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598009640 con 0x7f75a815f5b0 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75acb76700 1 -- 172.21.15.181:0/3365332077 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75980097a0 con 0x7f75a815f5b0 2020-11-16T22:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75acb76700 1 -- 172.21.15.181:0/3365332077 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598009b20 con 0x7f75a815f5b0 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- 172.21.15.181:0/3365332077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 msgr2=0x7f75a80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 --2- 172.21.15.181:0/3365332077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 0x7f75a80d1340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f759800b990 tx=0x7f7598006df0).stop 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- 172.21.15.181:0/3365332077 shutdown_connections 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 --2- 172.21.15.181:0/3365332077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a815f5b0 0x7f75a80d1340 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 --2- 172.21.15.181:0/3365332077 >> v2:172.21.15.175:3300/0 conn(0x7f75a80615f0 0x7f75a80d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.410+0000 7f75afddc700 1 -- 172.21.15.181:0/3365332077 >> 172.21.15.181:0/3365332077 conn(0x7f75a80d69a0 msgr2=0x7f75a80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- 172.21.15.181:0/3365332077 shutdown_connections 2020-11-16T22:34:39.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- 172.21.15.181:0/3365332077 wait complete. 2020-11-16T22:34:39.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 Processor -- start 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- start start 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 0x7f75a814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 0x7f75a8161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8161c10 con 0x7f75a80615f0 2020-11-16T22:34:39.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80b9ca0 con 0x7f75a815f5b0 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75afddc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80b9f40 con 0x7f75a80622a0 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75ad377700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75ae379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 0x7f75a8161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75ad377700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43114/0 (socket says 172.21.15.181:43114) 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75ad377700 1 -- 172.21.15.181:0/146134994 learned_addr learned my addr 172.21.15.181:0/146134994 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.414+0000 7f75adb78700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 0x7f75a814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 -- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 msgr2=0x7f75a8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 0x7f75a8161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 -- 172.21.15.181:0/146134994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 msgr2=0x7f75a814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 0x7f75a814e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 -- 172.21.15.181:0/146134994 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80ba1e0 con 0x7f75a80622a0 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ae379700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 0x7f75a8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75adb78700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 0x7f75a814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:39.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75ad377700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f75a400ebd0 tx=0x7f75a4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 -- 172.21.15.181:0/146134994 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400c8e0 con 0x7f75a80622a0 2020-11-16T22:34:39.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80ba480 con 0x7f75a80622a0 2020-11-16T22:34:39.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 -- 172.21.15.181:0/146134994 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400ca40 con 0x7f75a80622a0 2020-11-16T22:34:39.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 -- 172.21.15.181:0/146134994 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400cdc0 con 0x7f75a80622a0 2020-11-16T22:34:39.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a80bae60 con 0x7f75a80622a0 2020-11-16T22:34:39.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 -- 172.21.15.181:0/146134994 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f75a401a030 con 0x7f75a80622a0 2020-11-16T22:34:39.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 0x7f7580057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f7595ffb700 1 -- 172.21.15.181:0/146134994 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75a406d990 con 0x7f75a80622a0 2020-11-16T22:34:39.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.418+0000 7f75adb78700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 0x7f7580057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.422+0000 7f75adb78700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 0x7f7580057520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f7598006e30 tx=0x7f7598015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.422+0000 7f75be86d740 1 Processor -- start 2020-11-16T22:34:39.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.422+0000 7f75be86d740 1 -- start start 2020-11-16T22:34:39.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cf0bb0 0x1fb4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1fd1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e084b0 con 0x1f77470 2020-11-16T22:34:39.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ebc930 con 0x1cf0bb0 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d7c6f0 con 0x1d5e7b0 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43122/0 (socket says 172.21.15.181:43122) 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 -- 172.21.15.181:0/3461653860 learned_addr learned my addr 172.21.15.181:0/3461653860 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75ad377700 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cf0bb0 0x1fb4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75ae379700 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1fd1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 -- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cf0bb0 msgr2=0x1fb4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cf0bb0 0x1fb4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 -- 172.21.15.181:0/3461653860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 msgr2=0x1fd1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1fd1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 -- 172.21.15.181:0/3461653860 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d775a0 con 0x1d5e7b0 2020-11-16T22:34:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75adb78700 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f75a8160ce0 tx=0x7f759801d000).ready entity=mon.2 client_cookie=4e9738b419bbd73e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f7586ffd700 1 -- 172.21.15.181:0/3461653860 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759802b070 con 0x1d5e7b0 2020-11-16T22:34:39.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f7586ffd700 1 -- 172.21.15.181:0/3461653860 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759801c9f0 con 0x1d5e7b0 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f7586ffd700 1 -- 172.21.15.181:0/3461653860 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759801cd70 con 0x1d5e7b0 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 msgr2=0x1f75cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f75a8160ce0 tx=0x7f759801d000).stop 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- 172.21.15.181:0/3461653860 shutdown_connections 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1fd1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x1f75cd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 --2- 172.21.15.181:0/3461653860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cf0bb0 0x1fb4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- 172.21.15.181:0/3461653860 >> 172.21.15.181:0/3461653860 conn(0x1f2ce40 msgr2=0x1fd1580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- 172.21.15.181:0/3461653860 shutdown_connections 2020-11-16T22:34:39.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.426+0000 7f75be86d740 1 -- 172.21.15.181:0/3461653860 wait complete. 2020-11-16T22:34:39.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 -- start start 2020-11-16T22:34:39.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 Processor -- start 2020-11-16T22:34:39.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cf0bb0 0x1fdbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1f98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ea96e0 con 0x1f77470 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ea9820 con 0x1d5e7b0 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eaaa90 con 0x1cf0bb0 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 10 client.0 ll_register_callbacks cb 0x1e9fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35288/0 (socket says 172.21.15.181:35288) 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 -- 172.21.15.181:0/1300140822 learned_addr learned my addr 172.21.15.181:0/1300140822 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75adb78700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cf0bb0 0x1fdbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ae379700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1f98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 -- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cf0bb0 msgr2=0x1fdbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cf0bb0 0x1fdbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 -- 172.21.15.181:0/1300140822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 msgr2=0x1f98260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1f98260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 -- 172.21.15.181:0/1300140822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e9a280 con 0x1d5e7b0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75ad377700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f75a4043e60 tx=0x7f75a4009000).ready entity=mon.1 client_cookie=e34a40e6b2b07168 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f757bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a40441f0 con 0x1d5e7b0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75a4046ad0 con 0x1d5e7b0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.430+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f71e10 con 0x1d5e7b0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a4046c90 con 0x1d5e7b0 2020-11-16T22:34:39.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 10 client.7582 fetch_fsmap learned FSMap version 103 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x1ccffd0 con 0x1d5e7b0 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f71e10 con 0x1d5e7b0 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 103) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f75a406d3e0 con 0x1d5e7b0 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 10 client.7582 fetch_fsmap finished waiting for FSMap version 103 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 10 client.7582 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 20 client.7582 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f7a400 0x1f9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 4 client.7582 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:39.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1d9ac50 con 0x1f7a400 2020-11-16T22:34:39.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75adb78700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f7a400 0x1f9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 10 client.7582 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:39.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75adb78700 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f7a400 0x1f9e7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b9e893fd40f84baa server_cookie=1f833cd1c40f6cae in_seq=0 out_seq=0 2020-11-16T22:34:39.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 1 -- 172.21.15.181:0/1300140822 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d9ac50 con 0x1f7a400 2020-11-16T22:34:39.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 10 client.7582 handle_command_reply: tid=0 2020-11-16T22:34:39.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 client.7582 shutdown 2020-11-16T22:34:39.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 msgr2=0x1f76a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f75a4043e60 tx=0x7f75a4009000).stop 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 shutdown_connections 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f7a400 0x1f9e7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f77470 0x1f98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f757bfff700 0 client.7582 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cf0bb0 0x1fdbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 --2- 172.21.15.181:0/1300140822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f76a30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 >> 172.21.15.181:0/1300140822 conn(0x1f2ce40 msgr2=0x1fa1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 shutdown_connections 2020-11-16T22:34:39.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 1 -- 172.21.15.181:0/1300140822 wait complete. 2020-11-16T22:34:39.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.434+0000 7f75be86d740 20 client.7582 trim_cache size 0 max 16384 2020-11-16T22:34:39.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.450+0000 7f75be86d740 1 Processor -- start 2020-11-16T22:34:39.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.450+0000 7f75be86d740 1 -- start start 2020-11-16T22:34:39.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e81e40 con 0x1cf0bb0 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fb4700 con 0x1d5e7b0 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f79bf0 con 0x1f2ce40 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75ae379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43136/0 (socket says 172.21.15.181:43136) 2020-11-16T22:34:39.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 -- 172.21.15.181:0/274342832 learned_addr learned my addr 172.21.15.181:0/274342832 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75ad377700 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 -- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 msgr2=0x1f753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 -- 172.21.15.181:0/274342832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 msgr2=0x1f74ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f74ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 -- 172.21.15.181:0/274342832 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740081d0 con 0x1f2ce40 2020-11-16T22:34:39.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75adb78700 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7598029040 tx=0x7f7598038280).ready entity=mon.2 client_cookie=167abd354e4d6aad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75877fe700 1 -- 172.21.15.181:0/274342832 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759802b040 con 0x1f2ce40 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75877fe700 1 -- 172.21.15.181:0/274342832 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598037040 con 0x1f2ce40 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75877fe700 1 -- 172.21.15.181:0/274342832 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759801cd70 con 0x1f2ce40 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 msgr2=0x1f7c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7598029040 tx=0x7f7598038280).stop 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- 172.21.15.181:0/274342832 shutdown_connections 2020-11-16T22:34:39.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d5e7b0 0x1f753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f2ce40 0x1f7c7f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 --2- 172.21.15.181:0/274342832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f74ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- 172.21.15.181:0/274342832 >> 172.21.15.181:0/274342832 conn(0x7f7574002320 msgr2=0x1f98030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- 172.21.15.181:0/274342832 shutdown_connections 2020-11-16T22:34:39.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- 172.21.15.181:0/274342832 wait complete. 2020-11-16T22:34:39.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 -- start start 2020-11-16T22:34:39.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.454+0000 7f75be86d740 1 Processor -- start 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f9ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x2020be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa1a90 con 0x1cf0bb0 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2023290 con 0x1f2ce40 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x20234f0 con 0x1d5e7b0 2020-11-16T22:34:39.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 10 client.0 ll_register_callbacks cb 0x1fb6540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35302/0 (socket says 172.21.15.181:35302) 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 -- 172.21.15.181:0/3074393300 learned_addr learned my addr 172.21.15.181:0/3074393300 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ad377700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x2020be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75adb78700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f9ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 -- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 msgr2=0x2020be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x2020be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 -- 172.21.15.181:0/3074393300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 msgr2=0x1f9ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f9ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 -- 172.21.15.181:0/3074393300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d0e720 con 0x1f2ce40 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75adb78700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f9ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75ae379700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f75a00075f0 tx=0x7f75a000c040).ready entity=mon.1 client_cookie=60a4bcf0bc4f5a65 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:39.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0006e80 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f759801c9f0 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75a0017c90 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a0017e50 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 10 client.7588 fetch_fsmap learned FSMap version 103 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x1d19e00 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0026070 con 0x1f2ce40 2020-11-16T22:34:39.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 103) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f75a001e020 con 0x1f2ce40 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 10 client.7588 fetch_fsmap finished waiting for FSMap version 103 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 10 client.7588 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 20 client.7588 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f77470 0x1f74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 4 client.7588 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1da1f50 con 0x1f77470 2020-11-16T22:34:39.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75adb78700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f77470 0x1f74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f757bfff700 10 client.7588 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:39.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.458+0000 7f75adb78700 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f77470 0x1f74ef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=17c0fa839dfa0e87 server_cookie=e56522340fc03c78 in_seq=0 out_seq=0 2020-11-16T22:34:39.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f757bfff700 1 -- 172.21.15.181:0/3074393300 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1da1f50 con 0x1f77470 2020-11-16T22:34:39.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f757bfff700 10 client.7588 handle_command_reply: tid=0 2020-11-16T22:34:39.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 client.7588 shutdown 2020-11-16T22:34:39.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 msgr2=0x2031650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f75a00075f0 tx=0x7f75a000c040).stop 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 shutdown_connections 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f77470 0x1f74ef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f2ce40 0x2031650 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f757bfff700 0 client.7588 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cf0bb0 0x1f9ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 --2- 172.21.15.181:0/3074393300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d5e7b0 0x2020be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 >> 172.21.15.181:0/3074393300 conn(0x7f7574002320 msgr2=0x1fdcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 shutdown_connections 2020-11-16T22:34:39.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 1 -- 172.21.15.181:0/3074393300 wait complete. 2020-11-16T22:34:39.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75be86d740 20 client.7588 trim_cache size 0 max 16384 2020-11-16T22:34:39.476 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:39.476 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:34:39.476 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:34:39.476 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 msgr2=0x7f7580057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 0x7f7580057520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f7598006e30 tx=0x7f7598015000).stop 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 msgr2=0x7f75a8161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f75a400ebd0 tx=0x7f75a4006df0).stop 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 shutdown_connections 2020-11-16T22:34:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7580055060 0x7f7580057520 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75a815f5b0 0x7f75a8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a80615f0 0x7f75a814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 --2- 172.21.15.181:0/146134994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75a80622a0 0x7f75a8161230 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 >> 172.21.15.181:0/146134994 conn(0x7f75a80d69a0 msgr2=0x7f75a8062720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 shutdown_connections 2020-11-16T22:34:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.482+0000 7f75afddc700 1 -- 172.21.15.181:0/146134994 wait complete. 2020-11-16T22:34:39.488 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.618+0000 7f3c31ad5700 1 Processor -- start 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.618+0000 7f3c31ad5700 1 -- start start 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c31ad5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c31ad5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 0x7f3c2c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c31ad5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2c15d180 con 0x7f3c2c13e870 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c31ad5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2c15d810 con 0x7f3c2c0615f0 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c31ad5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c15dab0 con 0x7f3c2c13ec50 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c312d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 0x7f3c2c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c312d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 0x7f3c2c15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35312/0 (socket says 172.21.15.181:35312) 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c312d4700 1 -- 172.21.15.181:0/792862766 learned_addr learned my addr 172.21.15.181:0/792862766 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 -- 172.21.15.181:0/792862766 >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 msgr2=0x7f3c2c15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 --2- 172.21.15.181:0/792862766 >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 0x7f3c2c15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 -- 172.21.15.181:0/792862766 >> v1:172.21.15.175:6789/0 conn(0x7f3c2c13e870 legacy=0x7f3c2c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 -- 172.21.15.181:0/792862766 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c15dd50 con 0x7f3c2c13ec50 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2bfff700 1 --2- 172.21.15.181:0/792862766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c061100 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f3c1400b9a0 tx=0x7f3c14006df0).ready entity=mon.2 client_cookie=7e6e02d6d745f69 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2b7fe700 1 -- 172.21.15.181:0/792862766 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140097b0 con 0x7f3c2c13ec50 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2b7fe700 1 -- 172.21.15.181:0/792862766 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c14009910 con 0x7f3c2c13ec50 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.622+0000 7f3c2b7fe700 1 -- 172.21.15.181:0/792862766 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14009c90 con 0x7f3c2c13ec50 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 -- 172.21.15.181:0/792862766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 msgr2=0x7f3c2c061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/792862766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c061100 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f3c1400b9a0 tx=0x7f3c14006df0).stop 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 -- 172.21.15.181:0/792862766 shutdown_connections 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/792862766 >> v2:172.21.15.181:3300/0 conn(0x7f3c2c0615f0 0x7f3c2c15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/792862766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c061100 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 -- 172.21.15.181:0/792862766 >> 172.21.15.181:0/792862766 conn(0x7f3c2c0cf630 msgr2=0x7f3c2c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 -- 172.21.15.181:0/792862766 shutdown_connections 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.626+0000 7f3c31ad5700 1 -- 172.21.15.181:0/792862766 wait complete. 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 Processor -- start 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 -- start start 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2c13e870 0x7f3c2c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c0b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c0b8400 con 0x7f3c2c13e870 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c0cb000 con 0x7f3c2c0615f0 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c0cb2a0 con 0x7f3c2c13ec50 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c312d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c0b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c312d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c0b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43160/0 (socket says 172.21.15.181:43160) 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c312d4700 1 -- 172.21.15.181:0/485853803 learned_addr learned my addr 172.21.15.181:0/485853803 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35314/0 (socket says 172.21.15.181:35314) 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c2bfff700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2c13e870 0x7f3c2c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 -- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 msgr2=0x7f3c2c0b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c0b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 -- 172.21.15.181:0/485853803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2c13e870 msgr2=0x7f3c2c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2c13e870 0x7f3c2c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 -- 172.21.15.181:0/485853803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0cb540 con 0x7f3c2c0615f0 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c30ad3700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f3c1c00c710 tx=0x7f3c1c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c007020 con 0x7f3c2c0615f0 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c007180 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c0cb7e0 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.630+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0064f0 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c160550 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3c1c01a060 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c0c9b00 con 0x7f3c2c0615f0 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c28ff9700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 0x7f3c08057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c1c06d370 con 0x7f3c2c0615f0 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c2bfff700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 0x7f3c08057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.634+0000 7f3c2bfff700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 0x7f3c08057550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f3c14006e30 tx=0x7f3c14015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.638+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3c2c0c9b00 con 0x7f3c2c0615f0 2020-11-16T22:34:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.842+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c2c0cec80 con 0x7f3c2c0615f0 2020-11-16T22:34:39.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.842+0000 7f3c28ff9700 1 -- 172.21.15.181:0/485853803 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4666 (secure 0 0 0) 0x7f3c2c0cec80 con 0x7f3c2c0615f0 2020-11-16T22:34:39.834 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:39.834 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:36.781890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 msgr2=0x7f3c08057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 0x7f3c08057550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f3c14006e30 tx=0x7f3c14015000).stop 2020-11-16T22:34:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 msgr2=0x7f3c2c0d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f3c1c00c710 tx=0x7f3c1c007b90).stop 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 shutdown_connections 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3c08055090 0x7f3c08057550 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c2c13ec50 0x7f3c2c0b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c0615f0 0x7f3c2c0d2390 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 --2- 172.21.15.181:0/485853803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c2c13e870 0x7f3c2c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 >> 172.21.15.181:0/485853803 conn(0x7f3c2c0cf630 msgr2=0x7f3c2c15edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 shutdown_connections 2020-11-16T22:34:39.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.846+0000 7f3c31ad5700 1 -- 172.21.15.181:0/485853803 wait complete. 2020-11-16T22:34:39.838 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 103 2020-11-16T22:34:39.849 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:34:39.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 Processor -- start 2020-11-16T22:34:39.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 -- start start 2020-11-16T22:34:39.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc810166bb0 0x7fc810167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc810168390 con 0x7fc810167ac0 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8100b5f00 con 0x7fc810166bb0 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81638e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100b2060 con 0x7fc8100b3930 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43164/0 (socket says 172.21.15.181:43164) 2020-11-16T22:34:39.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc80f7fe700 1 -- 172.21.15.181:0/1240921820 learned_addr learned my addr 172.21.15.181:0/1240921820 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.982+0000 7fc81492b700 1 --2- 172.21.15.181:0/1240921820 >> v2:172.21.15.175:3300/0 conn(0x7fc810166bb0 0x7fc810167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80f7fe700 1 -- 172.21.15.181:0/1240921820 >> v2:172.21.15.175:3300/0 conn(0x7fc810166bb0 msgr2=0x7fc810167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/1240921820 >> v2:172.21.15.175:3300/0 conn(0x7fc810166bb0 0x7fc810167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80f7fe700 1 -- 172.21.15.181:0/1240921820 >> v1:172.21.15.175:6789/0 conn(0x7fc810167ac0 legacy=0x7fc8100b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80f7fe700 1 -- 172.21.15.181:0/1240921820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100b2300 con 0x7fc8100b3930 2020-11-16T22:34:39.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/1240921820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fc7f8006fd0 tx=0x7fc7f8006eb0).ready entity=mon.2 client_cookie=ac4b8b47f61ed8bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80effd700 1 -- 172.21.15.181:0/1240921820 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80098d0 con 0x7fc8100b3930 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80effd700 1 -- 172.21.15.181:0/1240921820 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8009a30 con 0x7fc8100b3930 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc80effd700 1 -- 172.21.15.181:0/1240921820 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8009db0 con 0x7fc8100b3930 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- 172.21.15.181:0/1240921820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 msgr2=0x7fc810167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 --2- 172.21.15.181:0/1240921820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fc7f8006fd0 tx=0x7fc7f8006eb0).stop 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- 172.21.15.181:0/1240921820 shutdown_connections 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 --2- 172.21.15.181:0/1240921820 >> v2:172.21.15.175:3300/0 conn(0x7fc810166bb0 0x7fc810167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 --2- 172.21.15.181:0/1240921820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8100b3930 0x7fc810167ea0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- 172.21.15.181:0/1240921820 >> 172.21.15.181:0/1240921820 conn(0x7fc810060bc0 msgr2=0x7fc8100d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- 172.21.15.181:0/1240921820 shutdown_connections 2020-11-16T22:34:39.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- 172.21.15.181:0/1240921820 wait complete. 2020-11-16T22:34:39.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 Processor -- start 2020-11-16T22:34:39.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.986+0000 7fc81638e700 1 -- start start 2020-11-16T22:34:39.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc810167ac0 0x7fc81009ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 0x7fc81015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810162300 con 0x7fc810167ac0 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101625a0 con 0x7fc81015f090 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc810175010 con 0x7fc810166bb0 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43170/0 (socket says 172.21.15.181:43170) 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 -- 172.21.15.181:0/3859195278 learned_addr learned my addr 172.21.15.181:0/3859195278 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81492b700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 0x7fc81015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc810167ac0 0x7fc81009ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 -- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 msgr2=0x7fc81015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 0x7fc81015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 -- 172.21.15.181:0/3859195278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc810167ac0 msgr2=0x7fc81009ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc810167ac0 0x7fc81009ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 -- 172.21.15.181:0/3859195278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8101752b0 con 0x7fc810166bb0 2020-11-16T22:34:39.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81492b700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 0x7fc81015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80ffff700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc80000c6e0 tx=0x7fc800007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 -- 172.21.15.181:0/3859195278 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800007020 con 0x7fc810166bb0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 -- 172.21.15.181:0/3859195278 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc800007180 con 0x7fc810166bb0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810175550 con 0x7fc810166bb0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 -- 172.21.15.181:0/3859195278 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000e560 con 0x7fc810166bb0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81015fa00 con 0x7fc810166bb0 2020-11-16T22:34:39.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 -- 172.21.15.181:0/3859195278 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc800018020 con 0x7fc810166bb0 2020-11-16T22:34:39.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 0x7fc7e0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 0x7fc7e0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.990+0000 7fc807fff700 1 -- 172.21.15.181:0/3859195278 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc80006d0d0 con 0x7fc810166bb0 2020-11-16T22:34:39.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 0x7fc7e0057550 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fc8100660e0 tx=0x7fc7f8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc824e1f740 1 Processor -- start 2020-11-16T22:34:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc824e1f740 1 -- start start 2020-11-16T22:34:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x106f7b0 0x1286cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.994+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1288470 0x12e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11194b0 con 0x1288470 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11cd930 con 0x106f7b0 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x108d6f0 con 0x1001bb0 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43180/0 (socket says 172.21.15.181:43180) 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2963230947 learned_addr learned my addr 172.21.15.181:0/2963230947 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80ffff700 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x106f7b0 0x1286cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc81492b700 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1288470 0x12e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x106f7b0 msgr2=0x1286cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x106f7b0 0x1286cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2963230947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1288470 msgr2=0x12e2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1288470 0x12e2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2963230947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10885a0 con 0x1001bb0 2020-11-16T22:34:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc7f8010000 tx=0x7fc7f801e000).ready entity=mon.2 client_cookie=72ccef970dd944a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc8057fa700 1 -- 172.21.15.181:0/2963230947 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800b700 con 0x1001bb0 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc8057fa700 1 -- 172.21.15.181:0/2963230947 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8011c80 con 0x1001bb0 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc8057fa700 1 -- 172.21.15.181:0/2963230947 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f801e460 con 0x1001bb0 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 msgr2=0x12c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc7f8010000 tx=0x7fc7f801e000).stop 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- 172.21.15.181:0/2963230947 shutdown_connections 2020-11-16T22:34:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1288470 0x12e2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x106f7b0 0x1286cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2963230947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1001bb0 0x12c5530 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- 172.21.15.181:0/2963230947 >> 172.21.15.181:0/2963230947 conn(0x123de40 msgr2=0x12e2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- 172.21.15.181:0/2963230947 shutdown_connections 2020-11-16T22:34:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- 172.21.15.181:0/2963230947 wait complete. 2020-11-16T22:34:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 -- start start 2020-11-16T22:34:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:39.998+0000 7fc824e1f740 1 Processor -- start 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x12b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1288470 0x12f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ba6e0 con 0x1001bb0 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ba820 con 0x1288470 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x11bba90 con 0x106f7b0 2020-11-16T22:34:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 10 client.0 ll_register_callbacks cb 0x11b0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43186/0 (socket says 172.21.15.181:43186) 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc81492b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1288470 0x12f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2484619627 learned_addr learned my addr 172.21.15.181:0/2484619627 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80ffff700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x12b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1288470 msgr2=0x12f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1288470 0x12f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2484619627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 msgr2=0x12b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x12b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 -- 172.21.15.181:0/2484619627 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11ab280 con 0x106f7b0 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80ffff700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x12b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fc7f8020040 tx=0x7fc7f8028000).ready entity=mon.2 client_cookie=f4dfe121d8ef4725 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8030070 con 0x106f7b0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7f800b700 con 0x106f7b0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1282e10 con 0x106f7b0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f8006920 con 0x106f7b0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 10 client.6968 fetch_fsmap learned FSMap version 103 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0xfe0fd0 con 0x106f7b0 2020-11-16T22:34:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1282e10 con 0x106f7b0 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 103) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fc7f8006230 con 0x106f7b0 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 10 client.6968 fetch_fsmap finished waiting for FSMap version 103 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 10 client.6968 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 20 client.6968 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x128e0a0 0x12efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 4 client.6968 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x10abc50 con 0x128e0a0 2020-11-16T22:34:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80ffff700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x128e0a0 0x12efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc7fe7fc700 10 client.6968 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:40.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.002+0000 7fc80ffff700 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x128e0a0 0x12efd50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=eac59b21fb8500c8 server_cookie=aa537a25f3a57ce3 in_seq=0 out_seq=0 2020-11-16T22:34:40.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/2484619627 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x10abc50 con 0x128e0a0 2020-11-16T22:34:40.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc7fe7fc700 10 client.6968 handle_command_reply: tid=0 2020-11-16T22:34:40.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 client.6968 shutdown 2020-11-16T22:34:40.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 msgr2=0x12f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fc7f8020040 tx=0x7fc7f8028000).stop 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 shutdown_connections 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x128e0a0 0x12efd50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1288470 0x12f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc7fe7fc700 0 client.6968 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x12b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 --2- 172.21.15.181:0/2484619627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x12f0520 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 >> 172.21.15.181:0/2484619627 conn(0x123de40 msgr2=0x12c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 shutdown_connections 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 1 -- 172.21.15.181:0/2484619627 wait complete. 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.006+0000 7fc824e1f740 20 client.6968 trim_cache size 0 max 16384 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.022+0000 7fc824e1f740 1 Processor -- start 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.022+0000 7fc824e1f740 1 -- start start 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 0x12afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x1285f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1192e40 con 0x1001bb0 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1289f30 con 0x123de40 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12fd7d0 con 0x106f7b0 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 0x12afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43194/0 (socket says 172.21.15.181:43194) 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 0x12afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35354/0 (socket says 172.21.15.181:35354) 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 -- 172.21.15.181:0/1126617838 learned_addr learned my addr 172.21.15.181:0/1126617838 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x1285f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 -- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 msgr2=0x12afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 0x12afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 -- 172.21.15.181:0/1126617838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 msgr2=0x1285f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:40.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x1285f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 -- 172.21.15.181:0/1126617838 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80081d0 con 0x106f7b0 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc81492b700 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc80800ccb0 tx=0x7fc808008040).ready entity=mon.2 client_cookie=20dc446336ab129f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc805ffb700 1 -- 172.21.15.181:0/1126617838 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808016bc0 con 0x106f7b0 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc805ffb700 1 -- 172.21.15.181:0/1126617838 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808016d40 con 0x106f7b0 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc805ffb700 1 -- 172.21.15.181:0/1126617838 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080186a0 con 0x106f7b0 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 msgr2=0x1286440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc80800ccb0 tx=0x7fc808008040).stop 2020-11-16T22:34:40.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- 172.21.15.181:0/1126617838 shutdown_connections 2020-11-16T22:34:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x1286440 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x123de40 0x12afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 --2- 172.21.15.181:0/1126617838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1001bb0 0x1285f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- 172.21.15.181:0/1126617838 >> 172.21.15.181:0/1126617838 conn(0x7fc7d8002320 msgr2=0x128f140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:40.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- 172.21.15.181:0/1126617838 shutdown_connections 2020-11-16T22:34:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- 172.21.15.181:0/1126617838 wait complete. 2020-11-16T22:34:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 -- start start 2020-11-16T22:34:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.026+0000 7fc824e1f740 1 Processor -- start 2020-11-16T22:34:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x13377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x123de40 0x12ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13400e0 con 0x123de40 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1340380 con 0x1001bb0 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1340620 con 0x106f7b0 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 10 client.0 ll_register_callbacks cb 0x12c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x13377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35358/0 (socket says 172.21.15.181:35358) 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 -- 172.21.15.181:0/442050465 learned_addr learned my addr 172.21.15.181:0/442050465 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc81492b700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x123de40 0x12ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 -- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 msgr2=0x13377c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x13377c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 -- 172.21.15.181:0/442050465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x123de40 msgr2=0x12ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:40.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x123de40 0x12ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 -- 172.21.15.181:0/442050465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x101f720 con 0x1001bb0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80ffff700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7fc800007fd0 tx=0x7fc800013000).ready entity=mon.1 client_cookie=642d29e7118100c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800001ea0 con 0x1001bb0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7f8011c80 con 0x1001bb0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc8000436a0 con 0x1001bb0 2020-11-16T22:34:40.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc800043860 con 0x1001bb0 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 10 client.7609 fetch_fsmap learned FSMap version 103 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x102ae00 con 0x1001bb0 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8011c80 con 0x1001bb0 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 103) v1 ==== 2607+0+0 (secure 0 0 0) 0x7fc800042d80 con 0x1001bb0 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 10 client.7609 fetch_fsmap finished waiting for FSMap version 103 2020-11-16T22:34:40.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 10 client.7609 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:34:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 20 client.7609 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1288470 0x133f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 4 client.7609 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x10b2f50 con 0x1288470 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1288470 0x133f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc7fe7fc700 10 client.7609 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.030+0000 7fc80f7fe700 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1288470 0x133f7f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=996b1208232ff8df server_cookie=4b8fca916c3a56aa in_seq=0 out_seq=0 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.046+0000 7fc7fe7fc700 1 -- 172.21.15.181:0/442050465 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x10b2f50 con 0x1288470 2020-11-16T22:34:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.046+0000 7fc7fe7fc700 10 client.7609 handle_command_reply: tid=0 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.046+0000 7fc824e1f740 1 client.7609 shutdown 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.046+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 msgr2=0x1334290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.046+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7fc800007fd0 tx=0x7fc800013000).stop 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 shutdown_connections 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1288470 0x133f7f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x123de40 0x12ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc7fe7fc700 0 client.7609 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:40.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1001bb0 0x1334290 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 --2- 172.21.15.181:0/442050465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x106f7b0 0x13377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 >> 172.21.15.181:0/442050465 conn(0x7fc7d8002320 msgr2=0x12c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 shutdown_connections 2020-11-16T22:34:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 1 -- 172.21.15.181:0/442050465 wait complete. 2020-11-16T22:34:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc824e1f740 20 client.7609 trim_cache size 0 max 16384 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 msgr2=0x7fc7e0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 0x7fc7e0057550 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fc8100660e0 tx=0x7fc7f8015000).stop 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 msgr2=0x7fc81009ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc80000c6e0 tx=0x7fc800007c50).stop 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 shutdown_connections 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc7e0055090 0x7fc7e0057550 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81015f090 0x7fc81015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc810166bb0 0x7fc81009ea70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 --2- 172.21.15.181:0/3859195278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc810167ac0 0x7fc81009ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 >> 172.21.15.181:0/3859195278 conn(0x7fc810060bc0 msgr2=0x7fc8100b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 shutdown_connections 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.050+0000 7fc81638e700 1 -- 172.21.15.181:0/3859195278 wait complete. 2020-11-16T22:34:40.832 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:40.833 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:34:40.833 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:34:40.833 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:40.833 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.970+0000 7fbf5fe1f700 1 Processor -- start 2020-11-16T22:34:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.970+0000 7fbf5fe1f700 1 -- start start 2020-11-16T22:34:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf5815f5b0 0x7fbf580d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf58160e40 con 0x7fbf580615f0 2020-11-16T22:34:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf58160ba0 con 0x7fbf580d1830 2020-11-16T22:34:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf581609a0 con 0x7fbf5815f5b0 2020-11-16T22:34:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5dbbb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf5815f5b0 0x7fbf580d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5e3bc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbf580d1830 0x7fbf58080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49672/0 (socket says 172.21.15.181:49672) 2020-11-16T22:34:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5e3bc700 1 -- 172.21.15.181:0/1961379237 learned_addr learned my addr 172.21.15.181:0/1961379237 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652393752 0 0) 0x7fbf58160ba0 con 0x7fbf580d1830 2020-11-16T22:34:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf40003240 con 0x7fbf580d1830 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5dbbb700 1 -- 172.21.15.181:0/1961379237 >> v1:172.21.15.181:6789/0 conn(0x7fbf580d1830 legacy=0x7fbf58080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354762676 0 0) 0x7fbf58160e40 con 0x7fbf580615f0 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5dbbb700 1 -- 172.21.15.181:0/1961379237 >> v1:172.21.15.175:6789/0 conn(0x7fbf580615f0 legacy=0x7fbf580622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5dbbb700 1 -- 172.21.15.181:0/1961379237 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58060970 con 0x7fbf5815f5b0 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5dbbb700 1 --2- 172.21.15.181:0/1961379237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf5815f5b0 0x7fbf580d1340 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fbf4c00b990 tx=0x7fbf4c006d90).ready entity=mon.1 client_cookie=fd345b57fbe4454c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c006030 con 0x7fbf5815f5b0 2020-11-16T22:34:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c006190 con 0x7fbf5815f5b0 2020-11-16T22:34:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5cbb9700 1 -- 172.21.15.181:0/1961379237 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c006510 con 0x7fbf5815f5b0 2020-11-16T22:34:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1961379237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf5815f5b0 msgr2=0x7fbf580d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1961379237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf5815f5b0 0x7fbf580d1340 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fbf4c00b990 tx=0x7fbf4c006d90).stop 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1961379237 shutdown_connections 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1961379237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf5815f5b0 0x7fbf580d1340 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1961379237 >> 172.21.15.181:0/1961379237 conn(0x7fbf580d69a0 msgr2=0x7fbf580d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1961379237 shutdown_connections 2020-11-16T22:34:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.974+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1961379237 wait complete. 2020-11-16T22:34:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 Processor -- start 2020-11-16T22:34:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- start start 2020-11-16T22:34:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf580615f0 0x7fbf580d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf58162580 0x7fbf5814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5816f180 con 0x7fbf580615f0 2020-11-16T22:34:41.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580ba7d0 con 0x7fbf580d1830 2020-11-16T22:34:41.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580b9d50 con 0x7fbf58162580 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5e3bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf58162580 0x7fbf5814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35370/0 (socket says 172.21.15.181:35370) 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 -- 172.21.15.181:0/1640635032 learned_addr learned my addr 172.21.15.181:0/1640635032 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5dbbb700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf580615f0 0x7fbf580d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 -- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf58162580 msgr2=0x7fbf5814b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf58162580 0x7fbf5814b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 -- 172.21.15.181:0/1640635032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf580615f0 msgr2=0x7fbf580d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf580615f0 0x7fbf580d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 -- 172.21.15.181:0/1640635032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580b9ff0 con 0x7fbf580d1830 2020-11-16T22:34:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5d3ba700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fbf5400c760 tx=0x7fbf54007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400e560 con 0x7fbf580d1830 2020-11-16T22:34:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400e6e0 con 0x7fbf580d1830 2020-11-16T22:34:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400eaa0 con 0x7fbf580d1830 2020-11-16T22:34:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf580ba290 con 0x7fbf580d1830 2020-11-16T22:34:41.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.978+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf580b9930 con 0x7fbf580d1830 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf580b9b80 con 0x7fbf580d1830 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbf54010070 con 0x7fbf580d1830 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf49ffb700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 0x7fbf30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf5406d6f0 con 0x7fbf580d1830 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf5dbbb700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 0x7fbf30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.982+0000 7fbf5dbbb700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 0x7fbf30057520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbf5805d5c0 tx=0x7fbf4c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:40.986+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbf54043840 con 0x7fbf580d1830 2020-11-16T22:34:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.186+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf58069eb0 con 0x7fbf580d1830 2020-11-16T22:34:41.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.190+0000 7fbf49ffb700 1 -- 172.21.15.181:0/1640635032 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4666 (secure 0 0 0) 0x7fbf54046e60 con 0x7fbf580d1830 2020-11-16T22:34:41.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 msgr2=0x7fbf30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 0x7fbf30057520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbf5805d5c0 tx=0x7fbf4c015000).stop 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 msgr2=0x7fbf5814b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fbf5400c760 tx=0x7fbf54007a90).stop 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 shutdown_connections 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbf30055060 0x7fbf30057520 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf58162580 0x7fbf5814b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf580615f0 0x7fbf580d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 --2- 172.21.15.181:0/1640635032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf580d1830 0x7fbf5814b340 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 >> 172.21.15.181:0/1640635032 conn(0x7fbf580d69a0 msgr2=0x7fbf580d48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 shutdown_connections 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.194+0000 7fbf5fe1f700 1 -- 172.21.15.181:0/1640635032 wait complete. 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 103 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:41.588 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:36.781890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:41.790 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i flush journal 2020-11-16T22:34:41.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 Processor -- start 2020-11-16T22:34:41.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- start start 2020-11-16T22:34:41.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 0x7f6d240b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.928 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2415e760 con 0x7f6d240b7ab0 2020-11-16T22:34:41.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2415ea00 con 0x7f6d240b58c0 2020-11-16T22:34:41.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2415eca0 con 0x7f6d240b5ca0 2020-11-16T22:34:41.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 0x7f6d240b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:41.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2930f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6d240b58c0 0x7f6d2415dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49682/0 (socket says 172.21.15.181:49682) 2020-11-16T22:34:41.929 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2930f700 1 -- 172.21.15.181:0/3921961687 learned_addr learned my addr 172.21.15.181:0/3921961687 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:41.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d1bfff700 1 -- 172.21.15.181:0/3921961687 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215811742 0 0) 0x7f6d2415ea00 con 0x7f6d240b58c0 2020-11-16T22:34:41.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d28b0e700 1 -- 172.21.15.181:0/3921961687 >> v1:172.21.15.181:6789/0 conn(0x7f6d240b58c0 legacy=0x7f6d2415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d28b0e700 1 -- 172.21.15.181:0/3921961687 >> v1:172.21.15.175:6789/0 conn(0x7f6d240b7ab0 legacy=0x7f6d240cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d28b0e700 1 -- 172.21.15.181:0/3921961687 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240ccfc0 con 0x7f6d240b5ca0 2020-11-16T22:34:41.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/3921961687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 0x7f6d240b75c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f6d2000b9d0 tx=0x7f6d20006df0).ready entity=mon.2 client_cookie=7e69530aeff26c17 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:41.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d1bfff700 1 -- 172.21.15.181:0/3921961687 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20009870 con 0x7f6d240b5ca0 2020-11-16T22:34:41.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d1bfff700 1 -- 172.21.15.181:0/3921961687 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d200099d0 con 0x7f6d240b5ca0 2020-11-16T22:34:41.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d1bfff700 1 -- 172.21.15.181:0/3921961687 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20009d50 con 0x7f6d240b5ca0 2020-11-16T22:34:41.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- 172.21.15.181:0/3921961687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 msgr2=0x7f6d240b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:41.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 --2- 172.21.15.181:0/3921961687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 0x7f6d240b75c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f6d2000b9d0 tx=0x7f6d20006df0).stop 2020-11-16T22:34:41.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- 172.21.15.181:0/3921961687 shutdown_connections 2020-11-16T22:34:41.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 --2- 172.21.15.181:0/3921961687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b5ca0 0x7f6d240b75c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:41.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- 172.21.15.181:0/3921961687 >> 172.21.15.181:0/3921961687 conn(0x7f6d24060bc0 msgr2=0x7f6d240d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.938+0000 7f6d2b573700 1 -- 172.21.15.181:0/3921961687 shutdown_connections 2020-11-16T22:34:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 -- 172.21.15.181:0/3921961687 wait complete. 2020-11-16T22:34:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 Processor -- start 2020-11-16T22:34:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 -- start start 2020-11-16T22:34:41.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 0x7f6d240cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 0x7f6d240cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d240b7ab0 0x7f6d24172b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:41.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24173810 con 0x7f6d240b7ab0 2020-11-16T22:34:41.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240c9860 con 0x7f6d240b5ca0 2020-11-16T22:34:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2b573700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240c9b00 con 0x7f6d240b58c0 2020-11-16T22:34:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2930f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 0x7f6d240cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 0x7f6d240cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2930f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 0x7f6d240cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43224/0 (socket says 172.21.15.181:43224) 2020-11-16T22:34:42.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.942+0000 7f6d2930f700 1 -- 172.21.15.181:0/4203103993 learned_addr learned my addr 172.21.15.181:0/4203103993 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d29b10700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d240b7ab0 0x7f6d24172b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 -- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 msgr2=0x7f6d240cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 0x7f6d240cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 -- 172.21.15.181:0/4203103993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d240b7ab0 msgr2=0x7f6d24172b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d240b7ab0 0x7f6d24172b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 -- 172.21.15.181:0/4203103993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240c9da0 con 0x7f6d240b5ca0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 0x7f6d240cdf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f6d20012000 tx=0x7f6d20015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 -- 172.21.15.181:0/4203103993 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2001e460 con 0x7f6d240b5ca0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 -- 172.21.15.181:0/4203103993 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2001e5c0 con 0x7f6d240b5ca0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d240ca040 con 0x7f6d240b5ca0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 -- 172.21.15.181:0/4203103993 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2001e940 con 0x7f6d240b5ca0 2020-11-16T22:34:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d24173280 con 0x7f6d240b5ca0 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 -- 172.21.15.181:0/4203103993 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6d20025030 con 0x7f6d240b5ca0 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 0x7f6cfc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d197fa700 1 -- 172.21.15.181:0/4203103993 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d2006f700 con 0x7f6d240b5ca0 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d2930f700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 0x7f6cfc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.946+0000 7f6d2930f700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 0x7f6cfc057520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f6d240cf9e0 tx=0x7f6d14007ac0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.950+0000 7f6d3a004740 1 Processor -- start 2020-11-16T22:34:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.950+0000 7f6d3a004740 1 -- start start 2020-11-16T22:34:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1789cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 0x17e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x161c4b0 con 0x15727b0 2020-11-16T22:34:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d0930 con 0x178b470 2020-11-16T22:34:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15906f0 con 0x1504bb0 2020-11-16T22:34:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d29b10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 0x17e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43232/0 (socket says 172.21.15.181:43232) 2020-11-16T22:34:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 -- 172.21.15.181:0/2772188352 learned_addr learned my addr 172.21.15.181:0/2772188352 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d2930f700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1789cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 -- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 msgr2=0x17e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 0x17e5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 -- 172.21.15.181:0/2772188352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 msgr2=0x1789cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1789cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 -- 172.21.15.181:0/2772188352 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x158b5a0 con 0x1504bb0 2020-11-16T22:34:42.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d2930f700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1789cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d29b10700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 0x17e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f6d20000dd0 tx=0x7f6d20011000).ready entity=mon.2 client_cookie=80fcf698bb51a16f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d027fc700 1 -- 172.21.15.181:0/2772188352 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200479c0 con 0x1504bb0 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d027fc700 1 -- 172.21.15.181:0/2772188352 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d20048b10 con 0x1504bb0 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d027fc700 1 -- 172.21.15.181:0/2772188352 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20045410 con 0x1504bb0 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 msgr2=0x17c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f6d20000dd0 tx=0x7f6d20011000).stop 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- 172.21.15.181:0/2772188352 shutdown_connections 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x178b470 0x17e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1789cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2772188352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17c8530 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- 172.21.15.181:0/2772188352 >> 172.21.15.181:0/2772188352 conn(0x1740e40 msgr2=0x17e5580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- 172.21.15.181:0/2772188352 shutdown_connections 2020-11-16T22:34:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.954+0000 7f6d3a004740 1 -- 172.21.15.181:0/2772188352 wait complete. 2020-11-16T22:34:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- start start 2020-11-16T22:34:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 Processor -- start 2020-11-16T22:34:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x178b470 0x17b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bd6e0 con 0x178b470 2020-11-16T22:34:42.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16bd820 con 0x15727b0 2020-11-16T22:34:42.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x16bea90 con 0x1504bb0 2020-11-16T22:34:42.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 10 client.0 ll_register_callbacks cb 0x16b3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35398/0 (socket says 172.21.15.181:35398) 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d2930f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 -- 172.21.15.181:0/1832739017 learned_addr learned my addr 172.21.15.181:0/1832739017 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d29b10700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x178b470 0x17b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 -- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 msgr2=0x17e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 -- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x178b470 msgr2=0x17b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x178b470 0x17b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 -- 172.21.15.181:0/1832739017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16ae280 con 0x15727b0 2020-11-16T22:34:42.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f6d20049550 tx=0x7f6d2006f030).ready entity=mon.1 client_cookie=387112a1ad390606 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:42.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:42.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2003d070 con 0x15727b0 2020-11-16T22:34:42.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1785e10 con 0x15727b0 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d20048790 con 0x15727b0 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d20048950 con 0x15727b0 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 10 client.7630 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x14e3fd0 con 0x15727b0 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1785e10 con 0x15727b0 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f6d2006f2b0 con 0x15727b0 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 10 client.7630 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 10 client.7630 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 20 client.7630 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x17f2470 0x17f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 4 client.7630 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x15aec50 con 0x17f2470 2020-11-16T22:34:42.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d2930f700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x17f2470 0x17f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.958+0000 7f6d2930f700 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x17f2470 0x17f33f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=3527491277c29f9 server_cookie=758f74cee43f71ab in_seq=0 out_seq=0 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6cf37fe700 10 client.7630 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6cf37fe700 1 -- 172.21.15.181:0/1832739017 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x15aec50 con 0x17f2470 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6cf37fe700 10 client.7630 handle_command_reply: tid=0 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 client.7630 shutdown 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 msgr2=0x17c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:42.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f6d20049550 tx=0x7f6d2006f030).stop 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 shutdown_connections 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x17f2470 0x17f33f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x178b470 0x17b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6cf37fe700 0 client.7630 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 --2- 172.21.15.181:0/1832739017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c6870 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 >> 172.21.15.181:0/1832739017 conn(0x1740e40 msgr2=0x17ca230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:42.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 shutdown_connections 2020-11-16T22:34:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 1 -- 172.21.15.181:0/1832739017 wait complete. 2020-11-16T22:34:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.962+0000 7f6d3a004740 20 client.7630 trim_cache size 0 max 16384 2020-11-16T22:34:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.978+0000 7f6d3a004740 1 Processor -- start 2020-11-16T22:34:42.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.978+0000 7f6d3a004740 1 -- start start 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1740e40 0x17c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1695e40 con 0x1740e40 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1790e40 con 0x15727b0 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x178b020 con 0x1504bb0 2020-11-16T22:34:42.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35404/0 (socket says 172.21.15.181:35404) 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 -- 172.21.15.181:0/323102293 learned_addr learned my addr 172.21.15.181:0/323102293 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 -- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 msgr2=0x17e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d2930f700 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1740e40 0x17c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 -- 172.21.15.181:0/323102293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1740e40 msgr2=0x17c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1740e40 0x17c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 -- 172.21.15.181:0/323102293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17c9970 con 0x15727b0 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d29b10700 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f6d1c001ed0 tx=0x7f6d1c007f90).ready entity=mon.1 client_cookie=f50c20bc8a45b97b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d02ffd700 1 -- 172.21.15.181:0/323102293 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c012d20 con 0x15727b0 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d02ffd700 1 -- 172.21.15.181:0/323102293 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c012e80 con 0x15727b0 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d02ffd700 1 -- 172.21.15.181:0/323102293 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c006ac0 con 0x15727b0 2020-11-16T22:34:42.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 msgr2=0x17c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f6d1c001ed0 tx=0x7f6d1c007f90).stop 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- 172.21.15.181:0/323102293 shutdown_connections 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x15727b0 0x17c8350 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1740e40 0x17c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 --2- 172.21.15.181:0/323102293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1504bb0 0x17e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:42.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- 172.21.15.181:0/323102293 >> 172.21.15.181:0/323102293 conn(0x7f6cf8002320 msgr2=0x17f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:42.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- 172.21.15.181:0/323102293 shutdown_connections 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- 172.21.15.181:0/323102293 wait complete. 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 -- start start 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.982+0000 7f6d3a004740 1 Processor -- start 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1841b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 0x17b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1845840 con 0x15727b0 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1791710 con 0x1504bb0 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1845aa0 con 0x1740e40 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 10 client.0 ll_register_callbacks cb 0x17cad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d29b10700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 0x17b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35410/0 (socket says 172.21.15.181:35410) 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d29b10700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 0x17b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43256/0 (socket says 172.21.15.181:43256) 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 -- 172.21.15.181:0/2966426196 learned_addr learned my addr 172.21.15.181:0/2966426196 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1841b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 -- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 msgr2=0x17b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 0x17b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 -- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 msgr2=0x1841b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1841b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 -- 172.21.15.181:0/2966426196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1792550 con 0x1504bb0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d2930f700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f6d14022b80 tx=0x7f6d14017040).ready entity=mon.1 client_cookie=75d22fb39d2c657c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d140146e0 con 0x1504bb0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d14014840 con 0x1504bb0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d20048b10 con 0x1504bb0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d14014c10 con 0x1504bb0 2020-11-16T22:34:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 10 client.7636 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x7f6cf80081d0 con 0x1504bb0 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.986+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d20048b10 con 0x1504bb0 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f6d14035020 con 0x1504bb0 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 10 client.7636 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 10 client.7636 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 20 client.7636 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x178b470 0x17b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 4 client.7636 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x15b5f50 con 0x178b470 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x178b470 0x17b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6cf37fe700 10 client.7636 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:41.990+0000 7f6d28b0e700 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x178b470 0x17b7aa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=df82ff28a474394 server_cookie=167467840ede29 in_seq=0 out_seq=0 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6cf37fe700 1 -- 172.21.15.181:0/2966426196 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x15b5f50 con 0x178b470 2020-11-16T22:34:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6cf37fe700 10 client.7636 handle_command_reply: tid=0 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 client.7636 shutdown 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 msgr2=0x179e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f6d14022b80 tx=0x7f6d14017040).stop 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 shutdown_connections 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x178b470 0x17b7aa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1740e40 0x17b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6cf37fe700 0 client.7636 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1504bb0 0x179e7f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 --2- 172.21.15.181:0/2966426196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x15727b0 0x1841b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 >> 172.21.15.181:0/2966426196 conn(0x7f6cf8002320 msgr2=0x17ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 shutdown_connections 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 1 -- 172.21.15.181:0/2966426196 wait complete. 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d3a004740 20 client.7636 trim_cache size 0 max 16384 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 msgr2=0x7f6cfc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 0x7f6cfc057520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f6d240cf9e0 tx=0x7f6d14007ac0).stop 2020-11-16T22:34:43.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 msgr2=0x7f6d240cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 0x7f6d240cdf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f6d20012000 tx=0x7f6d20015000).stop 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 shutdown_connections 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cfc055060 0x7f6cfc057520 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d240b7ab0 0x7f6d24172b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d240b58c0 0x7f6d240cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 --2- 172.21.15.181:0/4203103993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d240b5ca0 0x7f6d240cdf50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:43.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 >> 172.21.15.181:0/4203103993 conn(0x7f6d24060bc0 msgr2=0x7f6d240c77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 shutdown_connections 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:42.010+0000 7f6d2b573700 1 -- 172.21.15.181:0/4203103993 wait complete. 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:34:43.597 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:43.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:34:43.598 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:34:43.657 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/ 2020-11-16T22:34:43.657 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4 2020-11-16T22:34:43.657 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5 2020-11-16T22:34:43.657 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:34:43.658 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:34:43.658 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:34:43.662 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:43.662 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:43.662 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:43.663 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:43.663 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:34:43.663 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2020-11-16T22:34:43.964 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:43.964 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:43.965 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:43.965 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:43.965 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:34:43.966 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:44.070 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:44.373 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:44.679 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627781 2020-11-16T22:34:44.868 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:44.869 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:44.870 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:44.871 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:44.871 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:34:44.872 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627782 2020-11-16T22:34:45.212 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:34:45.213 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:34:45.213 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:34:45.213 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:34:45.214 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:34:45.214 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:34:45.327 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627783 2020-11-16T22:34:45.516 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:34:45.551 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 Processor -- start 2020-11-16T22:34:45.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 -- start start 2020-11-16T22:34:45.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd92620cb0 0x55cd9260d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 0x55cd9260dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 0x55cd9260eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9257f1f0 con 0x55cd92620cb0 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9254dbe0 con 0x55cd926aeb90 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd926b51c0 con 0x55cd926b55f0 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a5d00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 0x55cd9260eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 0x55cd9260dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a5d00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 0x55cd9260eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43270/0 (socket says 172.21.15.181:43270) 2020-11-16T22:34:45.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a5d00700 1 -- 172.21.15.181:0/4105660998 learned_addr learned my addr 172.21.15.181:0/4105660998 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:45.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a54ff700 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd92620cb0 0x55cd9260d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 -- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 msgr2=0x55cd9260eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:45.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 0x55cd9260eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 -- 172.21.15.181:0/4105660998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd92620cb0 msgr2=0x55cd9260d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:45.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd92620cb0 0x55cd9260d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.562+0000 7f27a4cfe700 1 -- 172.21.15.181:0/4105660998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd925778a0 con 0x55cd926aeb90 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 0x55cd9260dc70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f27900029a0 tx=0x7f2790006930).ready entity=mon.1 client_cookie=5d6c0b134ae79490 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f279f7fe700 1 -- 172.21.15.181:0/4105660998 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000d560 con 0x55cd926aeb90 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f279f7fe700 1 -- 172.21.15.181:0/4105660998 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000d6c0 con 0x55cd926aeb90 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f279f7fe700 1 -- 172.21.15.181:0/4105660998 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000da40 con 0x55cd926aeb90 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 msgr2=0x55cd9260dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 0x55cd9260dc70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f27900029a0 tx=0x7f2790006930).stop 2020-11-16T22:34:45.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/4105660998 shutdown_connections 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926b55f0 0x55cd9260eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd92620cb0 0x55cd9260d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/4105660998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd926aeb90 0x55cd9260dc70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/4105660998 >> 172.21.15.181:0/4105660998 conn(0x55cd926270d0 msgr2=0x55cd926b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/4105660998 shutdown_connections 2020-11-16T22:34:45.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/4105660998 wait complete. 2020-11-16T22:34:45.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 Processor -- start 2020-11-16T22:34:45.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.566+0000 7f27b3ef9c40 1 -- start start 2020-11-16T22:34:45.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd92620cb0 0x55cd926b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 0x55cd925d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd926b55f0 0x55cd926c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd926c5200 con 0x55cd926b55f0 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd926c5430 con 0x55cd92620cb0 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27b3ef9c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd926af0f0 con 0x55cd926aeb90 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 0x55cd925d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 0x55cd925d63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43274/0 (socket says 172.21.15.181:43274) 2020-11-16T22:34:45.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 -- 172.21.15.181:0/2517987055 learned_addr learned my addr 172.21.15.181:0/2517987055 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:45.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a54ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd92620cb0 0x55cd926b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a5d00700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd926b55f0 0x55cd926c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 -- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd92620cb0 msgr2=0x55cd926b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:45.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd92620cb0 0x55cd926b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 -- 172.21.15.181:0/2517987055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd926b55f0 msgr2=0x55cd926c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:45.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd926b55f0 0x55cd926c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 -- 172.21.15.181:0/2517987055 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd92573280 con 0x55cd926aeb90 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a5d00700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd926b55f0 0x55cd926c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.570+0000 7f27a4cfe700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 0x55cd925d63e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f2790002970 tx=0x7f2790015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 -- 172.21.15.181:0/2517987055 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790012b50 con 0x55cd926aeb90 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 -- 172.21.15.181:0/2517987055 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790012cb0 con 0x55cd926aeb90 2020-11-16T22:34:45.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd92551620 con 0x55cd926aeb90 2020-11-16T22:34:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 -- 172.21.15.181:0/2517987055 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279001d640 con 0x55cd926aeb90 2020-11-16T22:34:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd926af630 con 0x55cd926aeb90 2020-11-16T22:34:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 -- 172.21.15.181:0/2517987055 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f279001a030 con 0x55cd926aeb90 2020-11-16T22:34:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f278c053d10 0x7f278c0561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f279dffb700 1 -- 172.21.15.181:0/2517987055 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f279006e920 con 0x55cd926aeb90 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27a54ff700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f278c053d10 0x7f278c0561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55cd9269cd00 0x55cd926b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- osd_op(unknown.0.0:1 13.2 13:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x55cd926b7990 con 0x55cd9269cd00 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27a54ff700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f278c053d10 0x7f278c0561d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f27a0008450 tx=0x7f27a0013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.574+0000 7f27a5d00700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55cd9269cd00 0x55cd926b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.578+0000 7f27a5d00700 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55cd9269cd00 0x55cd926b74a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.578+0000 7f27a5d00700 1 -- 172.21.15.181:0/2517987055 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v97'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2798012040 con 0x55cd9269cd00 2020-11-16T22:34:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.578+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55cd9269cd00 msgr2=0x55cd926b74a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.578+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55cd9269cd00 0x55cd926b74a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.578+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f278c053d10 msgr2=0x7f278c0561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f278c053d10 0x7f278c0561d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f27a0008450 tx=0x7f27a0013000).stop 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 msgr2=0x55cd925d63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 --2- 172.21.15.181:0/2517987055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cd926aeb90 0x55cd925d63e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f2790002970 tx=0x7f2790015000).stop 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27a5d00700 1 -- 172.21.15.181:0/2517987055 reap_dead start 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 shutdown_connections 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 >> 172.21.15.181:0/2517987055 conn(0x55cd926270d0 msgr2=0x55cd9261f590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 shutdown_connections 2020-11-16T22:34:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:45.582+0000 7f27b3ef9c40 1 -- 172.21.15.181:0/2517987055 wait complete. 2020-11-16T22:34:46.180 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:34:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.222+0000 7f99f40b2c40 1 Processor -- start 2020-11-16T22:34:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 -- start start 2020-11-16T22:34:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 0x55ea05464c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea05501330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 0x55ea05453290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea053c21f0 con 0x55ea054f1690 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea05390be0 con 0x55ea05450760 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea05500710 con 0x55ea054fb450 2020-11-16T22:34:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e5eb9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 0x55ea05453290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 0x55ea05464c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e5eb9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 0x55ea05453290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43292/0 (socket says 172.21.15.181:43292) 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e5eb9700 1 -- 172.21.15.181:0/2344699765 learned_addr learned my addr 172.21.15.181:0/2344699765 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea05501330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 -- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 msgr2=0x55ea05453290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 0x55ea05453290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 -- 172.21.15.181:0/2344699765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 msgr2=0x55ea05501330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea05501330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 -- 172.21.15.181:0/2344699765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea053ba8a0 con 0x55ea05450760 2020-11-16T22:34:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea05501330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:46.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99e56b8700 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 0x55ea05464c00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f99d0001420 tx=0x7f99d0006890).ready entity=mon.1 client_cookie=c2fece3e4134b06d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:46.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99df7fe700 1 -- 172.21.15.181:0/2344699765 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0006320 con 0x55ea05450760 2020-11-16T22:34:46.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99df7fe700 1 -- 172.21.15.181:0/2344699765 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0004500 con 0x55ea05450760 2020-11-16T22:34:46.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99df7fe700 1 -- 172.21.15.181:0/2344699765 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0004880 con 0x55ea05450760 2020-11-16T22:34:46.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.226+0000 7f99f40b2c40 1 -- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 msgr2=0x55ea05464c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 0x55ea05464c00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f99d0001420 tx=0x7f99d0006890).stop 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- 172.21.15.181:0/2344699765 shutdown_connections 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea054fb450 0x55ea05453290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea05450760 0x55ea05464c00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/2344699765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea05501330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- 172.21.15.181:0/2344699765 >> 172.21.15.181:0/2344699765 conn(0x55ea0546b1d0 msgr2=0x55ea05454100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- 172.21.15.181:0/2344699765 shutdown_connections 2020-11-16T22:34:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- 172.21.15.181:0/2344699765 wait complete. 2020-11-16T22:34:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 Processor -- start 2020-11-16T22:34:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- start start 2020-11-16T22:34:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 0x55ea05452570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea054f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea054fb450 0x55ea054645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea05502fe0 con 0x55ea054f1690 2020-11-16T22:34:46.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea05503280 con 0x55ea054fb450 2020-11-16T22:34:46.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99f40b2c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea05503520 con 0x55ea05450760 2020-11-16T22:34:46.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.230+0000 7f99e56b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 0x55ea05452570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 0x55ea05452570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43294/0 (socket says 172.21.15.181:43294) 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e5eb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea054fb450 0x55ea054645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 -- 172.21.15.181:0/1984266694 learned_addr learned my addr 172.21.15.181:0/1984266694 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea054f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 -- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea054fb450 msgr2=0x55ea054645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ea054fb450 0x55ea054645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 -- 172.21.15.181:0/1984266694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 msgr2=0x55ea054f0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea054f0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 -- 172.21.15.181:0/1984266694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea053b6280 con 0x55ea05450760 2020-11-16T22:34:46.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea054f1690 0x55ea054f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:46.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99e56b8700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 0x55ea05452570 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x55ea054fbc80 tx=0x7f99d0012000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:46.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99ddffb700 1 -- 172.21.15.181:0/1984266694 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0014880 con 0x55ea05450760 2020-11-16T22:34:46.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99ddffb700 1 -- 172.21.15.181:0/1984266694 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d00149e0 con 0x55ea05450760 2020-11-16T22:34:46.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea05394620 con 0x55ea05450760 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99ddffb700 1 -- 172.21.15.181:0/1984266694 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0014d80 con 0x55ea05450760 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea05503a60 con 0x55ea05450760 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.234+0000 7f99ddffb700 1 -- 172.21.15.181:0/1984266694 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f99d0027070 con 0x55ea05450760 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99ddffb700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8053ce0 0x7f99c80561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99ddffb700 1 -- 172.21.15.181:0/1984266694 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99d006daf0 con 0x55ea05450760 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8053ce0 0x7f99c80561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55ea054f0510 0x55ea055046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- osd_op(unknown.0.0:1 13.2 13:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x55ea055018b0 con 0x55ea054f0510 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99e4eb7700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8053ce0 0x7f99c80561a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f99d800c010 tx=0x7f99d8013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99e5eb9700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55ea054f0510 0x55ea055046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.238+0000 7f99e5eb9700 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55ea054f0510 0x55ea055046b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99e5eb9700 1 -- 172.21.15.181:0/1984266694 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v97'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f99e0012040 con 0x55ea054f0510 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55ea054f0510 msgr2=0x55ea055046b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x55ea054f0510 0x55ea055046b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8053ce0 msgr2=0x7f99c80561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f99c8053ce0 0x7f99c80561a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f99d800c010 tx=0x7f99d8013000).stop 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 msgr2=0x55ea05452570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 --2- 172.21.15.181:0/1984266694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ea05450760 0x55ea05452570 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x55ea054fbc80 tx=0x7f99d0012000).stop 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99e5eb9700 1 -- 172.21.15.181:0/1984266694 reap_dead start 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 shutdown_connections 2020-11-16T22:34:46.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 >> 172.21.15.181:0/1984266694 conn(0x55ea0546b1d0 msgr2=0x55ea055008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:46.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 shutdown_connections 2020-11-16T22:34:46.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:46.242+0000 7f99f40b2c40 1 -- 172.21.15.181:0/1984266694 wait complete. 2020-11-16T22:34:46.838 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:34:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 Processor -- start 2020-11-16T22:34:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 -- start start 2020-11-16T22:34:47.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 0x560383259ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603832061f0 con 0x5603832aa850 2020-11-16T22:34:47.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603831d4be0 con 0x560383322b00 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603832abb60 con 0x560383323f50 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e3ddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e4dde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e4dde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35472/0 (socket says 172.21.15.181:35472) 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e3ddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43312/0 (socket says 172.21.15.181:43312) 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e45dd700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 0x560383259ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.090+0000 7fa0e4dde700 1 -- 172.21.15.181:0/3006382854 learned_addr learned my addr 172.21.15.181:0/3006382854 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:47.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 msgr2=0x56038333f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:47.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3006382854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 msgr2=0x560383259ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:47.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 0x560383259ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3006382854 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603831fe8a0 con 0x560383323f50 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e45dd700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 0x560383259ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e4dde700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fa0cc0019d0 tx=0x7fa0cc006aa0).ready entity=mon.2 client_cookie=58fe8dc4fc222bd4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e2dda700 1 -- 172.21.15.181:0/3006382854 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00d560 con 0x560383323f50 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e2dda700 1 -- 172.21.15.181:0/3006382854 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00d6c0 con 0x560383323f50 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0e2dda700 1 -- 172.21.15.181:0/3006382854 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00da60 con 0x560383323f50 2020-11-16T22:34:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 msgr2=0x5603832a9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fa0cc0019d0 tx=0x7fa0cc006aa0).stop 2020-11-16T22:34:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3006382854 shutdown_connections 2020-11-16T22:34:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560383322b00 0x56038333f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603832aa850 0x560383259ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3006382854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603832a9fd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3006382854 >> 172.21.15.181:0/3006382854 conn(0x5603832b1400 msgr2=0x5603832a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:47.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3006382854 shutdown_connections 2020-11-16T22:34:47.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.094+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3006382854 wait complete. 2020-11-16T22:34:47.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 Processor -- start 2020-11-16T22:34:47.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- start start 2020-11-16T22:34:47.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603832aa850 0x56038333fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603833225f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560383297ef0 0x560383299d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603832b3c20 con 0x560383297ef0 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603832b3ec0 con 0x5603832aa850 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603832b4160 con 0x560383323f50 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603833225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e45dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603832aa850 0x56038333fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603833225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43316/0 (socket says 172.21.15.181:43316) 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3707920418 learned_addr learned my addr 172.21.15.181:0/3707920418 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e4dde700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560383297ef0 0x560383299d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603832aa850 msgr2=0x56038333fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603832aa850 0x56038333fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3707920418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560383297ef0 msgr2=0x560383299d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560383297ef0 0x560383299d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 -- 172.21.15.181:0/3707920418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603831fa280 con 0x560383323f50 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e4dde700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560383297ef0 0x560383299d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e3ddc700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603833225f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fa0cc001970 tx=0x7fa0cc014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e15d7700 1 -- 172.21.15.181:0/3707920418 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00d560 con 0x560383323f50 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e15d7700 1 -- 172.21.15.181:0/3707920418 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc0065d0 con 0x560383323f50 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5603831d8620 con 0x560383323f50 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0e15d7700 1 -- 172.21.15.181:0/3707920418 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc013ea0 con 0x560383323f50 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.098+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56038334feb0 con 0x560383323f50 2020-11-16T22:34:47.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e15d7700 1 -- 172.21.15.181:0/3707920418 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa0cc025030 con 0x560383323f50 2020-11-16T22:34:47.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e15d7700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0c4053d10 0x7fa0c40561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e15d7700 1 -- 172.21.15.181:0/3707920418 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0cc06df50 con 0x560383323f50 2020-11-16T22:34:47.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e45dd700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0c4053d10 0x7fa0c40561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x5603832b3710 0x56038331bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- osd_op(unknown.0.0:1 13.5 13:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x5603832ae770 con 0x5603832b3710 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e4dde700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x5603832b3710 0x56038331bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e45dd700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0c4053d10 0x7fa0c40561d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x560383246380 tx=0x7fa0d4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.102+0000 7fa0e4dde700 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x5603832b3710 0x56038331bcc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0e4dde700 1 -- 172.21.15.181:0/3707920418 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v97'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0dc0096e0 con 0x5603832b3710 2020-11-16T22:34:47.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x5603832b3710 msgr2=0x56038331bcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x5603832b3710 0x56038331bcc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:47.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0c4053d10 msgr2=0x7fa0c40561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa0c4053d10 0x7fa0c40561d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x560383246380 tx=0x7fa0d4015000).stop 2020-11-16T22:34:47.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 msgr2=0x5603833225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:47.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 --2- 172.21.15.181:0/3707920418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560383323f50 0x5603833225f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fa0cc001970 tx=0x7fa0cc014000).stop 2020-11-16T22:34:47.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0e4dde700 1 -- 172.21.15.181:0/3707920418 reap_dead start 2020-11-16T22:34:47.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 shutdown_connections 2020-11-16T22:34:47.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 >> 172.21.15.181:0/3707920418 conn(0x5603832b1400 msgr2=0x5603832a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:47.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 shutdown_connections 2020-11-16T22:34:47.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.106+0000 7fa0f2fd7c40 1 -- 172.21.15.181:0/3707920418 wait complete. 2020-11-16T22:34:47.907 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:34:47.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.950+0000 7f08abd8dc40 1 Processor -- start 2020-11-16T22:34:47.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.950+0000 7f08abd8dc40 1 -- start start 2020-11-16T22:34:47.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad5af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6403a0 0x5578ad5ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:47.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ad50a1f0 con 0x5578ad6403a0 2020-11-16T22:34:47.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ad4d8be0 con 0x5578ad63ffc0 2020-11-16T22:34:47.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578ad5ade70 con 0x5578ad6498c0 2020-11-16T22:34:47.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43334/0 (socket says 172.21.15.181:43334) 2020-11-16T22:34:48.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 -- 172.21.15.181:0/1351843208 learned_addr learned my addr 172.21.15.181:0/1351843208 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:48.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089d393700 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad5af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089cb92700 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6403a0 0x5578ad5ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 -- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 msgr2=0x5578ad5af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad5af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 -- 172.21.15.181:0/1351843208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6403a0 msgr2=0x5578ad5ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6403a0 0x5578ad5ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 -- 172.21.15.181:0/1351843208 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ad5028a0 con 0x5578ad6498c0 2020-11-16T22:34:48.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f089db94700 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f08980019a0 tx=0x7f0898006a00).ready entity=mon.2 client_cookie=904293fdc4fc20f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:48.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08977fe700 1 -- 172.21.15.181:0/1351843208 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980062e0 con 0x5578ad6498c0 2020-11-16T22:34:48.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08977fe700 1 -- 172.21.15.181:0/1351843208 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898012450 con 0x5578ad6498c0 2020-11-16T22:34:48.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08977fe700 1 -- 172.21.15.181:0/1351843208 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980127d0 con 0x5578ad6498c0 2020-11-16T22:34:48.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 msgr2=0x5578ad63cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f08980019a0 tx=0x7f0898006a00).stop 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- 172.21.15.181:0/1351843208 shutdown_connections 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6498c0 0x5578ad63cc90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad5af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/1351843208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6403a0 0x5578ad5ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.954+0000 7f08abd8dc40 1 -- 172.21.15.181:0/1351843208 >> 172.21.15.181:0/1351843208 conn(0x5578ad5b31d0 msgr2=0x5578ad5b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- 172.21.15.181:0/1351843208 shutdown_connections 2020-11-16T22:34:48.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- 172.21.15.181:0/1351843208 wait complete. 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 Processor -- start 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- start start 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad59b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 0x5578ad65a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6498c0 0x5578ad63b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ad657550 con 0x5578ad6498c0 2020-11-16T22:34:48.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ad656d70 con 0x5578ad63ffc0 2020-11-16T22:34:48.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f08abd8dc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578ad657010 con 0x5578ad6403a0 2020-11-16T22:34:48.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 0x5578ad65a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089d393700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad59b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 0x5578ad65a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43338/0 (socket says 172.21.15.181:43338) 2020-11-16T22:34:48.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 -- 172.21.15.181:0/2586376461 learned_addr learned my addr 172.21.15.181:0/2586376461 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089db94700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6498c0 0x5578ad63b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 -- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 msgr2=0x5578ad59b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5578ad63ffc0 0x5578ad59b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 -- 172.21.15.181:0/2586376461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6498c0 msgr2=0x5578ad63b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578ad6498c0 0x5578ad63b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.958+0000 7f089cb92700 1 -- 172.21.15.181:0/2586376461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ad4fe280 con 0x5578ad6403a0 2020-11-16T22:34:48.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f089cb92700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 0x5578ad65a530 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x5578ad564270 tx=0x7f088800c040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:48.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 -- 172.21.15.181:0/2586376461 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088800f070 con 0x5578ad6403a0 2020-11-16T22:34:48.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 -- 172.21.15.181:0/2586376461 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0888010040 con 0x5578ad6403a0 2020-11-16T22:34:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5578ad4dc620 con 0x5578ad6403a0 2020-11-16T22:34:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 -- 172.21.15.181:0/2586376461 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888006d00 con 0x5578ad6403a0 2020-11-16T22:34:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578ad623690 con 0x5578ad6403a0 2020-11-16T22:34:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 -- 172.21.15.181:0/2586376461 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0888015620 con 0x5578ad6403a0 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0880053ce0 0x7f08800561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f0895ffb700 1 -- 172.21.15.181:0/2586376461 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f088806ce70 con 0x5578ad6403a0 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.962+0000 7f089d393700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0880053ce0 0x7f08800561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.966+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x5578ad649e70 0x5578ad6242e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.966+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- osd_op(unknown.0.0:1 13.1 13:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x5578ad59b850 con 0x5578ad649e70 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.966+0000 7f089db94700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x5578ad649e70 0x5578ad6242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.966+0000 7f089d393700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0880053ce0 0x7f08800561a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f089000ae50 tx=0x7f0890015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.966+0000 7f089db94700 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x5578ad649e70 0x5578ad6242e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f089db94700 1 -- 172.21.15.181:0/2586376461 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v97'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0898001590 con 0x5578ad649e70 2020-11-16T22:34:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x5578ad649e70 msgr2=0x5578ad6242e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x5578ad649e70 0x5578ad6242e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0880053ce0 msgr2=0x7f08800561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0880053ce0 0x7f08800561a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f089000ae50 tx=0x7f0890015000).stop 2020-11-16T22:34:48.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 msgr2=0x5578ad65a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 --2- 172.21.15.181:0/2586376461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5578ad6403a0 0x5578ad65a530 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x5578ad564270 tx=0x7f088800c040).stop 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f089db94700 1 -- 172.21.15.181:0/2586376461 reap_dead start 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 shutdown_connections 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 >> 172.21.15.181:0/2586376461 conn(0x5578ad5b31d0 msgr2=0x5578ad5ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 shutdown_connections 2020-11-16T22:34:48.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:47.970+0000 7f08abd8dc40 1 -- 172.21.15.181:0/2586376461 wait complete. 2020-11-16T22:34:48.772 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:34:48.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 Processor -- start 2020-11-16T22:34:48.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 -- start start 2020-11-16T22:34:48.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092ae5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 0x560092a55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 0x560092b00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:48.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600929c61f0 con 0x560092a6ada0 2020-11-16T22:34:48.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560092994be0 con 0x560092afd880 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x560092a55930 con 0x560092afd4a0 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf7aed4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 0x560092a55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf7bed6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 0x560092b00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf7aed4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 0x560092a55350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43356/0 (socket says 172.21.15.181:43356) 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf7aed4700 1 -- 172.21.15.181:0/3323742468 learned_addr learned my addr 172.21.15.181:0/3323742468 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.818+0000 7fcf7b6d5700 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092ae5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:48.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 -- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 msgr2=0x560092a55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:48.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 0x560092a55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 -- 172.21.15.181:0/3323742468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 msgr2=0x560092ae5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:48.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092ae5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:48.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 -- 172.21.15.181:0/3323742468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600929be8a0 con 0x560092afd880 2020-11-16T22:34:48.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 0x560092b00cf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fcf74009b40 tx=0x7fcf7400c040).ready entity=mon.1 client_cookie=e4eed26ce2ba15d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:49.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf79ed2700 1 -- 172.21.15.181:0/3323742468 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400f040 con 0x560092afd880 2020-11-16T22:34:49.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf79ed2700 1 -- 172.21.15.181:0/3323742468 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf74008a50 con 0x560092afd880 2020-11-16T22:34:49.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf79ed2700 1 -- 172.21.15.181:0/3323742468 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74008dd0 con 0x560092afd880 2020-11-16T22:34:49.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 msgr2=0x560092b00cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 0x560092b00cf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fcf74009b40 tx=0x7fcf7400c040).stop 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/3323742468 shutdown_connections 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd880 0x560092b00cf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092ae5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/3323742468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd4a0 0x560092a55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/3323742468 >> 172.21.15.181:0/3323742468 conn(0x560092a713e0 msgr2=0x560092a53980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/3323742468 shutdown_connections 2020-11-16T22:34:49.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/3323742468 wait complete. 2020-11-16T22:34:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 Processor -- start 2020-11-16T22:34:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.822+0000 7fcf8a0cfc40 1 -- start start 2020-11-16T22:34:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092a73d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd4a0 0x560092a6a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 0x560092b0dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x560092a53cb0 con 0x560092a6ada0 2020-11-16T22:34:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560092b0e270 con 0x560092afd4a0 2020-11-16T22:34:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x560092aff680 con 0x560092afd880 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 0x560092b0dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 0x560092b0dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43362/0 (socket says 172.21.15.181:43362) 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 -- 172.21.15.181:0/198373993 learned_addr learned my addr 172.21.15.181:0/198373993 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7aed4700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd4a0 0x560092a6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7b6d5700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092a73d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 -- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd4a0 msgr2=0x560092a6a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560092afd4a0 0x560092a6a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 -- 172.21.15.181:0/198373993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 msgr2=0x560092a73d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560092a6ada0 0x560092a73d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 -- 172.21.15.181:0/198373993 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600929ba280 con 0x560092afd880 2020-11-16T22:34:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf7bed6700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 0x560092b0dd80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcf74014bc0 tx=0x7fcf74018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf73fff700 1 -- 172.21.15.181:0/198373993 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400f040 con 0x560092afd880 2020-11-16T22:34:49.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf73fff700 1 -- 172.21.15.181:0/198373993 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7401e460 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf73fff700 1 -- 172.21.15.181:0/198373993 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7401e7e0 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560092998620 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.826+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560092a53ea0 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf73fff700 1 -- 172.21.15.181:0/198373993 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcf7401ec10 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf73fff700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf5c053ce0 0x7fcf5c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf73fff700 1 -- 172.21.15.181:0/198373993 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf7406e750 con 0x560092afd880 2020-11-16T22:34:49.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf7b6d5700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf5c053ce0 0x7fcf5c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x560092ae5000 0x560092a698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- osd_op(unknown.0.0:1 13.5 13:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x560092a19040 con 0x560092ae5000 2020-11-16T22:34:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf7aed4700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x560092ae5000 0x560092a698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf7b6d5700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf5c053ce0 0x7fcf5c0561a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fcf64000f40 tx=0x7fcf64006880).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.830+0000 7fcf7aed4700 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x560092ae5000 0x560092a698a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf7aed4700 1 -- 172.21.15.181:0/198373993 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v97'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcf6c0054b0 con 0x560092ae5000 2020-11-16T22:34:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x560092ae5000 msgr2=0x560092a698a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x560092ae5000 0x560092a698a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf5c053ce0 msgr2=0x7fcf5c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf5c053ce0 0x7fcf5c0561a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fcf64000f40 tx=0x7fcf64006880).stop 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 msgr2=0x560092b0dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 --2- 172.21.15.181:0/198373993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560092afd880 0x560092b0dd80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fcf74014bc0 tx=0x7fcf74018000).stop 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf7bed6700 1 -- 172.21.15.181:0/198373993 reap_dead start 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 shutdown_connections 2020-11-16T22:34:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.834+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 >> 172.21.15.181:0/198373993 conn(0x560092a713e0 msgr2=0x560092b00c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.838+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 shutdown_connections 2020-11-16T22:34:49.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:48.838+0000 7fcf8a0cfc40 1 -- 172.21.15.181:0/198373993 wait complete. 2020-11-16T22:34:49.640 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:34:49.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 Processor -- start 2020-11-16T22:34:49.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 -- start start 2020-11-16T22:34:49.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 0x56189ab77a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189ab784a0 0x56189aaee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:49.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189aa411f0 con 0x56189ab784a0 2020-11-16T22:34:49.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189aa0fbe0 con 0x56189aaeba60 2020-11-16T22:34:49.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f476dd84c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56189ab77030 con 0x56189aaebec0 2020-11-16T22:34:49.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475eb89700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 0x56189ab77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475eb89700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 0x56189ab77a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43378/0 (socket says 172.21.15.181:43378) 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475eb89700 1 -- 172.21.15.181:0/50901277 learned_addr learned my addr 172.21.15.181:0/50901277 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35534/0 (socket says 172.21.15.181:35534) 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475fb8b700 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189ab784a0 0x56189aaee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 -- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 msgr2=0x56189ab77a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:49.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 0x56189ab77a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 -- 172.21.15.181:0/50901277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189ab784a0 msgr2=0x56189aaee6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189ab784a0 0x56189aaee6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 -- 172.21.15.181:0/50901277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56189aa398a0 con 0x56189aaeba60 2020-11-16T22:34:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.686+0000 7f475f38a700 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f4748012000 tx=0x7f4748015000).ready entity=mon.1 client_cookie=5233c580410c05c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f475db87700 1 -- 172.21.15.181:0/50901277 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800a5e0 con 0x56189aaeba60 2020-11-16T22:34:49.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f475db87700 1 -- 172.21.15.181:0/50901277 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748005420 con 0x56189aaeba60 2020-11-16T22:34:49.883 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f475db87700 1 -- 172.21.15.181:0/50901277 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748005590 con 0x56189aaeba60 2020-11-16T22:34:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 msgr2=0x56189aaea800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f4748012000 tx=0x7f4748015000).stop 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- 172.21.15.181:0/50901277 shutdown_connections 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189ab784a0 0x56189aaee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189aaea800 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 --2- 172.21.15.181:0/50901277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189aaebec0 0x56189ab77a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- 172.21.15.181:0/50901277 >> 172.21.15.181:0/50901277 conn(0x56189aaec3e0 msgr2=0x56189aaeb090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- 172.21.15.181:0/50901277 shutdown_connections 2020-11-16T22:34:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- 172.21.15.181:0/50901277 wait complete. 2020-11-16T22:34:50.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 Processor -- start 2020-11-16T22:34:50.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.690+0000 7f476dd84c40 1 -- start start 2020-11-16T22:34:50.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189ab75f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189aaebec0 0x56189aaeed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189ab784a0 0x56189ab89320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189aa953c0 con 0x56189aaebec0 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189aa9adc0 con 0x56189aaeba60 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56189aa9b060 con 0x56189ab784a0 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189ab75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475fb8b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189ab784a0 0x56189ab89320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189ab75f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35540/0 (socket says 172.21.15.181:35540) 2020-11-16T22:34:50.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 -- 172.21.15.181:0/2211257533 learned_addr learned my addr 172.21.15.181:0/2211257533 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:50.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475fb8b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189ab784a0 0x56189ab89320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43384/0 (socket says 172.21.15.181:43384) 2020-11-16T22:34:50.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475eb89700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189aaebec0 0x56189aaeed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 -- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189ab784a0 msgr2=0x56189ab89320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56189ab784a0 0x56189ab89320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 -- 172.21.15.181:0/2211257533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189aaebec0 msgr2=0x56189aaeed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189aaebec0 0x56189aaeed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 -- 172.21.15.181:0/2211257533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56189aa35280 con 0x56189aaeba60 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475eb89700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56189aaebec0 0x56189aaeed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f475f38a700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189ab75f50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f474801e040 tx=0x7f474801b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f4757fff700 1 -- 172.21.15.181:0/2211257533 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474801c870 con 0x56189aaeba60 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f4757fff700 1 -- 172.21.15.181:0/2211257533 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474801c9d0 con 0x56189aaeba60 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f4757fff700 1 -- 172.21.15.181:0/2211257533 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474801cd70 con 0x56189aaeba60 2020-11-16T22:34:50.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56189aa13620 con 0x56189aaeba60 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56189ab88e80 con 0x56189aaeba60 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.694+0000 7f4757fff700 1 -- 172.21.15.181:0/2211257533 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4748037070 con 0x56189aaeba60 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f4757fff700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4740053ce0 0x7f47400561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f4757fff700 1 -- 172.21.15.181:0/2211257533 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f474806ff80 con 0x56189aaeba60 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f475eb89700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4740053ce0 0x7f47400561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f476dd84c40 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x56189aaee410 0x56189ab95480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- osd_op(unknown.0.0:1 13.5 13:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56189ab771f0 con 0x56189aaee410 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f475eb89700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4740053ce0 0x7f47400561a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x56189aad07b0 tx=0x7f475000c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f475fb8b700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x56189aaee410 0x56189ab95480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.698+0000 7f475fb8b700 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x56189aaee410 0x56189ab95480 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f475fb8b700 1 -- 172.21.15.181:0/2211257533 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v97'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4758002580 con 0x56189aaee410 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x56189aaee410 msgr2=0x56189ab95480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x56189aaee410 0x56189ab95480 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4740053ce0 msgr2=0x7f47400561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4740053ce0 0x7f47400561a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x56189aad07b0 tx=0x7f475000c040).stop 2020-11-16T22:34:50.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 msgr2=0x56189ab75f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 --2- 172.21.15.181:0/2211257533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56189aaeba60 0x56189ab75f50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f474801e040 tx=0x7f474801b000).stop 2020-11-16T22:34:50.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f475fb8b700 1 -- 172.21.15.181:0/2211257533 reap_dead start 2020-11-16T22:34:50.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 shutdown_connections 2020-11-16T22:34:50.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 >> 172.21.15.181:0/2211257533 conn(0x56189aaec3e0 msgr2=0x56189aae4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:50.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 shutdown_connections 2020-11-16T22:34:50.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:49.702+0000 7f476dd84c40 1 -- 172.21.15.181:0/2211257533 wait complete. 2020-11-16T22:34:50.503 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:50.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 Processor -- start 2020-11-16T22:34:50.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 -- start start 2020-11-16T22:34:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 0x7f3398161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33980c85e0 0x7f3398160810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3398161a10 con 0x7f3398162bd0 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33980c8a40 con 0x7f3398160430 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339d127700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3398160d00 con 0x7f33980c85e0 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33980c85e0 0x7f3398160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33980c85e0 0x7f3398160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35552/0 (socket says 172.21.15.181:35552) 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 -- 172.21.15.181:0/3043312581 learned_addr learned my addr 172.21.15.181:0/3043312581 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f3396d9d700 1 --2- 172.21.15.181:0/3043312581 >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 0x7f3398161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 -- 172.21.15.181:0/3043312581 >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 msgr2=0x7f3398161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 --2- 172.21.15.181:0/3043312581 >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 0x7f3398161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 -- 172.21.15.181:0/3043312581 >> v1:172.21.15.175:6789/0 conn(0x7f3398162bd0 legacy=0x7f33980600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 -- 172.21.15.181:0/3043312581 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33980cf030 con 0x7f33980c85e0 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f3396d9d700 1 --2- 172.21.15.181:0/3043312581 >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 0x7f3398161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.642+0000 7f339659c700 1 --2- 172.21.15.181:0/3043312581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33980c85e0 0x7f3398160810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f339805d8f0 tx=0x7f3380009290).ready entity=mon.1 client_cookie=d7f118a9cb538d22 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3395d9b700 1 -- 172.21.15.181:0/3043312581 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000bbc0 con 0x7f33980c85e0 2020-11-16T22:34:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3395d9b700 1 -- 172.21.15.181:0/3043312581 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338000bd20 con 0x7f33980c85e0 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3395d9b700 1 -- 172.21.15.181:0/3043312581 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380011610 con 0x7f33980c85e0 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- 172.21.15.181:0/3043312581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33980c85e0 msgr2=0x7f3398160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- 172.21.15.181:0/3043312581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33980c85e0 0x7f3398160810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f339805d8f0 tx=0x7f3380009290).stop 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- 172.21.15.181:0/3043312581 shutdown_connections 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- 172.21.15.181:0/3043312581 >> v2:172.21.15.175:3300/0 conn(0x7f3398160430 0x7f3398161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- 172.21.15.181:0/3043312581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33980c85e0 0x7f3398160810 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- 172.21.15.181:0/3043312581 >> 172.21.15.181:0/3043312581 conn(0x7f33980d3680 msgr2=0x7f33980d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:50.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- 172.21.15.181:0/3043312581 shutdown_connections 2020-11-16T22:34:50.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- 172.21.15.181:0/3043312581 wait complete. 2020-11-16T22:34:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 Processor -- start 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- start start 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 0x7f33981419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980c9080 con 0x7f3398162bd0 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981636a0 con 0x7f3398160430 2020-11-16T22:34:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339d127700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3398163940 con 0x7f33980c85e0 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3396d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3396d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43398/0 (socket says 172.21.15.181:43398) 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35560/0 (socket says 172.21.15.181:35560) 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f3396d9d700 1 -- 172.21.15.181:0/2233140744 learned_addr learned my addr 172.21.15.181:0/2233140744 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.646+0000 7f339759e700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 0x7f33981419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 -- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 msgr2=0x7f3398162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 -- 172.21.15.181:0/2233140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 msgr2=0x7f33981419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 0x7f33981419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 -- 172.21.15.181:0/2233140744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398163be0 con 0x7f3398160430 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339759e700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 0x7f33981419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f3396d9d700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339659c700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f339805d8f0 tx=0x7f3380016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338001b5e0 con 0x7f3398160430 2020-11-16T22:34:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338001b740 con 0x7f3398160430 2020-11-16T22:34:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3398163e80 con 0x7f3398160430 2020-11-16T22:34:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338001bae0 con 0x7f3398160430 2020-11-16T22:34:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981643c0 con 0x7f3398160430 2020-11-16T22:34:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33981634d0 con 0x7f3398160430 2020-11-16T22:34:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3380017070 con 0x7f3398160430 2020-11-16T22:34:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 0x7f3374057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f338006fcd0 con 0x7f3398160430 2020-11-16T22:34:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.650+0000 7f3396d9d700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 0x7f3374057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.654+0000 7f3396d9d700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 0x7f3374057550 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f339000bbd0 tx=0x7f3390008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:50.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.662+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3380045fb0 con 0x7f3398160430 2020-11-16T22:34:50.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3398164550 con 0x7f3398160430 2020-11-16T22:34:50.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f338effd700 1 -- 172.21.15.181:0/2233140744 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4668 (secure 0 0 0) 0x7f3380015070 con 0x7f3398160430 2020-11-16T22:34:50.856 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:50.856 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:41.810527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:50.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 msgr2=0x7f3374057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 0x7f3374057550 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f339000bbd0 tx=0x7f3390008040).stop 2020-11-16T22:34:50.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 msgr2=0x7f33980c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:50.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f339805d8f0 tx=0x7f3380016000).stop 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 shutdown_connections 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3374055090 0x7f3374057550 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3398162bd0 0x7f33981419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33980c85e0 0x7f3398162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 --2- 172.21.15.181:0/2233140744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3398160430 0x7f33980c8b90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:50.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 >> 172.21.15.181:0/2233140744 conn(0x7f33980d3680 msgr2=0x7f339815f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:50.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 shutdown_connections 2020-11-16T22:34:50.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:50.866+0000 7f339d127700 1 -- 172.21.15.181:0/2233140744 wait complete. 2020-11-16T22:34:50.860 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 104 2020-11-16T22:34:50.871 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub start /d1/d2/d3 recursive force 2020-11-16T22:34:50.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.002+0000 7f51fec34700 1 Processor -- start 2020-11-16T22:34:50.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.002+0000 7f51fec34700 1 -- start start 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f51f8162aa0 0x7f51f8162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f80b14a0 con 0x7f51f80b23c0 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f80b1740 con 0x7f51f8162aa0 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80b19e0 con 0x7f51f8163ac0 2020-11-16T22:34:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fd1d1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f51f80b23c0 0x7f51f80600f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49872/0 (socket says 172.21.15.181:49872) 2020-11-16T22:34:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3655518750 learned_addr learned my addr 172.21.15.181:0/3655518750 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3655518750 >> v2:172.21.15.175:3300/0 conn(0x7f51f8162aa0 0x7f51f8162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f77fe700 1 -- 172.21.15.181:0/3655518750 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424741182 0 0) 0x7f51f80b14a0 con 0x7f51f80b23c0 2020-11-16T22:34:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 -- 172.21.15.181:0/3655518750 >> v2:172.21.15.175:3300/0 conn(0x7f51f8162aa0 msgr2=0x7f51f8162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 --2- 172.21.15.181:0/3655518750 >> v2:172.21.15.175:3300/0 conn(0x7f51f8162aa0 0x7f51f8162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 -- 172.21.15.181:0/3655518750 >> v1:172.21.15.181:6789/0 conn(0x7f51f80b23c0 legacy=0x7f51f80600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 -- 172.21.15.181:0/3655518750 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f80b1c80 con 0x7f51f8163ac0 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f7fff700 1 --2- 172.21.15.181:0/3655518750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8163f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f51e000b9a0 tx=0x7f51e0006df0).ready entity=mon.2 client_cookie=3164cbbbdb64ce4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f77fe700 1 -- 172.21.15.181:0/3655518750 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00097b0 con 0x7f51f8163ac0 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f77fe700 1 -- 172.21.15.181:0/3655518750 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0009910 con 0x7f51f8163ac0 2020-11-16T22:34:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51f77fe700 1 -- 172.21.15.181:0/3655518750 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0009c90 con 0x7f51f8163ac0 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- 172.21.15.181:0/3655518750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 msgr2=0x7f51f8163f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 --2- 172.21.15.181:0/3655518750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8163f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f51e000b9a0 tx=0x7f51e0006df0).stop 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- 172.21.15.181:0/3655518750 shutdown_connections 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 --2- 172.21.15.181:0/3655518750 >> v2:172.21.15.175:3300/0 conn(0x7f51f8162aa0 0x7f51f8162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 --2- 172.21.15.181:0/3655518750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8163f40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.006+0000 7f51fec34700 1 -- 172.21.15.181:0/3655518750 >> 172.21.15.181:0/3655518750 conn(0x7f51f80d35d0 msgr2=0x7f51f80d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.010+0000 7f51fec34700 1 -- 172.21.15.181:0/3655518750 shutdown_connections 2020-11-16T22:34:51.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.010+0000 7f51fec34700 1 -- 172.21.15.181:0/3655518750 wait complete. 2020-11-16T22:34:51.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.010+0000 7f51fec34700 1 Processor -- start 2020-11-16T22:34:51.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.010+0000 7f51fec34700 1 -- start start 2020-11-16T22:34:51.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f80b23c0 0x7f51f80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81430d0 con 0x7f51f80b23c0 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80cb920 con 0x7f51f8162aa0 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80cbbc0 con 0x7f51f8163ac0 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35572/0 (socket says 172.21.15.181:35572) 2020-11-16T22:34:51.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43416/0 (socket says 172.21.15.181:43416) 2020-11-16T22:34:51.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f7fff700 1 -- 172.21.15.181:0/3327859387 learned_addr learned my addr 172.21.15.181:0/3327859387 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f80b23c0 0x7f51f80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 msgr2=0x7f51f80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3327859387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f80b23c0 msgr2=0x7f51f80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f80b23c0 0x7f51f80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3327859387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f80cbe60 con 0x7f51f8163ac0 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f7fff700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f51f0007fd0 tx=0x7f51f0007c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f4ff9700 1 -- 172.21.15.181:0/3327859387 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f000ce50 con 0x7f51f8163ac0 2020-11-16T22:34:51.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f4ff9700 1 -- 172.21.15.181:0/3327859387 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f0005bd0 con 0x7f51f8163ac0 2020-11-16T22:34:51.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f80cc100 con 0x7f51f8163ac0 2020-11-16T22:34:51.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f4ff9700 1 -- 172.21.15.181:0/3327859387 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007110 con 0x7f51f8163ac0 2020-11-16T22:34:51.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f80cb610 con 0x7f51f8163ac0 2020-11-16T22:34:51.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.014+0000 7f51f4ff9700 1 -- 172.21.15.181:0/3327859387 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51f0010030 con 0x7f51f8163ac0 2020-11-16T22:34:51.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.018+0000 7f51f4ff9700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 0x7f51d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.018+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 0x7f51d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.018+0000 7f51f4ff9700 1 -- 172.21.15.181:0/3327859387 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51f006dba0 con 0x7f51f8163ac0 2020-11-16T22:34:51.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.018+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 0x7f51d4057520 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f51e8006fd0 tx=0x7f51e8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 Processor -- start 2020-11-16T22:34:51.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 -- start start 2020-11-16T22:34:51.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1545860 0x175cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x175e520 0x17b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ef560 con 0x175e520 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16a39e0 con 0x1545860 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15637a0 con 0x14d7c60 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43422/0 (socket says 172.21.15.181:43422) 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 -- 172.21.15.181:0/2439239930 learned_addr learned my addr 172.21.15.181:0/2439239930 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51fc9d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1545860 0x175cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x175e520 0x17b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 -- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1545860 msgr2=0x175cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1545860 0x175cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 -- 172.21.15.181:0/2439239930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x175e520 msgr2=0x17b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x175e520 0x17b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 -- 172.21.15.181:0/2439239930 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x155e650 con 0x14d7c60 2020-11-16T22:34:51.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51f7fff700 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f51f80b1b20 tx=0x7f51e0015000).ready entity=mon.2 client_cookie=8271878dcd30658 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51edffb700 1 -- 172.21.15.181:0/2439239930 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e000b6f0 con 0x14d7c60 2020-11-16T22:34:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51edffb700 1 -- 172.21.15.181:0/2439239930 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0009a80 con 0x14d7c60 2020-11-16T22:34:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.022+0000 7f51edffb700 1 -- 172.21.15.181:0/2439239930 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011c80 con 0x14d7c60 2020-11-16T22:34:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 msgr2=0x179b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f51f80b1b20 tx=0x7f51e0015000).stop 2020-11-16T22:34:51.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- 172.21.15.181:0/2439239930 shutdown_connections 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x175e520 0x17b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1545860 0x175cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- 172.21.15.181:0/2439239930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x179b5e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- 172.21.15.181:0/2439239930 >> 172.21.15.181:0/2439239930 conn(0x1713ef0 msgr2=0x17b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- 172.21.15.181:0/2439239930 shutdown_connections 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- 172.21.15.181:0/2439239930 wait complete. 2020-11-16T22:34:51.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 -- start start 2020-11-16T22:34:51.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 Processor -- start 2020-11-16T22:34:51.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17b8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.026+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 0x17b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1690790 con 0x1545860 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16908d0 con 0x175e520 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1691b40 con 0x14d7c60 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 10 client.0 ll_register_callbacks cb 0x1686b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 0x17b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43426/0 (socket says 172.21.15.181:43426) 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 -- 172.21.15.181:0/3921344995 learned_addr learned my addr 172.21.15.181:0/3921344995 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51f7fff700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 msgr2=0x17c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3921344995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 msgr2=0x17b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 -- 172.21.15.181:0/3921344995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1681330 con 0x175e520 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 0x17b8b00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f51f0000f40 tx=0x7f51f0001300).ready entity=mon.1 client_cookie=e4fa682753b1cd48 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0047080 con 0x175e520 2020-11-16T22:34:51.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51f00471e0 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1758ec0 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f0011570 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 10 client.7702 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x14b7080 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1758ec0 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f51f00438b0 con 0x175e520 2020-11-16T22:34:51.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 10 client.7702 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 10 client.7702 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 20 client.7702 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x17c3cf0 0x17c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 4 client.7702 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1581d00 con 0x17c3cf0 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x17c3cf0 0x17c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 10 client.7702 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x17c3cf0 0x17c65c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9377b95064722594 server_cookie=da282abe9d7f887a in_seq=0 out_seq=0 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 1 -- 172.21.15.181:0/3921344995 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1581d00 con 0x17c3cf0 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 10 client.7702 handle_command_reply: tid=0 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 client.7702 shutdown 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 msgr2=0x17b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 0x17b8b00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f51f0000f40 tx=0x7f51f0001300).stop 2020-11-16T22:34:51.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 shutdown_connections 2020-11-16T22:34:51.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x17c3cf0 0x17c65c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x175e520 0x17b8b00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 --2- 172.21.15.181:0/3921344995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f51c6ffd700 0 client.7702 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 >> 172.21.15.181:0/3921344995 conn(0x1713ef0 msgr2=0x179b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 shutdown_connections 2020-11-16T22:34:51.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.030+0000 7f520d6c5740 1 -- 172.21.15.181:0/3921344995 wait complete. 2020-11-16T22:34:51.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.034+0000 7f520d6c5740 20 client.7702 trim_cache size 0 max 16384 2020-11-16T22:34:51.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 Processor -- start 2020-11-16T22:34:51.042 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 -- start start 2020-11-16T22:34:51.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x1765260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.043 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x175ebd0 con 0x1545860 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1668ef0 con 0x1713ef0 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x179bcf0 con 0x14d7c60 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.050+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43434/0 (socket says 172.21.15.181:43434) 2020-11-16T22:34:51.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 -- 172.21.15.181:0/205339609 learned_addr learned my addr 172.21.15.181:0/205339609 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x1765260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 -- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 msgr2=0x1765260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x1765260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 -- 172.21.15.181:0/205339609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 msgr2=0x17c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 -- 172.21.15.181:0/205339609 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc0081d0 con 0x14d7c60 2020-11-16T22:34:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:51.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51f7fff700 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f51e0019000 tx=0x7f51e0021070).ready entity=mon.2 client_cookie=361e6055b01495ac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51ee7fc700 1 -- 172.21.15.181:0/205339609 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e002f070 con 0x14d7c60 2020-11-16T22:34:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51ee7fc700 1 -- 172.21.15.181:0/205339609 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0030040 con 0x14d7c60 2020-11-16T22:34:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f51ee7fc700 1 -- 172.21.15.181:0/205339609 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011c80 con 0x14d7c60 2020-11-16T22:34:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 msgr2=0x17c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f51e0019000 tx=0x7f51e0021070).stop 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- 172.21.15.181:0/205339609 shutdown_connections 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x1765260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 --2- 172.21.15.181:0/205339609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c24b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- 172.21.15.181:0/205339609 >> 172.21.15.181:0/205339609 conn(0x7f51cc002320 msgr2=0x1788630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- 172.21.15.181:0/205339609 shutdown_connections 2020-11-16T22:34:51.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- 172.21.15.181:0/205339609 wait complete. 2020-11-16T22:34:51.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 -- start start 2020-11-16T22:34:51.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.054+0000 7f520d6c5740 1 Processor -- start 2020-11-16T22:34:51.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x17b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1799900 con 0x1545860 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1799b30 con 0x1713ef0 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1799dd0 con 0x14d7c60 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43440/0 (socket says 172.21.15.181:43440) 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 10 client.0 ll_register_callbacks cb 0x179dcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:51.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 -- 172.21.15.181:0/4046321365 learned_addr learned my addr 172.21.15.181:0/4046321365 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fd1d1700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x17b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51f7fff700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 -- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 msgr2=0x17b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x17b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 -- 172.21.15.181:0/4046321365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 msgr2=0x17fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 -- 172.21.15.181:0/4046321365 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x14f57d0 con 0x14d7c60 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51fc9d0700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f51e8024ba0 tx=0x7f51e8026040).ready entity=mon.2 client_cookie=898e52e6979242c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8029040 con 0x14d7c60 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e0009a80 con 0x14d7c60 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51e802fc20 con 0x14d7c60 2020-11-16T22:34:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e8024e80 con 0x14d7c60 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 10 client.7064 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x1500eb0 con 0x14d7c60 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0009a80 con 0x14d7c60 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f51e8035020 con 0x14d7c60 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 10 client.7064 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 10 client.7064 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 20 client.7064 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x175e520 0x17b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 4 client.7064 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x1589000 con 0x175e520 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51f7fff700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x175e520 0x17b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.058+0000 7f51c6ffd700 10 client.7064 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51f7fff700 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x175e520 0x17b5420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b7a8e4f283fd0f2d server_cookie=5fbb1c3a6b898134 in_seq=0 out_seq=0 2020-11-16T22:34:51.058 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:34:51.065+0000 7fd16e0d0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1) n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@4},l=6914 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55923443f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.058 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:34:51.065+0000 7fd16e0d0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/AsLsXsFsx@2},l=6914 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5592345c8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51c6ffd700 1 -- 172.21.15.181:0/4046321365 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1589000 con 0x175e520 2020-11-16T22:34:51.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51c6ffd700 10 client.7064 handle_command_reply: tid=0 2020-11-16T22:34:51.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 client.7064 shutdown 2020-11-16T22:34:51.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 msgr2=0x17c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f51e8024ba0 tx=0x7f51e8026040).stop 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 shutdown_connections 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x175e520 0x17b5420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1713ef0 0x17b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x14d7c60 0x17c03d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 --2- 172.21.15.181:0/4046321365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1545860 0x17fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 >> 172.21.15.181:0/4046321365 conn(0x7f51cc002320 msgr2=0x1788630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51c6ffd700 0 client.7064 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 shutdown_connections 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 1 -- 172.21.15.181:0/4046321365 wait complete. 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f520d6c5740 20 client.7064 trim_cache size 0 max 16384 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 msgr2=0x7f51d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 0x7f51d4057520 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f51e8006fd0 tx=0x7f51e8006df0).stop 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 msgr2=0x7f51f8166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f51f0007fd0 tx=0x7f51f0007c50).stop 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 shutdown_connections 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f51d4055060 0x7f51d4057520 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51f8163ac0 0x7f51f8166950 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f80b23c0 0x7f51f80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 --2- 172.21.15.181:0/3327859387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8162aa0 0x7f51f80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 >> 172.21.15.181:0/3327859387 conn(0x7f51f80d35d0 msgr2=0x7f51f80d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 shutdown_connections 2020-11-16T22:34:51.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f51fec34700 1 -- 172.21.15.181:0/3327859387 wait complete. 2020-11-16T22:34:51.063 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f57517fb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1) n(v4 rc2020-11-16T22:34:21.372304+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@5},l=6914 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561470c0bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:34:21.372304+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.063 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f5757807700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1) n(v3 rc2020-11-16T22:34:21.372304+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6914=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561470c0c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:34:21.372304+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.063 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:51.063 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2020-11-16T22:34:51.064 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:34:51.064 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2020-11-16T22:34:51.064 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:51.064 INFO:tasks.ceph.mds.b.smithi181.stderr:2020-11-16T22:34:51.062+0000 7f4984ce9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1) n(v2 rc2020-11-16T22:34:21.372304+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@4},l=6914 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x560b0b97ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:34:21.372304+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.064 INFO:tasks.ceph.mds.b.smithi181.stderr:2020-11-16T22:34:51.066+0000 7f498acf5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1) n(v1 rc2020-11-16T22:34:21.372304+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6914=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x560b0b97f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:34:21.372304+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:34:51.071 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:51.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.202+0000 7f4059317700 1 Processor -- start 2020-11-16T22:34:51.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.202+0000 7f4059317700 1 -- start start 2020-11-16T22:34:51.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4054160690 0x7f4054066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405415e2e0 con 0x7f40540b2190 2020-11-16T22:34:51.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40540ce460 con 0x7f405415de80 2020-11-16T22:34:51.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4054066a60 con 0x7f4054160690 2020-11-16T22:34:51.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f40537fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f405415de80 0x7f405415ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49912/0 (socket says 172.21.15.181:49912) 2020-11-16T22:34:51.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f40537fe700 1 -- 172.21.15.181:0/1020601153 learned_addr learned my addr 172.21.15.181:0/1020601153 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:51.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609279591 0 0) 0x7f40540ce460 con 0x7f405415de80 2020-11-16T22:34:51.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4058b16700 1 --2- 172.21.15.181:0/1020601153 >> v2:172.21.15.175:3300/0 conn(0x7f4054160690 0x7f4054066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4034003240 con 0x7f405415de80 2020-11-16T22:34:51.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 575333147 0 0) 0x7f4034003240 con 0x7f405415de80 2020-11-16T22:34:51.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 >> v2:172.21.15.175:3300/0 conn(0x7f4054160690 msgr2=0x7f4054066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 --2- 172.21.15.181:0/1020601153 >> v2:172.21.15.175:3300/0 conn(0x7f4054160690 0x7f4054066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 >> v1:172.21.15.175:6789/0 conn(0x7f40540b2190 legacy=0x7f40540cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540ce700 con 0x7f405415de80 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f4044001860 con 0x7f405415de80 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 >> v1:172.21.15.181:6789/0 conn(0x7f405415de80 legacy=0x7f405415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 0x7f4034003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40340044e0 0x7f4034004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4034004e50 0x7f403400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054066a60 con 0x7f40340044e0 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034005300 con 0x7f4034004e50 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403400bc20 con 0x7f4034003780 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 0x7f4034003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4058b16700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4034004e50 0x7f403400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f40537fe700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40340044e0 0x7f4034004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 -- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4034004e50 msgr2=0x7f403400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4034004e50 0x7f403400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 -- 172.21.15.181:0/1020601153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40340044e0 msgr2=0x7f4034004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40340044e0 0x7f4034004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 -- 172.21.15.181:0/1020601153 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034003240 con 0x7f4034003780 2020-11-16T22:34:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4058b16700 1 -- 172.21.15.181:0/1020601153 reap_dead start 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4053fff700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 0x7f4034003ba0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f403c001770 tx=0x7f403c007b00).ready entity=mon.2 client_cookie=6dabcb349ca6536a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c007020 con 0x7f4034003780 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c007180 con 0x7f4034003780 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4052ffd700 1 -- 172.21.15.181:0/1020601153 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c00c5e0 con 0x7f4034003780 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 msgr2=0x7f4034003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 0x7f4034003ba0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f403c001770 tx=0x7f403c007b00).stop 2020-11-16T22:34:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- 172.21.15.181:0/1020601153 shutdown_connections 2020-11-16T22:34:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 --2- 172.21.15.181:0/1020601153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4034003780 0x7f4034003ba0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- 172.21.15.181:0/1020601153 >> 172.21.15.181:0/1020601153 conn(0x7f40540cfe90 msgr2=0x7f40540d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- 172.21.15.181:0/1020601153 shutdown_connections 2020-11-16T22:34:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.206+0000 7f4059317700 1 -- 172.21.15.181:0/1020601153 wait complete. 2020-11-16T22:34:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 Processor -- start 2020-11-16T22:34:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- start start 2020-11-16T22:34:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40541352f0 0x7f40540b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4054170820 0x7f4054061190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40541716c0 con 0x7f4054170820 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054171180 con 0x7f4054134f10 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4054171420 con 0x7f40541352f0 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35618/0 (socket says 172.21.15.181:35618) 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 -- 172.21.15.181:0/3407801477 learned_addr learned my addr 172.21.15.181:0/3407801477 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f40537fe700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40541352f0 0x7f40540b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4058b16700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4054170820 0x7f4054061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 -- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40541352f0 msgr2=0x7f40540b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40541352f0 0x7f40540b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 -- 172.21.15.181:0/3407801477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4054170820 msgr2=0x7f4054061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4054170820 0x7f4054061190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 -- 172.21.15.181:0/3407801477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405415c900 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4053fff700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f403c001740 tx=0x7f403c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c018030 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c006700 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405415cba0 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c006ac0 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.210+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4054135770 con 0x7f4054134f10 2020-11-16T22:34:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4054044f60 con 0x7f4054134f10 2020-11-16T22:34:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f403c026070 con 0x7f4054134f10 2020-11-16T22:34:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f404bfff700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 0x7f4030057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f403c06e400 con 0x7f4054134f10 2020-11-16T22:34:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f40537fe700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 0x7f4030057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.214+0000 7f40537fe700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 0x7f4030057520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f4044001990 tx=0x7f405415a510).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.218+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f403c044660 con 0x7f4054134f10 2020-11-16T22:34:51.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.418+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4054156d80 con 0x7f4054134f10 2020-11-16T22:34:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.418+0000 7f404bfff700 1 -- 172.21.15.181:0/3407801477 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4668 (secure 0 0 0) 0x7f403c001e60 con 0x7f4054134f10 2020-11-16T22:34:51.409 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:51.409 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:41.810527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:51.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.418+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 msgr2=0x7f4030057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.418+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 0x7f4030057520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f4044001990 tx=0x7f405415a510).stop 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 msgr2=0x7f405415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f403c001740 tx=0x7f403c015000).stop 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 shutdown_connections 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4030055060 0x7f4030057520 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4054170820 0x7f4054061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054134f10 0x7f405415d2e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 --2- 172.21.15.181:0/3407801477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40541352f0 0x7f40540b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 >> 172.21.15.181:0/3407801477 conn(0x7f40540cfe90 msgr2=0x7f40540d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 shutdown_connections 2020-11-16T22:34:51.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.422+0000 7f4059317700 1 -- 172.21.15.181:0/3407801477 wait complete. 2020-11-16T22:34:51.413 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 104 2020-11-16T22:34:51.428 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub pause 2020-11-16T22:34:51.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.562+0000 7f1b14949700 1 Processor -- start 2020-11-16T22:34:51.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.562+0000 7f1b14949700 1 -- start start 2020-11-16T22:34:51.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b14949700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b1015d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b14949700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100ce210 con 0x7f1b1015d080 2020-11-16T22:34:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b14949700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1015d950 con 0x7f1b100600f0 2020-11-16T22:34:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b14949700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10066a20 con 0x7f1b100cdd90 2020-11-16T22:34:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1b100600f0 0x7f1b10060580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49934/0 (socket says 172.21.15.181:49934) 2020-11-16T22:34:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0ffff700 1 -- 172.21.15.181:0/2994858186 learned_addr learned my addr 172.21.15.181:0/2994858186 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0effd700 1 --2- 172.21.15.181:0/2994858186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b1015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124308588 0 0) 0x7f1b1015d950 con 0x7f1b100600f0 2020-11-16T22:34:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1afc003240 con 0x7f1b100600f0 2020-11-16T22:34:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 643286413 0 0) 0x7f1afc003240 con 0x7f1b100600f0 2020-11-16T22:34:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0effd700 1 -- 172.21.15.181:0/2994858186 >> v1:172.21.15.181:6789/0 conn(0x7f1b100600f0 legacy=0x7f1b10060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0effd700 1 -- 172.21.15.181:0/2994858186 >> v1:172.21.15.175:6789/0 conn(0x7f1b1015d080 legacy=0x7f1b10066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0effd700 1 -- 172.21.15.181:0/2994858186 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10065150 con 0x7f1b100cdd90 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0effd700 1 --2- 172.21.15.181:0/2994858186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b1015d460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f1b0400b950 tx=0x7f1b04006df0).ready entity=mon.2 client_cookie=45bb19bfe643bdfa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08001860 con 0x7f1b100cdd90 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04012460 con 0x7f1b100cdd90 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.566+0000 7f1b0e7fc700 1 -- 172.21.15.181:0/2994858186 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b040127e0 con 0x7f1b100cdd90 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- 172.21.15.181:0/2994858186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 msgr2=0x7f1b1015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 --2- 172.21.15.181:0/2994858186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b1015d460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f1b0400b950 tx=0x7f1b04006df0).stop 2020-11-16T22:34:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- 172.21.15.181:0/2994858186 shutdown_connections 2020-11-16T22:34:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 --2- 172.21.15.181:0/2994858186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b1015d460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- 172.21.15.181:0/2994858186 >> 172.21.15.181:0/2994858186 conn(0x7f1b100cfe90 msgr2=0x7f1b100d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- 172.21.15.181:0/2994858186 shutdown_connections 2020-11-16T22:34:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- 172.21.15.181:0/2994858186 wait complete. 2020-11-16T22:34:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 Processor -- start 2020-11-16T22:34:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.570+0000 7f1b14949700 1 -- start start 2020-11-16T22:34:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b100600f0 0x7f1b1016f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100b6380 con 0x7f1b100600f0 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10062610 con 0x7f1b1015d080 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1016f970 con 0x7f1b100cdd90 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43476/0 (socket says 172.21.15.181:43476) 2020-11-16T22:34:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35636/0 (socket says 172.21.15.181:35636) 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0effd700 1 -- 172.21.15.181:0/3159544579 learned_addr learned my addr 172.21.15.181:0/3159544579 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b100600f0 0x7f1b1016f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 msgr2=0x7f1b100b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3159544579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b100600f0 msgr2=0x7f1b1016f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b100600f0 0x7f1b1016f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3159544579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1016fc10 con 0x7f1b1015d080 2020-11-16T22:34:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f1b0800cc60 tx=0x7f1b08002190).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1aef7fe700 1 -- 172.21.15.181:0/3159544579 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08004a90 con 0x7f1b1015d080 2020-11-16T22:34:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1016feb0 con 0x7f1b1015d080 2020-11-16T22:34:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1aef7fe700 1 -- 172.21.15.181:0/3159544579 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08004bf0 con 0x7f1b1015d080 2020-11-16T22:34:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1aef7fe700 1 -- 172.21.15.181:0/3159544579 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08007170 con 0x7f1b1015d080 2020-11-16T22:34:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.574+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b10062840 con 0x7f1b1015d080 2020-11-16T22:34:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.578+0000 7f1aef7fe700 1 -- 172.21.15.181:0/3159544579 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1b08010070 con 0x7f1b1015d080 2020-11-16T22:34:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.578+0000 7f1aef7fe700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 0x7f1ae8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.578+0000 7f1aef7fe700 1 -- 172.21.15.181:0/3159544579 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b0806d9f0 con 0x7f1b1015d080 2020-11-16T22:34:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.578+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 0x7f1ae8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.578+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 0x7f1ae8057520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f1b00000f20 tx=0x7f1b000019d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 Processor -- start 2020-11-16T22:34:51.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- start start 2020-11-16T22:34:51.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1586710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x1584660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x140f4a0 con 0x13657a0 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14c3920 con 0x12f7ba0 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13836e0 con 0x1583fc0 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43484/0 (socket says 172.21.15.181:43484) 2020-11-16T22:34:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3108035813 learned_addr learned my addr 172.21.15.181:0/3108035813 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x1584660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1586710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 msgr2=0x1584660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x1584660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3108035813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 msgr2=0x1586710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1586710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 -- 172.21.15.181:0/3108035813 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x137e590 con 0x1583fc0 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f1b08047840 tx=0x7f1b08005000).ready entity=mon.2 client_cookie=9a653e267014148b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1aecff9700 1 -- 172.21.15.181:0/3108035813 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0804e070 con 0x1583fc0 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1aecff9700 1 -- 172.21.15.181:0/3108035813 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08013cd0 con 0x1583fc0 2020-11-16T22:34:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1aecff9700 1 -- 172.21.15.181:0/3108035813 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08047010 con 0x1583fc0 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 msgr2=0x157d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f1b08047840 tx=0x7f1b08005000).stop 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- 172.21.15.181:0/3108035813 shutdown_connections 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1583fc0 0x157d6f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1586710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3108035813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x1584660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- 172.21.15.181:0/3108035813 >> 172.21.15.181:0/3108035813 conn(0x1533e30 msgr2=0x1580390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- 172.21.15.181:0/3108035813 shutdown_connections 2020-11-16T22:34:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.582+0000 7f1b2463c740 1 -- 172.21.15.181:0/3108035813 wait complete. 2020-11-16T22:34:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- start start 2020-11-16T22:34:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 Processor -- start 2020-11-16T22:34:51.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1583fc0 0x15800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x14b06d0 con 0x1583fc0 2020-11-16T22:34:51.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14b0810 con 0x13657a0 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14b1a80 con 0x12f7ba0 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 10 client.0 ll_register_callbacks cb 0x14a6ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43490/0 (socket says 172.21.15.181:43490) 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3310318579 learned_addr learned my addr 172.21.15.181:0/3310318579 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1583fc0 0x15800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 -- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 msgr2=0x15aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 -- 172.21.15.181:0/3310318579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1583fc0 msgr2=0x15800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1583fc0 0x15800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 -- 172.21.15.181:0/3310318579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x14a1270 con 0x13657a0 2020-11-16T22:34:51.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15d7f80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f1b0400b150 tx=0x7f1b100b3710).ready entity=mon.1 client_cookie=b2cff265192884cd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04021440 con 0x13657a0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b04018070 con 0x13657a0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15646b0 con 0x13657a0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b040217b0 con 0x13657a0 2020-11-16T22:34:51.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 10 client.7726 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x12d6fc0 con 0x13657a0 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15646b0 con 0x13657a0 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1b04021910 con 0x13657a0 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 10 client.7726 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 10 client.7726 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:51.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 20 client.7726 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x15d8470 0x1581910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 4 client.7726 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x13a1c40 con 0x15d8470 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.586+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x15d8470 0x1581910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1ae5ffb700 10 client.7726 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x15d8470 0x1581910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5e00a845024d59f7 server_cookie=c428c93812c52be3 in_seq=0 out_seq=0 2020-11-16T22:34:51.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3310318579 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x13a1c40 con 0x15d8470 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1ae5ffb700 10 client.7726 handle_command_reply: tid=0 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 client.7726 shutdown 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 msgr2=0x15d7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15d7f80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f1b0400b150 tx=0x7f1b100b3710).stop 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 shutdown_connections 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x15d8470 0x1581910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1583fc0 0x15800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12f7ba0 0x15aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1ae5ffb700 0 client.7726 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3310318579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15d7f80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 >> 172.21.15.181:0/3310318579 conn(0x1533e30 msgr2=0x15b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 shutdown_connections 2020-11-16T22:34:51.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 1 -- 172.21.15.181:0/3310318579 wait complete. 2020-11-16T22:34:51.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.590+0000 7f1b2463c740 20 client.7726 trim_cache size 0 max 16384 2020-11-16T22:34:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.606+0000 7f1b2463c740 1 Processor -- start 2020-11-16T22:34:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.606+0000 7f1b2463c740 1 -- start start 2020-11-16T22:34:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 0x15b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1488e30 con 0x12f7ba0 2020-11-16T22:34:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1583900 con 0x13657a0 2020-11-16T22:34:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15f3a50 con 0x1533e30 2020-11-16T22:34:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43498/0 (socket says 172.21.15.181:43498) 2020-11-16T22:34:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3025067832 learned_addr learned my addr 172.21.15.181:0/3025067832 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 0x15b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 msgr2=0x15b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3025067832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 msgr2=0x15b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 0x15b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3025067832 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0081d0 con 0x1533e30 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 0x15b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f1b00006b90 tx=0x7f1b00019040).ready entity=mon.2 client_cookie=abe51b006d0dcd28 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1aed7fa700 1 -- 172.21.15.181:0/3025067832 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0001c040 con 0x1533e30 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1aed7fa700 1 -- 172.21.15.181:0/3025067832 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00021b20 con 0x1533e30 2020-11-16T22:34:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1aed7fa700 1 -- 172.21.15.181:0/3025067832 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00021ea0 con 0x1533e30 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 msgr2=0x15d8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f1b00006b90 tx=0x7f1b00019040).stop 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- 172.21.15.181:0/3025067832 shutdown_connections 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13657a0 0x15b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d8850 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3025067832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12f7ba0 0x15b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- 172.21.15.181:0/3025067832 >> 172.21.15.181:0/3025067832 conn(0x7f1adc002320 msgr2=0x1578790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- 172.21.15.181:0/3025067832 shutdown_connections 2020-11-16T22:34:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- 172.21.15.181:0/3025067832 wait complete. 2020-11-16T22:34:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.610+0000 7f1b2463c740 1 -- start start 2020-11-16T22:34:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 Processor -- start 2020-11-16T22:34:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1632bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1636230 con 0x13657a0 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x16364d0 con 0x12f7ba0 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1580f00 con 0x1533e30 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35660/0 (socket says 172.21.15.181:35660) 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3850280941 learned_addr learned my addr 172.21.15.181:0/3850280941 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 10 client.0 ll_register_callbacks cb 0x1584d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0ffff700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1632bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 msgr2=0x15d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3850280941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 msgr2=0x1632bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1632bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 -- 172.21.15.181:0/3850280941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1315710 con 0x12f7ba0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b0f7fe700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f1b00006fd0 tx=0x7f1b00014000).ready entity=mon.1 client_cookie=29bd42f7e9faf1cd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1ae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0001c040 con 0x12f7ba0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b00016810 con 0x12f7ba0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b08013cd0 con 0x12f7ba0 2020-11-16T22:34:51.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b00016be0 con 0x12f7ba0 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 10 client.7732 fetch_fsmap learned FSMap version 104 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.614+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x1320df0 con 0x12f7ba0 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08013cd0 con 0x12f7ba0 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 104) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1b00002d30 con 0x12f7ba0 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 10 client.7732 fetch_fsmap finished waiting for FSMap version 104 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 10 client.7732 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 20 client.7732 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1583fc0 0x157d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 4 client.7732 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0x13a8f40 con 0x1583fc0 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1583fc0 0x157d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 10 client.7732 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b0effd700 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1583fc0 0x157d6f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d128e203e17cc831 server_cookie=3c9aa2d5260713ae in_seq=0 out_seq=0 2020-11-16T22:34:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 1 -- 172.21.15.181:0/3850280941 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x13a8f40 con 0x1583fc0 2020-11-16T22:34:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 10 client.7732 handle_command_reply: tid=0 2020-11-16T22:34:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 client.7732 shutdown 2020-11-16T22:34:51.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 msgr2=0x15cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f1b00006fd0 tx=0x7f1b00014000).stop 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 shutdown_connections 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1583fc0 0x157d6f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1533e30 0x15d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1ae5ffb700 0 client.7732 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:51.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12f7ba0 0x15cd780 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 --2- 172.21.15.181:0/3850280941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13657a0 0x1632bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 >> 172.21.15.181:0/3850280941 conn(0x7f1adc002320 msgr2=0x1578790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 shutdown_connections 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 1 -- 172.21.15.181:0/3850280941 wait complete. 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b2463c740 20 client.7732 trim_cache size 0 max 16384 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 msgr2=0x7f1ae8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 0x7f1ae8057520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f1b00000f20 tx=0x7f1b000019d0).stop 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 msgr2=0x7f1b100b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f1b0800cc60 tx=0x7f1b08002190).stop 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 shutdown_connections 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1ae8055060 0x7f1ae8057520 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1015d080 0x7f1b100b7da0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b100600f0 0x7f1b1016f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 --2- 172.21.15.181:0/3159544579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b100cdd90 0x7f1b100b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 >> 172.21.15.181:0/3159544579 conn(0x7f1b100cfe90 msgr2=0x7f1b10066230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 shutdown_connections 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.618+0000 7f1b14949700 1 -- 172.21.15.181:0/3159544579 wait complete. 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:34:51.619 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:51.625 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:51.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.754+0000 7fc296d04700 1 Processor -- start 2020-11-16T22:34:51.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.754+0000 7fc296d04700 1 -- start start 2020-11-16T22:34:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc2900cac00 0x7fc2900cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 0x7fc2900c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc290162500 con 0x7fc290160780 2020-11-16T22:34:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2901627a0 con 0x7fc2900cac00 2020-11-16T22:34:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc290162a40 con 0x7fc290066c40 2020-11-16T22:34:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 0x7fc2900c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc295501700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc290160780 0x7fc290066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49972/0 (socket says 172.21.15.181:49972) 2020-11-16T22:34:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc295501700 1 -- 172.21.15.181:0/3341213973 learned_addr learned my addr 172.21.15.181:0/3341213973 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc295d02700 1 --2- 172.21.15.181:0/3341213973 >> v2:172.21.15.175:3300/0 conn(0x7fc2900cac00 0x7fc2900cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 -- 172.21.15.181:0/3341213973 >> v2:172.21.15.175:3300/0 conn(0x7fc2900cac00 msgr2=0x7fc2900cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 --2- 172.21.15.181:0/3341213973 >> v2:172.21.15.175:3300/0 conn(0x7fc2900cac00 0x7fc2900cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 -- 172.21.15.181:0/3341213973 >> v1:172.21.15.181:6789/0 conn(0x7fc290160780 legacy=0x7fc290066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc294d00700 1 -- 172.21.15.181:0/3341213973 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221227026 0 0) 0x7fc290162500 con 0x7fc290160780 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 -- 172.21.15.181:0/3341213973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290162ce0 con 0x7fc290066c40 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296503700 1 --2- 172.21.15.181:0/3341213973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 0x7fc2900c9790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fc288009c90 tx=0x7fc2880091d0).ready entity=mon.2 client_cookie=5b511b33e65edd4c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc294d00700 1 -- 172.21.15.181:0/3341213973 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28800bbc0 con 0x7fc290066c40 2020-11-16T22:34:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc294d00700 1 -- 172.21.15.181:0/3341213973 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28800bd20 con 0x7fc290066c40 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc294d00700 1 -- 172.21.15.181:0/3341213973 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288011610 con 0x7fc290066c40 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- 172.21.15.181:0/3341213973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 msgr2=0x7fc2900c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 --2- 172.21.15.181:0/3341213973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 0x7fc2900c9790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fc288009c90 tx=0x7fc2880091d0).stop 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- 172.21.15.181:0/3341213973 shutdown_connections 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 --2- 172.21.15.181:0/3341213973 >> v2:172.21.15.175:3300/0 conn(0x7fc2900cac00 0x7fc2900cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 --2- 172.21.15.181:0/3341213973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290066c40 0x7fc2900c9790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- 172.21.15.181:0/3341213973 >> 172.21.15.181:0/3341213973 conn(0x7fc2900d4590 msgr2=0x7fc2900d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- 172.21.15.181:0/3341213973 shutdown_connections 2020-11-16T22:34:51.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.758+0000 7fc296d04700 1 -- 172.21.15.181:0/3341213973 wait complete. 2020-11-16T22:34:51.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 Processor -- start 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- start start 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc290066c40 0x7fc290164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2900cac00 0x7fc290164f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900ca670 con 0x7fc290066c40 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc290170070 con 0x7fc2900cac00 2020-11-16T22:34:51.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900c9750 con 0x7fc290160780 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43516/0 (socket says 172.21.15.181:43516) 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 -- 172.21.15.181:0/1221468867 learned_addr learned my addr 172.21.15.181:0/1221468867 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc295501700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2900cac00 0x7fc290164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc295d02700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc290066c40 0x7fc290164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 -- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2900cac00 msgr2=0x7fc290164f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2900cac00 0x7fc290164f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 -- 172.21.15.181:0/1221468867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc290066c40 msgr2=0x7fc290164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc290066c40 0x7fc290164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 -- 172.21.15.181:0/1221468867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900c99f0 con 0x7fc290160780 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296503700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fc288009210 tx=0x7fc288016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28800be90 con 0x7fc290160780 2020-11-16T22:34:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc288011e20 con 0x7fc290160780 2020-11-16T22:34:51.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288010df0 con 0x7fc290160780 2020-11-16T22:34:51.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900c9c90 con 0x7fc290160780 2020-11-16T22:34:51.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2900ca1d0 con 0x7fc290160780 2020-11-16T22:34:51.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc288017070 con 0x7fc290160780 2020-11-16T22:34:51.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.762+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2900ca390 con 0x7fc290160780 2020-11-16T22:34:51.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.766+0000 7fc27dffb700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 0x7fc268057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:51.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.766+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc2880703f0 con 0x7fc290160780 2020-11-16T22:34:51.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.766+0000 7fc295d02700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 0x7fc268057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:51.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.766+0000 7fc295d02700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 0x7fc268057520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fc28c00b500 tx=0x7fc28c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:51.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.770+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc288046650 con 0x7fc290160780 2020-11-16T22:34:51.884 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:34:51.893+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:34:51.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.974+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc290044f60 con 0x7fc290160780 2020-11-16T22:34:51.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc27dffb700 1 -- 172.21.15.181:0/1221468867 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4669 (secure 0 0 0) 0x7fc288015020 con 0x7fc290160780 2020-11-16T22:34:51.967 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:51.967 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:51.894687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:51.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 msgr2=0x7fc268057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 0x7fc268057520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fc28c00b500 tx=0x7fc28c008040).stop 2020-11-16T22:34:51.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 msgr2=0x7fc2900d37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:51.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fc288009210 tx=0x7fc288016000).stop 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 shutdown_connections 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc268055060 0x7fc268057520 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290160780 0x7fc2900d37b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc290066c40 0x7fc290164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 --2- 172.21.15.181:0/1221468867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2900cac00 0x7fc290164f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 >> 172.21.15.181:0/1221468867 conn(0x7fc2900d4590 msgr2=0x7fc290063ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:51.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 shutdown_connections 2020-11-16T22:34:51.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:51.978+0000 7fc296d04700 1 -- 172.21.15.181:0/1221468867 wait complete. 2020-11-16T22:34:51.971 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 105 2020-11-16T22:34:51.981 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:34:52.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.114+0000 7f2d9b59e700 1 Processor -- start 2020-11-16T22:34:52.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.114+0000 7f2d9b59e700 1 -- start start 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d9c161480 0x7f2d9c164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2d9c164e10 0x7f2d9c062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c0c8db0 con 0x7f2d9c161860 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c0c8330 con 0x7f2d9c164e10 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c0c85d0 con 0x7f2d9c161480 2020-11-16T22:34:52.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d9c161480 0x7f2d9c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2d9c161860 0x7f2d9c062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49988/0 (socket says 172.21.15.181:49988) 2020-11-16T22:34:52.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/2971108819 learned_addr learned my addr 172.21.15.181:0/2971108819 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:52.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/2971108819 >> v2:172.21.15.175:3300/0 conn(0x7f2d9c164e10 0x7f2d9c062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 -- 172.21.15.181:0/2971108819 >> v2:172.21.15.175:3300/0 conn(0x7f2d9c164e10 msgr2=0x7f2d9c062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/2971108819 >> v2:172.21.15.175:3300/0 conn(0x7f2d9c164e10 0x7f2d9c062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 -- 172.21.15.181:0/2971108819 >> v1:172.21.15.181:6789/0 conn(0x7f2d9c161860 legacy=0x7f2d9c062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 -- 172.21.15.181:0/2971108819 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c0c8870 con 0x7f2d9c161480 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/2971108819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c164920 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2d9c0c8260 tx=0x7f2d94006df0).ready entity=mon.1 client_cookie=9604a6dc3ebd76aa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9959a700 1 -- 172.21.15.181:0/2971108819 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940097b0 con 0x7f2d9c161480 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9959a700 1 -- 172.21.15.181:0/2971108819 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94009910 con 0x7f2d9c161480 2020-11-16T22:34:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9959a700 1 -- 172.21.15.181:0/2971108819 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94009c90 con 0x7f2d9c161480 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- 172.21.15.181:0/2971108819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 msgr2=0x7f2d9c164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/2971108819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c164920 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2d9c0c8260 tx=0x7f2d94006df0).stop 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- 172.21.15.181:0/2971108819 shutdown_connections 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/2971108819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c164920 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/2971108819 >> v2:172.21.15.175:3300/0 conn(0x7f2d9c164e10 0x7f2d9c062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- 172.21.15.181:0/2971108819 >> 172.21.15.181:0/2971108819 conn(0x7f2d9c0d2dc0 msgr2=0x7f2d9c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- 172.21.15.181:0/2971108819 shutdown_connections 2020-11-16T22:34:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.118+0000 7f2d9b59e700 1 -- 172.21.15.181:0/2971108819 wait complete. 2020-11-16T22:34:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 Processor -- start 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- start start 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 0x7f2d9c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d9c0b7430 0x7f2d9c15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c174f20 con 0x7f2d9c0b7430 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15de20 con 0x7f2d9c161480 2020-11-16T22:34:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15e0c0 con 0x7f2d9c161860 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 0x7f2d9c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35688/0 (socket says 172.21.15.181:35688) 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 0x7f2d9c0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43532/0 (socket says 172.21.15.181:43532) 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 -- 172.21.15.181:0/1886618396 learned_addr learned my addr 172.21.15.181:0/1886618396 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d9c0b7430 0x7f2d9c15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 -- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 msgr2=0x7f2d9c0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 0x7f2d9c0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 -- 172.21.15.181:0/1886618396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d9c0b7430 msgr2=0x7f2d9c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d9c0b7430 0x7f2d9c15f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 -- 172.21.15.181:0/1886618396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c15e360 con 0x7f2d9c161480 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f2d9400be30 tx=0x7f2d94015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d927fc700 1 -- 172.21.15.181:0/1886618396 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94011dd0 con 0x7f2d9c161480 2020-11-16T22:34:52.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d927fc700 1 -- 172.21.15.181:0/1886618396 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9401e460 con 0x7f2d9c161480 2020-11-16T22:34:52.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c15e600 con 0x7f2d9c161480 2020-11-16T22:34:52.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d927fc700 1 -- 172.21.15.181:0/1886618396 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9401e800 con 0x7f2d9c161480 2020-11-16T22:34:52.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.122+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c15dae0 con 0x7f2d9c161480 2020-11-16T22:34:52.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.126+0000 7f2d927fc700 1 -- 172.21.15.181:0/1886618396 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2d94025030 con 0x7f2d9c161480 2020-11-16T22:34:52.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.126+0000 7f2d927fc700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 0x7f2d6c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.126+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 0x7f2d6c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.126+0000 7f2d927fc700 1 -- 172.21.15.181:0/1886618396 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d9406f540 con 0x7f2d9c161480 2020-11-16T22:34:52.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.126+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 0x7f2d6c057520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f2d9c162f80 tx=0x7f2d84006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 Processor -- start 2020-11-16T22:34:52.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 -- start start 2020-11-16T22:34:52.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 0x1108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 0x1106670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 0x10ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf914b0 con 0xee77b0 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1045930 con 0xe79bb0 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf056f0 con 0x1105fd0 2020-11-16T22:34:52.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2d9ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 0x10ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 0x1106670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2d9ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 0x10ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43542/0 (socket says 172.21.15.181:43542) 2020-11-16T22:34:52.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/2628368371 learned_addr learned my addr 172.21.15.181:0/2628368371 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.130+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 0x1108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 -- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 msgr2=0x10ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 0x10ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 -- 172.21.15.181:0/2628368371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 msgr2=0x1108720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 0x1108720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 -- 172.21.15.181:0/2628368371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xf005a0 con 0xe79bb0 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 0x1108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:52.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 0x1106670 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f2d84006fd0 tx=0x7f2d84013000).ready entity=mon.1 client_cookie=25f7b23bb9cea1f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d8b7fe700 1 -- 172.21.15.181:0/2628368371 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84014020 con 0xe79bb0 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d8b7fe700 1 -- 172.21.15.181:0/2628368371 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84014180 con 0xe79bb0 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2d8b7fe700 1 -- 172.21.15.181:0/2628368371 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011c60 con 0xe79bb0 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 msgr2=0x1106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 0x1106670 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f2d84006fd0 tx=0x7f2d84013000).stop 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- 172.21.15.181:0/2628368371 shutdown_connections 2020-11-16T22:34:52.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1105fd0 0x10ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xee77b0 0x1108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 --2- 172.21.15.181:0/2628368371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xe79bb0 0x1106670 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- 172.21.15.181:0/2628368371 >> 172.21.15.181:0/2628368371 conn(0x10b5e40 msgr2=0x11023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- 172.21.15.181:0/2628368371 shutdown_connections 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- 172.21.15.181:0/2628368371 wait complete. 2020-11-16T22:34:52.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 -- start start 2020-11-16T22:34:52.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.134+0000 7f2daf929740 1 Processor -- start 2020-11-16T22:34:52.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x11653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1105fd0 0x11658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x10326e0 con 0x1105fd0 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1032820 con 0xee77b0 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1033a90 con 0xe79bb0 2020-11-16T22:34:52.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x11653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43544/0 (socket says 172.21.15.181:43544) 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x11653b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35704/0 (socket says 172.21.15.181:35704) 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 10 client.0 ll_register_callbacks cb 0x1028ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 -- 172.21.15.181:0/559712429 learned_addr learned my addr 172.21.15.181:0/559712429 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1105fd0 0x11658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 -- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 msgr2=0x11653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x11653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 -- 172.21.15.181:0/559712429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1105fd0 msgr2=0x11658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1105fd0 0x11658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 -- 172.21.15.181:0/559712429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1023280 con 0xe79bb0 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2d94045940 tx=0x7f2d9406f040).ready entity=mon.2 client_cookie=ea62f0f0e1e8bb39 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94048b80 con 0xe79bb0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d9406f2c0 con 0xe79bb0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10e66c0 con 0xe79bb0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d94045440 con 0xe79bb0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 10 client.7100 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0xe58fd0 con 0xe79bb0 2020-11-16T22:34:52.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10e66c0 con 0xe79bb0 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2d940483a0 con 0xe79bb0 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 10 client.7100 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 10 client.7100 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 20 client.7100 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1153d50 0x11256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 4 client.7100 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xf23c50 con 0x1153d50 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1153d50 0x11256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 10 client.7100 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1153d50 0x11256c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=ab05fbbd7f8f79e5 server_cookie=418696914ff502b9 in_seq=0 out_seq=0 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 1 -- 172.21.15.181:0/559712429 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xf23c50 con 0x1153d50 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2d88ff9700 10 client.7100 handle_command_reply: tid=0 2020-11-16T22:34:52.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.138+0000 7f2daf929740 1 client.7100 shutdown 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 msgr2=0x1102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2d94045940 tx=0x7f2d9406f040).stop 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 shutdown_connections 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1153d50 0x11256c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1105fd0 0x11658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1102510 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2d88ff9700 0 client.7100 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 --2- 172.21.15.181:0/559712429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x11653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 >> 172.21.15.181:0/559712429 conn(0x10b5e40 msgr2=0x113b990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 shutdown_connections 2020-11-16T22:34:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 1 -- 172.21.15.181:0/559712429 wait complete. 2020-11-16T22:34:52.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.142+0000 7f2daf929740 20 client.7100 trim_cache size 0 max 16384 2020-11-16T22:34:52.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 Processor -- start 2020-11-16T22:34:52.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 -- start start 2020-11-16T22:34:52.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 0x1138910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1136d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x100ae40 con 0x10b5e40 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x115ac30 con 0xee77b0 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1100f80 con 0xe79bb0 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35710/0 (socket says 172.21.15.181:35710) 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/1492622335 learned_addr learned my addr 172.21.15.181:0/1492622335 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 0x1138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 msgr2=0x1136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/1492622335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 msgr2=0x1138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 0x1138910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 -- 172.21.15.181:0/1492622335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640081d0 con 0xee77b0 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 0x1138910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.158+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2d8c005f00 tx=0x7f2d8c009040).ready entity=mon.1 client_cookie=6b3f8903837c1bc4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2d8bfff700 1 -- 172.21.15.181:0/1492622335 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c001a90 con 0xee77b0 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2d8bfff700 1 -- 172.21.15.181:0/1492622335 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c001bf0 con 0xee77b0 2020-11-16T22:34:52.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2d8bfff700 1 -- 172.21.15.181:0/1492622335 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c001f70 con 0xee77b0 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 msgr2=0x1104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f2d8c005f00 tx=0x7f2d8c009040).stop 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- 172.21.15.181:0/1492622335 shutdown_connections 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xee77b0 0x1104f50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10b5e40 0x1138910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 --2- 172.21.15.181:0/1492622335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xe79bb0 0x1136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- 172.21.15.181:0/1492622335 >> 172.21.15.181:0/1492622335 conn(0x7f2d64002320 msgr2=0x1153540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- 172.21.15.181:0/1492622335 shutdown_connections 2020-11-16T22:34:52.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- 172.21.15.181:0/1492622335 wait complete. 2020-11-16T22:34:52.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 -- start start 2020-11-16T22:34:52.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.162+0000 7f2daf929740 1 Processor -- start 2020-11-16T22:34:52.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe79bb0 0x11b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 0x11bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1165940 con 0xe79bb0 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b7650 con 0x10b5e40 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1157890 con 0xee77b0 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 10 client.0 ll_register_callbacks cb 0x11068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43560/0 (socket says 172.21.15.181:43560) 2020-11-16T22:34:52.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 -- 172.21.15.181:0/3955594406 learned_addr learned my addr 172.21.15.181:0/3955594406 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d9ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 0x11bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe79bb0 0x11b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 -- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 msgr2=0x11bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 0x11bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 -- 172.21.15.181:0/3955594406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe79bb0 msgr2=0x11b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe79bb0 0x11b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 -- 172.21.15.181:0/3955594406 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe97720 con 0xee77b0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d9ad9d700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 0x11bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d99d9b700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f2d8400bf30 tx=0x7f2d84017000).ready entity=mon.2 client_cookie=9562186b60dc7659 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84028040 con 0xee77b0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d84014180 con 0xee77b0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d84020880 con 0xee77b0 2020-11-16T22:34:52.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8402c280 con 0xee77b0 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 10 client.7106 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0xea2e00 con 0xee77b0 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84014180 con 0xee77b0 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2d84020c50 con 0xee77b0 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 10 client.7106 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:52.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 10 client.7106 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 20 client.7106 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1105fd0 0x1154130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 4 client.7106 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xf2af50 con 0x1105fd0 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1105fd0 0x1154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 10 client.7106 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:52.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d9a59c700 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1105fd0 0x1154130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=6d975b8921994942 server_cookie=2cd85e6faedf0c07 in_seq=0 out_seq=0 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 1 -- 172.21.15.181:0/3955594406 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0xf2af50 con 0x1105fd0 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 10 client.7106 handle_command_reply: tid=0 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 client.7106 shutdown 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 msgr2=0x11b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f2d8400bf30 tx=0x7f2d84017000).stop 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 shutdown_connections 2020-11-16T22:34:52.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x1105fd0 0x1154130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2d88ff9700 0 client.7106 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10b5e40 0x11bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xe79bb0 0x11b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 --2- 172.21.15.181:0/3955594406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xee77b0 0x11b8230 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 >> 172.21.15.181:0/3955594406 conn(0x7f2d64002320 msgr2=0x1153430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 shutdown_connections 2020-11-16T22:34:52.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.166+0000 7f2daf929740 1 -- 172.21.15.181:0/3955594406 wait complete. 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2daf929740 20 client.7106 trim_cache size 0 max 16384 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stdout: "e8986808-291a-4767-bae7-a1c0041da596": { 2020-11-16T22:34:52.167 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:34:52.168 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:34:52.168 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:34:52.168 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:52.168 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:52.168 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:52.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 msgr2=0x7f2d6c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 0x7f2d6c057520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f2d9c162f80 tx=0x7f2d84006df0).stop 2020-11-16T22:34:52.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 msgr2=0x7f2d9c0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f2d9400be30 tx=0x7f2d94015000).stop 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 shutdown_connections 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2d6c055060 0x7f2d6c057520 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d9c0b7430 0x7f2d9c15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c161480 0x7f2d9c0b5280 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 --2- 172.21.15.181:0/1886618396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c161860 0x7f2d9c0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 >> 172.21.15.181:0/1886618396 conn(0x7f2d9c0d2dc0 msgr2=0x7f2d9c163150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 shutdown_connections 2020-11-16T22:34:52.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.170+0000 7f2d9b59e700 1 -- 172.21.15.181:0/1886618396 wait complete. 2020-11-16T22:34:52.175 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:52.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.306+0000 7fad92f8d700 1 Processor -- start 2020-11-16T22:34:52.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.306+0000 7fad92f8d700 1 -- start start 2020-11-16T22:34:52.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad92f8d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad92f8d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad8c15d460 0x7fad8c066310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad92f8d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c065150 con 0x7fad8c0600f0 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad92f8d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0653f0 con 0x7fad8c15d460 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad92f8d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c065690 con 0x7fad8c15d080 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad91f8b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad91f8b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43566/0 (socket says 172.21.15.181:43566) 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad91f8b700 1 -- 172.21.15.181:0/1042960154 learned_addr learned my addr 172.21.15.181:0/1042960154 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad8c15d460 0x7fad8c066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 -- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 msgr2=0x7fad8c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad90f89700 1 -- 172.21.15.181:0/1042960154 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296425840 0 0) 0x7fad8c065150 con 0x7fad8c0600f0 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 -- 172.21.15.181:0/1042960154 >> v1:172.21.15.181:6789/0 conn(0x7fad8c0600f0 legacy=0x7fad8c0605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 -- 172.21.15.181:0/1042960154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c065930 con 0x7fad8c15d460 2020-11-16T22:34:52.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad91f8b700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad9178a700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c15d460 0x7fad8c066310 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fad7c00b9a0 tx=0x7fad7c006df0).ready entity=mon.1 client_cookie=4b1c50d2338c2658 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad90f89700 1 -- 172.21.15.181:0/1042960154 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0097b0 con 0x7fad8c15d460 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad90f89700 1 -- 172.21.15.181:0/1042960154 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c009910 con 0x7fad8c15d460 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.310+0000 7fad90f89700 1 -- 172.21.15.181:0/1042960154 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c009c90 con 0x7fad8c15d460 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c15d460 msgr2=0x7fad8c066310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c15d460 0x7fad8c066310 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fad7c00b9a0 tx=0x7fad7c006df0).stop 2020-11-16T22:34:52.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- 172.21.15.181:0/1042960154 shutdown_connections 2020-11-16T22:34:52.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d080 0x7fad8c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- 172.21.15.181:0/1042960154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c15d460 0x7fad8c066310 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- 172.21.15.181:0/1042960154 >> 172.21.15.181:0/1042960154 conn(0x7fad8c0cfe90 msgr2=0x7fad8c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- 172.21.15.181:0/1042960154 shutdown_connections 2020-11-16T22:34:52.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- 172.21.15.181:0/1042960154 wait complete. 2020-11-16T22:34:52.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 Processor -- start 2020-11-16T22:34:52.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- start start 2020-11-16T22:34:52.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 0x7fad8c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad8c15d080 0x7fad8c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 0x7fad8c16f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c172650 con 0x7fad8c15d080 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1728f0 con 0x7fad8c0600f0 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad92f8d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c172b90 con 0x7fad8c15d460 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad9278c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 0x7fad8c16f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 0x7fad8c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad9278c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 0x7fad8c16f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43574/0 (socket says 172.21.15.181:43574) 2020-11-16T22:34:52.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad9278c700 1 -- 172.21.15.181:0/3481665775 learned_addr learned my addr 172.21.15.181:0/3481665775 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad9178a700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad8c15d080 0x7fad8c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 -- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 msgr2=0x7fad8c16f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 0x7fad8c16f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 -- 172.21.15.181:0/3481665775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad8c15d080 msgr2=0x7fad8c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad8c15d080 0x7fad8c154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 -- 172.21.15.181:0/3481665775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c172e30 con 0x7fad8c0600f0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.314+0000 7fad91f8b700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 0x7fad8c0b3620 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fad8800ecf0 tx=0x7fad88006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800b580 con 0x7fad8c0600f0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800ca10 con 0x7fad8c0600f0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800cd90 con 0x7fad8c0600f0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c1730d0 con 0x7fad8c0600f0 2020-11-16T22:34:52.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c16f0a0 con 0x7fad8c0600f0 2020-11-16T22:34:52.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c16f290 con 0x7fad8c0600f0 2020-11-16T22:34:52.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fad88022070 con 0x7fad8c0600f0 2020-11-16T22:34:52.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 0x7fad64057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad9178a700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 0x7fad64057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad8806e3b0 con 0x7fad8c0600f0 2020-11-16T22:34:52.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.318+0000 7fad9178a700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 0x7fad64057520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fad7c00b9a0 tx=0x7fad7c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.326+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fad88044610 con 0x7fad8c0600f0 2020-11-16T22:34:52.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.526+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad8c044f60 con 0x7fad8c0600f0 2020-11-16T22:34:52.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.526+0000 7fad79ffb700 1 -- 172.21.15.181:0/3481665775 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4669 (secure 0 0 0) 0x7fad88047c30 con 0x7fad8c0600f0 2020-11-16T22:34:52.519 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:52.519 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:51.894687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 msgr2=0x7fad64057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 0x7fad64057520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fad7c00b9a0 tx=0x7fad7c015000).stop 2020-11-16T22:34:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 msgr2=0x7fad8c0b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 0x7fad8c0b3620 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fad8800ecf0 tx=0x7fad88006eb0).stop 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 shutdown_connections 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fad64055060 0x7fad64057520 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c15d460 0x7fad8c16f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c0600f0 0x7fad8c0b3620 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 --2- 172.21.15.181:0/3481665775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad8c15d080 0x7fad8c154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 >> 172.21.15.181:0/3481665775 conn(0x7fad8c0cfe90 msgr2=0x7fad8c156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 shutdown_connections 2020-11-16T22:34:52.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.530+0000 7fad92f8d700 1 -- 172.21.15.181:0/3481665775 wait complete. 2020-11-16T22:34:52.523 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 105 2020-11-16T22:34:52.534 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:34:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.670+0000 7fbd12317700 1 Processor -- start 2020-11-16T22:34:52.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.670+0000 7fbd12317700 1 -- start start 2020-11-16T22:34:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd0c15f5b0 0x7fbd0c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 0x7fbd0c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c160e40 con 0x7fbd0c0615f0 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c160ba0 con 0x7fbd0c15f5b0 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c060970 con 0x7fbd0c0622a0 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43580/0 (socket says 172.21.15.181:43580) 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd0c15f5b0 0x7fbd0c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 -- 172.21.15.181:0/2003131388 learned_addr learned my addr 172.21.15.181:0/2003131388 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 0x7fbd0c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 -- 172.21.15.181:0/2003131388 >> v2:172.21.15.181:3300/0 conn(0x7fbd0c15f5b0 msgr2=0x7fbd0c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.181:3300/0 conn(0x7fbd0c15f5b0 0x7fbd0c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 -- 172.21.15.181:0/2003131388 >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 msgr2=0x7fbd0c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 0x7fbd0c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 -- 172.21.15.181:0/2003131388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c060c10 con 0x7fbd0c0622a0 2020-11-16T22:34:52.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 0x7fbd0c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd108b4700 1 --2- 172.21.15.181:0/2003131388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fbd0c083ad0 tx=0x7fbd04009290).ready entity=mon.2 client_cookie=81e574d9f7216427 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0affd700 1 -- 172.21.15.181:0/2003131388 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400bbc0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0affd700 1 -- 172.21.15.181:0/2003131388 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0400bd20 con 0x7fbd0c0622a0 2020-11-16T22:34:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd0affd700 1 -- 172.21.15.181:0/2003131388 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011610 con 0x7fbd0c0622a0 2020-11-16T22:34:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- 172.21.15.181:0/2003131388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 msgr2=0x7fbd0c062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- 172.21.15.181:0/2003131388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fbd0c083ad0 tx=0x7fbd04009290).stop 2020-11-16T22:34:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- 172.21.15.181:0/2003131388 shutdown_connections 2020-11-16T22:34:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- 172.21.15.181:0/2003131388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c062700 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.181:3300/0 conn(0x7fbd0c15f5b0 0x7fbd0c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 --2- 172.21.15.181:0/2003131388 >> v2:172.21.15.175:3300/0 conn(0x7fbd0c0615f0 0x7fbd0c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- 172.21.15.181:0/2003131388 >> 172.21.15.181:0/2003131388 conn(0x7fbd0c0d69a0 msgr2=0x7fbd0c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- 172.21.15.181:0/2003131388 shutdown_connections 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- 172.21.15.181:0/2003131388 wait complete. 2020-11-16T22:34:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 Processor -- start 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.674+0000 7fbd12317700 1 -- start start 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0615f0 0x7fbd0c0bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 0x7fbd0c163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0bbd60 con 0x7fbd0c0615f0 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0bc000 con 0x7fbd0c15f5b0 2020-11-16T22:34:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0bc2a0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 0x7fbd0c163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43588/0 (socket says 172.21.15.181:43588) 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 -- 172.21.15.181:0/3353855434 learned_addr learned my addr 172.21.15.181:0/3353855434 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0615f0 0x7fbd0c0bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 -- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 msgr2=0x7fbd0c163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 0x7fbd0c163b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 -- 172.21.15.181:0/3353855434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0615f0 msgr2=0x7fbd0c0bb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0615f0 0x7fbd0c0bb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 -- 172.21.15.181:0/3353855434 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0bc540 con 0x7fbd0c0622a0 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd108b4700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 0x7fbd0c163b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fbcf400d900 tx=0x7fbcf4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd03fff700 1 -- 172.21.15.181:0/3353855434 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf400f5e0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c0bc7e0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd03fff700 1 -- 172.21.15.181:0/3353855434 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf400f740 con 0x7fbd0c0622a0 2020-11-16T22:34:52.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c0bbae0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd03fff700 1 -- 172.21.15.181:0/3353855434 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf400fac0 con 0x7fbd0c0622a0 2020-11-16T22:34:52.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.678+0000 7fbd03fff700 1 -- 172.21.15.181:0/3353855434 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbcf401c460 con 0x7fbd0c0622a0 2020-11-16T22:34:52.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.682+0000 7fbd03fff700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 0x7fbce8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.682+0000 7fbd03fff700 1 -- 172.21.15.181:0/3353855434 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcf406dd60 con 0x7fbd0c0622a0 2020-11-16T22:34:52.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.682+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 0x7fbce8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.682+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 0x7fbce8057520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fbcfc006fd0 tx=0x7fbcfc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 Processor -- start 2020-11-16T22:34:52.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 -- start start 2020-11-16T22:34:52.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c85530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c48470 0x1ca2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad94b0 con 0x19c1bb0 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b8d930 con 0x1a2f7b0 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a4d6f0 con 0x1c48470 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c48470 0x1ca2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35754/0 (socket says 172.21.15.181:35754) 2020-11-16T22:34:52.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 -- 172.21.15.181:0/2002829857 learned_addr learned my addr 172.21.15.181:0/2002829857 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c85530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 -- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c48470 msgr2=0x1ca2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c48470 0x1ca2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 -- 172.21.15.181:0/2002829857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 msgr2=0x1c85530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c85530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 -- 172.21.15.181:0/2002829857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a485a0 con 0x1a2f7b0 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fbcfc013c30 tx=0x7fbcfc012000).ready entity=mon.1 client_cookie=b66cc43fe3e042d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd017fa700 1 -- 172.21.15.181:0/2002829857 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc013240 con 0x1a2f7b0 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd017fa700 1 -- 172.21.15.181:0/2002829857 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc011580 con 0x1a2f7b0 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.686+0000 7fbd017fa700 1 -- 172.21.15.181:0/2002829857 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0133a0 con 0x1a2f7b0 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 msgr2=0x1c46cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fbcfc013c30 tx=0x7fbcfc012000).stop 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/2002829857 shutdown_connections 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c48470 0x1ca2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1c46cd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- 172.21.15.181:0/2002829857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c85530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/2002829857 >> 172.21.15.181:0/2002829857 conn(0x1bfde40 msgr2=0x1ca2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/2002829857 shutdown_connections 2020-11-16T22:34:52.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/2002829857 wait complete. 2020-11-16T22:34:52.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- start start 2020-11-16T22:34:52.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 Processor -- start 2020-11-16T22:34:52.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 0x1cae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c48470 0x1caabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b7a6e0 con 0x1a2f7b0 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b7a820 con 0x1c48470 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b7ba90 con 0x19c1bb0 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 10 client.0 ll_register_callbacks cb 0x1b70ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43600/0 (socket says 172.21.15.181:43600) 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 -- 172.21.15.181:0/286691984 learned_addr learned my addr 172.21.15.181:0/286691984 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd108b4700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c48470 0x1caabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 0x1cae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 -- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c48470 msgr2=0x1caabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c48470 0x1caabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 -- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 msgr2=0x1cae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 0x1cae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 -- 172.21.15.181:0/286691984 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b6b280 con 0x19c1bb0 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 0x1cae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fbcfc011550 tx=0x7fbcfc015000).ready entity=mon.2 client_cookie=95a4ccf2168578fe server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc027070 con 0x19c1bb0 2020-11-16T22:34:52.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcfc01f660 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c42e10 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcfc01fa30 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 10 client.7124 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.690+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x19a0fd0 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c42e10 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fbcfc01fb90 con 0x19c1bb0 2020-11-16T22:34:52.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 10 client.7124 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 10 client.7124 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 20 client.7124 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1cb0d20 0x1c9d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 4 client.7124 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1a6bc50 con 0x1cb0d20 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1cb0d20 0x1c9d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 10 client.7124 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:52.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1cb0d20 0x1c9d250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=80ba66e4988afb23 server_cookie=c0916b5993fb2a63 in_seq=0 out_seq=0 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 1 -- 172.21.15.181:0/286691984 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1a6bc50 con 0x1cb0d20 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 10 client.7124 handle_command_reply: tid=0 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 client.7124 shutdown 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 msgr2=0x1cafde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fbcfc011550 tx=0x7fbcfc015000).stop 2020-11-16T22:34:52.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 shutdown_connections 2020-11-16T22:34:52.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1cb0d20 0x1c9d250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c48470 0x1caabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1cafde0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 --2- 172.21.15.181:0/286691984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1a2f7b0 0x1cae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbce27fc700 0 client.7124 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 >> 172.21.15.181:0/286691984 conn(0x1bfde40 msgr2=0x1c854f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 shutdown_connections 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 1 -- 172.21.15.181:0/286691984 wait complete. 2020-11-16T22:34:52.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.694+0000 7fbd20da8740 20 client.7124 trim_cache size 0 max 16384 2020-11-16T22:34:52.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 Processor -- start 2020-11-16T22:34:52.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- start start 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 0x1c72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c4e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b52e40 con 0x19c1bb0 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cadd30 con 0x1a2f7b0 2020-11-16T22:34:52.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c47f10 con 0x1bfde40 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 0x1c72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 0x1c72800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43608/0 (socket says 172.21.15.181:43608) 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35768/0 (socket says 172.21.15.181:35768) 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd0bfff700 1 -- 172.21.15.181:0/1655981731 learned_addr learned my addr 172.21.15.181:0/1655981731 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c4e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 -- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 msgr2=0x1c72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 0x1c72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 -- 172.21.15.181:0/1655981731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 msgr2=0x1c4e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c4e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 -- 172.21.15.181:0/1655981731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80081d0 con 0x1a2f7b0 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd108b4700 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fbd04013000 tx=0x7fbd04016000).ready entity=mon.1 client_cookie=2318be48ce86c1c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd01ffb700 1 -- 172.21.15.181:0/1655981731 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04022070 con 0x1a2f7b0 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd01ffb700 1 -- 172.21.15.181:0/1655981731 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd04023040 con 0x1a2f7b0 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd01ffb700 1 -- 172.21.15.181:0/1655981731 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011610 con 0x1a2f7b0 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 msgr2=0x1caded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fbd04013000 tx=0x7fbd04016000).stop 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- 172.21.15.181:0/1655981731 shutdown_connections 2020-11-16T22:34:52.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1caded0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bfde40 0x1c72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 --2- 172.21.15.181:0/1655981731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19c1bb0 0x1c4e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- 172.21.15.181:0/1655981731 >> 172.21.15.181:0/1655981731 conn(0x7fbcd8002320 msgr2=0x1c66790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- 172.21.15.181:0/1655981731 shutdown_connections 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.714+0000 7fbd20da8740 1 -- 172.21.15.181:0/1655981731 wait complete. 2020-11-16T22:34:52.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- start start 2020-11-16T22:34:52.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 Processor -- start 2020-11-16T22:34:52.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1cfca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 0x1c66790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c70ed0 con 0x1bfde40 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c6ac40 con 0x1a2f7b0 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c6aee0 con 0x19c1bb0 2020-11-16T22:34:52.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 10 client.0 ll_register_callbacks cb 0x1c87ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43614/0 (socket says 172.21.15.181:43614) 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1cfca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 -- 172.21.15.181:0/4010880795 learned_addr learned my addr 172.21.15.181:0/4010880795 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd108b4700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 0x1c66790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 -- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 msgr2=0x1cfca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1cfca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 -- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 msgr2=0x1c66790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 0x1c66790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 -- 172.21.15.181:0/4010880795 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19df720 con 0x19c1bb0 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd108b4700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 0x1c66790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbcfc0134d0 tx=0x7fbcfc027000).ready entity=mon.2 client_cookie=c5122263beda35e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:52.716 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc02cf00 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcfc02d060 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcfc011580 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcfc02d430 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 10 client.7130 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x19eae00 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc011580 con 0x19c1bb0 2020-11-16T22:34:52.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fbcfc040480 con 0x19c1bb0 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 10 client.7130 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 10 client.7130 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 20 client.7130 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1c48470 0x1cae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 4 client.7130 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1a72f50 con 0x1c48470 2020-11-16T22:34:52.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.718+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1c48470 0x1cae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbce27fc700 10 client.7130 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1c48470 0x1cae420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=5e6c8e2bd823042c server_cookie=4b47a043e9b87379 in_seq=0 out_seq=0 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbce27fc700 1 -- 172.21.15.181:0/4010880795 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1a72f50 con 0x1c48470 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbce27fc700 10 client.7130 handle_command_reply: tid=0 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 client.7130 shutdown 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 msgr2=0x1c70810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbcfc0134d0 tx=0x7fbcfc027000).stop 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 shutdown_connections 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1c48470 0x1cae420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bfde40 0x1c66790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19c1bb0 0x1c70810 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 --2- 172.21.15.181:0/4010880795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1a2f7b0 0x1cfca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbce27fc700 0 client.7130 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:34:52.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 >> 172.21.15.181:0/4010880795 conn(0x7fbcd8002320 msgr2=0x1cb01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 shutdown_connections 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 1 -- 172.21.15.181:0/4010880795 wait complete. 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd20da8740 20 client.7130 trim_cache size 0 max 16384 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 msgr2=0x7fbce8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 0x7fbce8057520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fbcfc006fd0 tx=0x7fbcfc008040).stop 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 msgr2=0x7fbd0c163670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fbcf400d900 tx=0x7fbcf4008040).stop 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 shutdown_connections 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbce8055060 0x7fbce8057520 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c15f5b0 0x7fbd0c163b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0615f0 0x7fbd0c0bb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 --2- 172.21.15.181:0/3353855434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd0c0622a0 0x7fbd0c163670 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 >> 172.21.15.181:0/3353855434 conn(0x7fbd0c0d69a0 msgr2=0x7fbd0c061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 shutdown_connections 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.722+0000 7fbd12317700 1 -- 172.21.15.181:0/3353855434 wait complete. 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout: "e8986808-291a-4767-bae7-a1c0041da596": { 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:34:52.723 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:34:52.724 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:34:52.724 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:52.724 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:52.724 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:52.730 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:52.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.866+0000 7f576d515700 1 Processor -- start 2020-11-16T22:34:52.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.866+0000 7f576d515700 1 -- start start 2020-11-16T22:34:52.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576d515700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 0x7f5768061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576d515700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57680615f0 0x7f576815e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576d515700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576815d180 con 0x7f576813e870 2020-11-16T22:34:52.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576d515700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576815d810 con 0x7f57680615f0 2020-11-16T22:34:52.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576d515700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576815dab0 con 0x7f576813ec50 2020-11-16T22:34:52.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5767fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f576813e870 0x7f576815fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50082/0 (socket says 172.21.15.181:50082) 2020-11-16T22:34:52.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5767fff700 1 -- 172.21.15.181:0/643132089 learned_addr learned my addr 172.21.15.181:0/643132089 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:52.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 --2- 172.21.15.181:0/643132089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 0x7f5768061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f576cd14700 1 --2- 172.21.15.181:0/643132089 >> v2:172.21.15.175:3300/0 conn(0x7f57680615f0 0x7f576815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5766ffd700 1 -- 172.21.15.181:0/643132089 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928295577 0 0) 0x7f576815d180 con 0x7f576813e870 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5766ffd700 1 -- 172.21.15.181:0/643132089 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5748003240 con 0x7f576813e870 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 -- 172.21.15.181:0/643132089 >> v2:172.21.15.175:3300/0 conn(0x7f57680615f0 msgr2=0x7f576815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 --2- 172.21.15.181:0/643132089 >> v2:172.21.15.175:3300/0 conn(0x7f57680615f0 0x7f576815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 -- 172.21.15.181:0/643132089 >> v1:172.21.15.181:6789/0 conn(0x7f576813e870 legacy=0x7f576815fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 -- 172.21.15.181:0/643132089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576815dd50 con 0x7f576813ec50 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f57677fe700 1 --2- 172.21.15.181:0/643132089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 0x7f5768061100 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f575000b9a0 tx=0x7f5750006df0).ready entity=mon.2 client_cookie=678f55c0c851fd9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5766ffd700 1 -- 172.21.15.181:0/643132089 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750009680 con 0x7f576813ec50 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5766ffd700 1 -- 172.21.15.181:0/643132089 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57500097e0 con 0x7f576813ec50 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.870+0000 7f5766ffd700 1 -- 172.21.15.181:0/643132089 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750009b60 con 0x7f576813ec50 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 -- 172.21.15.181:0/643132089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 msgr2=0x7f5768061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 --2- 172.21.15.181:0/643132089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 0x7f5768061100 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f575000b9a0 tx=0x7f5750006df0).stop 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 -- 172.21.15.181:0/643132089 shutdown_connections 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 --2- 172.21.15.181:0/643132089 >> v2:172.21.15.175:3300/0 conn(0x7f57680615f0 0x7f576815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 --2- 172.21.15.181:0/643132089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813ec50 0x7f5768061100 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 -- 172.21.15.181:0/643132089 >> 172.21.15.181:0/643132089 conn(0x7f57680cf630 msgr2=0x7f57680cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 -- 172.21.15.181:0/643132089 shutdown_connections 2020-11-16T22:34:52.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.874+0000 7f576d515700 1 -- 172.21.15.181:0/643132089 wait complete. 2020-11-16T22:34:52.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 Processor -- start 2020-11-16T22:34:52.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 -- start start 2020-11-16T22:34:52.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 0x7f57680c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f576813ec50 0x7f576813b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768137670 con 0x7f576813ec50 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f5767fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576816d020 con 0x7f57680615f0 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f576d515700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576816d2c0 con 0x7f576813e870 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 0x7f57680c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f5767fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35784/0 (socket says 172.21.15.181:35784) 2020-11-16T22:34:52.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f5767fff700 1 -- 172.21.15.181:0/3221097913 learned_addr learned my addr 172.21.15.181:0/3221097913 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 -- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 msgr2=0x7f57681393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 -- 172.21.15.181:0/3221097913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f576813ec50 msgr2=0x7f576813b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f576813ec50 0x7f576813b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f57677fe700 1 -- 172.21.15.181:0/3221097913 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576816d560 con 0x7f576813e870 2020-11-16T22:34:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.878+0000 7f5767fff700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f57677fe700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 0x7f57680c9040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f5750012000 tx=0x7f5750015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750011cc0 con 0x7f576813e870 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750011e20 con 0x7f576813e870 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575001e670 con 0x7f576813e870 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576816d800 con 0x7f576813e870 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576816dd40 con 0x7f576813e870 2020-11-16T22:34:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768044f60 con 0x7f576813e870 2020-11-16T22:34:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f575001a040 con 0x7f576813e870 2020-11-16T22:34:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 0x7f5744057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f575006f180 con 0x7f576813e870 2020-11-16T22:34:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f5767fff700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 0x7f5744057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:52.876 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.882+0000 7f5767fff700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 0x7f5744057520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f5758001890 tx=0x7f5758007ae0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:52.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:52.890+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5750045380 con 0x7f576813e870 2020-11-16T22:34:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.090+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f576805c310 con 0x7f576813e870 2020-11-16T22:34:53.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f575ffff700 1 -- 172.21.15.181:0/3221097913 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4669 (secure 0 0 0) 0x7f5750023070 con 0x7f576813e870 2020-11-16T22:34:53.083 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:53.084 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:51.894687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:53.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 msgr2=0x7f5744057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 0x7f5744057520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f5758001890 tx=0x7f5758007ae0).stop 2020-11-16T22:34:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 msgr2=0x7f57680c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 0x7f57680c9040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f5750012000 tx=0x7f5750015000).stop 2020-11-16T22:34:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 shutdown_connections 2020-11-16T22:34:53.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5744055060 0x7f5744057520 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f576813ec50 0x7f576813b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57680615f0 0x7f57681393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 --2- 172.21.15.181:0/3221097913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f576813e870 0x7f57680c9040 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 >> 172.21.15.181:0/3221097913 conn(0x7f57680cf630 msgr2=0x7f576815e960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 shutdown_connections 2020-11-16T22:34:53.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.094+0000 7f576d515700 1 -- 172.21.15.181:0/3221097913 wait complete. 2020-11-16T22:34:53.088 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 105 2020-11-16T22:34:53.099 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:34:53.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.238+0000 7f11ce733700 1 Processor -- start 2020-11-16T22:34:53.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.238+0000 7f11ce733700 1 -- start start 2020-11-16T22:34:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11c815f5b0 0x7f11c80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c8160e40 con 0x7f11c80615f0 2020-11-16T22:34:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c8160ba0 con 0x7f11c80d1830 2020-11-16T22:34:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c81609a0 con 0x7f11c815f5b0 2020-11-16T22:34:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11c815f5b0 0x7f11c80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11cccd0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f11c80d1830 0x7f11c8080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50100/0 (socket says 172.21.15.181:50100) 2020-11-16T22:34:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11c815f5b0 0x7f11c80d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35794/0 (socket says 172.21.15.181:35794) 2020-11-16T22:34:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11cccd0700 1 -- 172.21.15.181:0/1367502292 learned_addr learned my addr 172.21.15.181:0/1367502292 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 -- 172.21.15.181:0/1367502292 >> v1:172.21.15.181:6789/0 conn(0x7f11c80d1830 legacy=0x7f11c8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c6ffd700 1 -- 172.21.15.181:0/1367502292 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380262038 0 0) 0x7f11c8160e40 con 0x7f11c80615f0 2020-11-16T22:34:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 -- 172.21.15.181:0/1367502292 >> v1:172.21.15.175:6789/0 conn(0x7f11c80615f0 legacy=0x7f11c80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 -- 172.21.15.181:0/1367502292 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8060970 con 0x7f11c815f5b0 2020-11-16T22:34:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c7fff700 1 --2- 172.21.15.181:0/1367502292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c815f5b0 0x7f11c80d1340 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f11b800b910 tx=0x7f11b8006d90).ready entity=mon.1 client_cookie=9e4601e513376abc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c6ffd700 1 -- 172.21.15.181:0/1367502292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8009730 con 0x7f11c815f5b0 2020-11-16T22:34:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c6ffd700 1 -- 172.21.15.181:0/1367502292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8009890 con 0x7f11c815f5b0 2020-11-16T22:34:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11c6ffd700 1 -- 172.21.15.181:0/1367502292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8009c10 con 0x7f11c815f5b0 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- 172.21.15.181:0/1367502292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c815f5b0 msgr2=0x7f11c80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 --2- 172.21.15.181:0/1367502292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c815f5b0 0x7f11c80d1340 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f11b800b910 tx=0x7f11b8006d90).stop 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- 172.21.15.181:0/1367502292 shutdown_connections 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 --2- 172.21.15.181:0/1367502292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c815f5b0 0x7f11c80d1340 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- 172.21.15.181:0/1367502292 >> 172.21.15.181:0/1367502292 conn(0x7f11c80d69a0 msgr2=0x7f11c80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.236 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- 172.21.15.181:0/1367502292 shutdown_connections 2020-11-16T22:34:53.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.242+0000 7f11ce733700 1 -- 172.21.15.181:0/1367502292 wait complete. 2020-11-16T22:34:53.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 Processor -- start 2020-11-16T22:34:53.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 -- start start 2020-11-16T22:34:53.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11c80615f0 0x7f11c80b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 0x7f11c80bba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b7220 con 0x7f11c815f5b0 2020-11-16T22:34:53.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80bc4b0 con 0x7f11c80d1830 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11ce733700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b9930 con 0x7f11c80615f0 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.246+0000 7f11c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35798/0 (socket says 172.21.15.181:35798) 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 -- 172.21.15.181:0/3516392524 learned_addr learned my addr 172.21.15.181:0/3516392524 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11c80615f0 0x7f11c80b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11cccd0700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 0x7f11c80bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.241 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 -- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11c80615f0 msgr2=0x7f11c80b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11c80615f0 0x7f11c80b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 -- 172.21.15.181:0/3516392524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 msgr2=0x7f11c80bba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 0x7f11c80bba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 -- 172.21.15.181:0/3516392524 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80b9bd0 con 0x7f11c80d1830 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11cccd0700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 0x7f11c80bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f11bc00c730 tx=0x7f11bc007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.242 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 -- 172.21.15.181:0/3516392524 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00e560 con 0x7f11c80d1830 2020-11-16T22:34:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 -- 172.21.15.181:0/3516392524 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc00e6c0 con 0x7f11c80d1830 2020-11-16T22:34:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80b9e70 con 0x7f11c80d1830 2020-11-16T22:34:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 -- 172.21.15.181:0/3516392524 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00ea80 con 0x7f11c80d1830 2020-11-16T22:34:53.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80b8980 con 0x7f11c80d1830 2020-11-16T22:34:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 -- 172.21.15.181:0/3516392524 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f11bc019030 con 0x7f11c80d1830 2020-11-16T22:34:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 0x7f11a0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.250+0000 7f11a7fff700 1 -- 172.21.15.181:0/3516392524 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11bc06d2d0 con 0x7f11c80d1830 2020-11-16T22:34:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 0x7f11a0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 0x7f11a0057520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f11b8006dd0 tx=0x7f11b8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 Processor -- start 2020-11-16T22:34:53.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 -- start start 2020-11-16T22:34:53.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 0x1415c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x124c4b0 con 0x1134bb0 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1300930 con 0x11a27b0 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x11c06f0 con 0x13bb470 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11cccd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 0x1415c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35808/0 (socket says 172.21.15.181:35808) 2020-11-16T22:34:53.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11cccd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 0x1415c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43652/0 (socket says 172.21.15.181:43652) 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 -- 172.21.15.181:0/446222564 learned_addr learned my addr 172.21.15.181:0/446222564 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 -- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 msgr2=0x1415c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 0x1415c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 -- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 msgr2=0x13b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 -- 172.21.15.181:0/446222564 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11bb5a0 con 0x1134bb0 2020-11-16T22:34:53.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c7fff700 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:53.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11c77fe700 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13f8530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f11bc046d70 tx=0x7f11bc01e040).ready entity=mon.0 client_cookie=690aec04f3f07c70 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11a57fa700 1 -- 172.21.15.181:0/446222564 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc001ea0 con 0x1134bb0 2020-11-16T22:34:53.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11a57fa700 1 -- 172.21.15.181:0/446222564 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc06d0d0 con 0x1134bb0 2020-11-16T22:34:53.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.254+0000 7f11a57fa700 1 -- 172.21.15.181:0/446222564 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc046790 con 0x1134bb0 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/446222564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 msgr2=0x13f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13f8530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f11bc046d70 tx=0x7f11bc01e040).stop 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/446222564 shutdown_connections 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13bb470 0x1415c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a27b0 0x13b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/446222564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13f8530 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/446222564 >> 172.21.15.181:0/446222564 conn(0x1370e40 msgr2=0x1415580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/446222564 shutdown_connections 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/446222564 wait complete. 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- start start 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 Processor -- start 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1134bb0 0x1422ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x13f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x12ed6e0 con 0x11a27b0 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x12ed820 con 0x13bb470 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12eea90 con 0x1134bb0 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 10 client.0 ll_register_callbacks cb 0x12e3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35816/0 (socket says 172.21.15.181:35816) 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 -- 172.21.15.181:0/1569173166 learned_addr learned my addr 172.21.15.181:0/1569173166 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1134bb0 0x1422ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11c77fe700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x13f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 -- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1134bb0 msgr2=0x1422ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1134bb0 0x1422ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 -- 172.21.15.181:0/1569173166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 msgr2=0x13f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x13f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 -- 172.21.15.181:0/1569173166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x12de280 con 0x13bb470 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11cccd0700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f11c0010890 tx=0x7f11c0009040).ready entity=mon.1 client_cookie=d941faee907e03c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f119e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0005b60 con 0x13bb470 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11c0005cc0 con 0x13bb470 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13b5e10 con 0x13bb470 2020-11-16T22:34:53.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11c0005e80 con 0x13bb470 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 10 client.7807 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.258+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x1113fd0 con 0x13bb470 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13b5e10 con 0x13bb470 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f11c00072e0 con 0x13bb470 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 10 client.7807 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 10 client.7807 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:53.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 20 client.7807 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13d2560 0x13d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 4 client.7807 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x11dec50 con 0x13d2560 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11c7fff700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13d2560 0x13d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 10 client.7807 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11c7fff700 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13d2560 0x13d2940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=885b0397fd282959 server_cookie=899c96da6877c35c in_seq=0 out_seq=0 2020-11-16T22:34:53.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 1 -- 172.21.15.181:0/1569173166 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x11dec50 con 0x13d2560 2020-11-16T22:34:53.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 10 client.7807 handle_command_reply: tid=0 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 client.7807 shutdown 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 msgr2=0x13e5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f11c0010890 tx=0x7f11c0009040).stop 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 shutdown_connections 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13d2560 0x13d2940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x13bb470 0x13e5ec0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1134bb0 0x1422ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/1569173166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x13f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f119e7fc700 0 client.7807 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 >> 172.21.15.181:0/1569173166 conn(0x1370e40 msgr2=0x13bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 shutdown_connections 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 1 -- 172.21.15.181:0/1569173166 wait complete. 2020-11-16T22:34:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.262+0000 7f11dd1c4740 20 client.7807 trim_cache size 0 max 16384 2020-11-16T22:34:53.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 Processor -- start 2020-11-16T22:34:53.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- start start 2020-11-16T22:34:53.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11a27b0 0x13c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x12c5e40 con 0x1134bb0 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x141ef20 con 0x1370e40 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1421070 con 0x11a27b0 2020-11-16T22:34:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11cccd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11a27b0 0x13c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35824/0 (socket says 172.21.15.181:35824) 2020-11-16T22:34:53.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 -- 172.21.15.181:0/2355265251 learned_addr learned my addr 172.21.15.181:0/2355265251 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c77fe700 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 -- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11a27b0 msgr2=0x13c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11a27b0 0x13c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 -- 172.21.15.181:0/2355265251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 msgr2=0x13c0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13c0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 -- 172.21.15.181:0/2355265251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940081d0 con 0x1370e40 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11c7fff700 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f11b800b7f0 tx=0x7f11b8030280).ready entity=mon.1 client_cookie=7dad729695c82dcf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11a5ffb700 1 -- 172.21.15.181:0/2355265251 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8011610 con 0x1370e40 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11a5ffb700 1 -- 172.21.15.181:0/2355265251 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8011770 con 0x1370e40 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11a5ffb700 1 -- 172.21.15.181:0/2355265251 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8011af0 con 0x1370e40 2020-11-16T22:34:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 msgr2=0x13bac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f11b800b7f0 tx=0x7f11b8030280).stop 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- 172.21.15.181:0/2355265251 shutdown_connections 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11a27b0 0x13c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1370e40 0x13bac20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/2355265251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1134bb0 0x13c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- 172.21.15.181:0/2355265251 >> 172.21.15.181:0/2355265251 conn(0x7f1194002320 msgr2=0x13ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- 172.21.15.181:0/2355265251 shutdown_connections 2020-11-16T22:34:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.282+0000 7f11dd1c4740 1 -- 172.21.15.181:0/2355265251 wait complete. 2020-11-16T22:34:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 -- start start 2020-11-16T22:34:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 Processor -- start 2020-11-16T22:34:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x1475d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 0x1476280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.286+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13e6c30 con 0x11a27b0 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x14727e0 con 0x1134bb0 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1476a70 con 0x1370e40 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35826/0 (socket says 172.21.15.181:35826) 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 10 client.0 ll_register_callbacks cb 0x13fab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 -- 172.21.15.181:0/3371130229 learned_addr learned my addr 172.21.15.181:0/3371130229 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11cccd0700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 0x1476280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x1475d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 -- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 msgr2=0x1476280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 0x1476280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 -- 172.21.15.181:0/3371130229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 msgr2=0x1475d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x1475d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 -- 172.21.15.181:0/3371130229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1152720 con 0x1134bb0 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11cccd0700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 0x1476280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c7fff700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f11b801f040 tx=0x7f11b8032000).ready entity=mon.1 client_cookie=3835eaed56ce2e1c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b801d620 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11b8022040 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11bc06d0d0 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b801d780 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 10 client.7813 fetch_fsmap learned FSMap version 105 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x115de00 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8046020 con 0x1134bb0 2020-11-16T22:34:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 105) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f11b8027040 con 0x1134bb0 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 10 client.7813 fetch_fsmap finished waiting for FSMap version 105 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 10 client.7813 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 20 client.7813 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13bb470 0x13e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 4 client.7813 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x11e5f50 con 0x13bb470 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13bb470 0x13e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 10 client.7813 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11c77fe700 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13bb470 0x13e2a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=16aa8e905441802c server_cookie=5bc5f305d295b678 in_seq=0 out_seq=0 2020-11-16T22:34:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 1 -- 172.21.15.181:0/3371130229 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x11e5f50 con 0x13bb470 2020-11-16T22:34:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f119e7fc700 10 client.7813 handle_command_reply: tid=0 2020-11-16T22:34:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 client.7813 shutdown 2020-11-16T22:34:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 msgr2=0x14587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f11b801f040 tx=0x7f11b8032000).stop 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 shutdown_connections 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x13bb470 0x13e2a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1370e40 0x1476280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.290+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1134bb0 0x14587f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11dd1c4740 1 --2- 172.21.15.181:0/3371130229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11a27b0 0x1475d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 >> 172.21.15.181:0/3371130229 conn(0x7f1194002320 msgr2=0x14122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f119e7fc700 0 client.7813 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 shutdown_connections 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11dd1c4740 1 -- 172.21.15.181:0/3371130229 wait complete. 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11dd1c4740 20 client.7813 trim_cache size 0 max 16384 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 msgr2=0x7f11a0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 0x7f11a0057520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f11b8006dd0 tx=0x7f11b8015000).stop 2020-11-16T22:34:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 msgr2=0x7f11c80b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f11bc00c730 tx=0x7f11bc007a40).stop 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 shutdown_connections 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11a0055060 0x7f11a0057520 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c815f5b0 0x7f11c80bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11c80615f0 0x7f11c80b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 --2- 172.21.15.181:0/3516392524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11c80d1830 0x7f11c80b8490 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 >> 172.21.15.181:0/3516392524 conn(0x7f11c80d69a0 msgr2=0x7f11c815c130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 shutdown_connections 2020-11-16T22:34:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.294+0000 7f11ce733700 1 -- 172.21.15.181:0/3516392524 wait complete. 2020-11-16T22:34:53.291 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "e8986808-291a-4767-bae7-a1c0041da596": { 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:34:53.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:53.293 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:34:53.293 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:53.298 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:34:53.299 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 Processor -- start 2020-11-16T22:34:53.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- start start 2020-11-16T22:34:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fedcc161480 0x7fedcc164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0c8db0 con 0x7fedcc164e10 2020-11-16T22:34:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0c8330 con 0x7fedcc161860 2020-11-16T22:34:53.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0c85d0 con 0x7fedcc161480 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fedcc161860 0x7fedcc0644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50140/0 (socket says 172.21.15.181:50140) 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcb7fe700 1 -- 172.21.15.181:0/2387445838 learned_addr learned my addr 172.21.15.181:0/2387445838 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcaffd700 1 --2- 172.21.15.181:0/2387445838 >> v2:172.21.15.175:3300/0 conn(0x7fedcc161480 0x7fedcc164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851712481 0 0) 0x7fedcc0c8330 con 0x7fedcc161860 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003240 con 0x7fedcc161860 2020-11-16T22:34:53.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 361732466 0 0) 0x7fedac003240 con 0x7fedcc161860 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcaffd700 1 -- 172.21.15.181:0/2387445838 >> v1:172.21.15.181:6789/0 conn(0x7fedcc161860 legacy=0x7fedcc0644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcaffd700 1 -- 172.21.15.181:0/2387445838 >> v1:172.21.15.175:6789/0 conn(0x7fedcc164e10 legacy=0x7fedcc062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcaffd700 1 -- 172.21.15.181:0/2387445838 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c8870 con 0x7fedcc161480 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcaffd700 1 --2- 172.21.15.181:0/2387445838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc161480 0x7fedcc164920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fedbc00b920 tx=0x7fedbc006df0).ready entity=mon.0 client_cookie=f52a0739dfdc9261 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4001860 con 0x7fedcc161480 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc011400 con 0x7fedcc161480 2020-11-16T22:34:53.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedca7fc700 1 -- 172.21.15.181:0/2387445838 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc011780 con 0x7fedcc161480 2020-11-16T22:34:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- 172.21.15.181:0/2387445838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc161480 msgr2=0x7fedcc164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2387445838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc161480 0x7fedcc164920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fedbc00b920 tx=0x7fedbc006df0).stop 2020-11-16T22:34:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- 172.21.15.181:0/2387445838 shutdown_connections 2020-11-16T22:34:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2387445838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc161480 0x7fedcc164920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- 172.21.15.181:0/2387445838 >> 172.21.15.181:0/2387445838 conn(0x7fedcc0d2dc0 msgr2=0x7fedcc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- 172.21.15.181:0/2387445838 shutdown_connections 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.430+0000 7fedcbfff700 1 -- 172.21.15.181:0/2387445838 wait complete. 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 Processor -- start 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- start start 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 0x7fedcc0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0cdee0 0x7fedcc0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ceb80 con 0x7fedcc0cdee0 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1738f0 con 0x7fedcc161860 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc173b90 con 0x7fedcc161480 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43682/0 (socket says 172.21.15.181:43682) 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 -- 172.21.15.181:0/2815568548 learned_addr learned my addr 172.21.15.181:0/2815568548 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedc27fc700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 0x7fedcc0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcb7fe700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0cdee0 0x7fedcc0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 -- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 msgr2=0x7fedcc0c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 0x7fedcc0c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 -- 172.21.15.181:0/2815568548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0cdee0 msgr2=0x7fedcc0ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0cdee0 0x7fedcc0ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 -- 172.21.15.181:0/2815568548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc173e30 con 0x7fedcc161480 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedc27fc700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 0x7fedcc0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcaffd700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fedbc009340 tx=0x7fedbc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc021070 con 0x7fedcc161480 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc009c00 con 0x7fedcc161480 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc1740d0 con 0x7fedcc161480 2020-11-16T22:34:53.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01d450 con 0x7fedcc161480 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.434+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0ce850 con 0x7fedcc161480 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fedbc00f070 con 0x7fedcc161480 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc044f60 con 0x7fedcc161480 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedc37fe700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 0x7feda8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fedbc06f550 con 0x7fedcc161480 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedc27fc700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 0x7feda8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.438+0000 7fedc27fc700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 0x7feda8057520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fedcc0c91e0 tx=0x7fedb4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.442+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fedbc0457b0 con 0x7fedcc161480 2020-11-16T22:34:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.638+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedcc0671e0 con 0x7fedcc161480 2020-11-16T22:34:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedc37fe700 1 -- 172.21.15.181:0/2815568548 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4669 (secure 0 0 0) 0x7fedbc048dd0 con 0x7fedcc161480 2020-11-16T22:34:53.631 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:53.632 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:51.894687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 msgr2=0x7feda8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 0x7feda8057520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fedcc0c91e0 tx=0x7fedb4009040).stop 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 msgr2=0x7fedcc0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fedbc009340 tx=0x7fedbc015000).stop 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 shutdown_connections 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7feda8055060 0x7feda8057520 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedcc0cdee0 0x7fedcc0ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc161480 0x7fedcc0c73f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 --2- 172.21.15.181:0/2815568548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc161860 0x7fedcc0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 >> 172.21.15.181:0/2815568548 conn(0x7fedcc0d2dc0 msgr2=0x7fedcc0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 shutdown_connections 2020-11-16T22:34:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.642+0000 7fedcbfff700 1 -- 172.21.15.181:0/2815568548 wait complete. 2020-11-16T22:34:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 105 2020-11-16T22:34:53.646 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'b', 'i'] 2020-11-16T22:34:53.647 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:34:53.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.774+0000 7f922f617700 1 Processor -- start 2020-11-16T22:34:53.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.774+0000 7f922f617700 1 -- start start 2020-11-16T22:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92280612a0 0x7f9228061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9228066420 0x7f922815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9228062b70 con 0x7f9228065fa0 2020-11-16T22:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9228061b70 con 0x7f9228066420 2020-11-16T22:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922815db10 con 0x7f92280612a0 2020-11-16T22:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92280612a0 0x7f9228061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92280612a0 0x7f9228061680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35848/0 (socket says 172.21.15.181:35848) 2020-11-16T22:34:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922cbb2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9228065fa0 0x7f922815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50154/0 (socket says 172.21.15.181:50154) 2020-11-16T22:34:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 -- 172.21.15.181:0/1706766544 learned_addr learned my addr 172.21.15.181:0/1706766544 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922dbb4700 1 --2- 172.21.15.181:0/1706766544 >> v2:172.21.15.175:3300/0 conn(0x7f9228066420 0x7f922815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 -- 172.21.15.181:0/1706766544 >> v2:172.21.15.175:3300/0 conn(0x7f9228066420 msgr2=0x7f922815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 --2- 172.21.15.181:0/1706766544 >> v2:172.21.15.175:3300/0 conn(0x7f9228066420 0x7f922815fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 -- 172.21.15.181:0/1706766544 >> v1:172.21.15.181:6789/0 conn(0x7f9228065fa0 legacy=0x7f922815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 -- 172.21.15.181:0/1706766544 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280cd790 con 0x7f92280612a0 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f9217fff700 1 -- 172.21.15.181:0/1706766544 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527889026 0 0) 0x7f9228062b70 con 0x7f9228065fa0 2020-11-16T22:34:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922d3b3700 1 --2- 172.21.15.181:0/1706766544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280612a0 0x7f9228061680 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f92280675a0 tx=0x7f9224006eb0).ready entity=mon.1 client_cookie=55d63d0df12d2eb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f9217fff700 1 -- 172.21.15.181:0/1706766544 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240098a0 con 0x7f92280612a0 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f9217fff700 1 -- 172.21.15.181:0/1706766544 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9224009a00 con 0x7f92280612a0 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f9217fff700 1 -- 172.21.15.181:0/1706766544 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009d80 con 0x7f92280612a0 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- 172.21.15.181:0/1706766544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280612a0 msgr2=0x7f9228061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 --2- 172.21.15.181:0/1706766544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280612a0 0x7f9228061680 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f92280675a0 tx=0x7f9224006eb0).stop 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- 172.21.15.181:0/1706766544 shutdown_connections 2020-11-16T22:34:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 --2- 172.21.15.181:0/1706766544 >> v2:172.21.15.175:3300/0 conn(0x7f9228066420 0x7f922815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 --2- 172.21.15.181:0/1706766544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280612a0 0x7f9228061680 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- 172.21.15.181:0/1706766544 >> 172.21.15.181:0/1706766544 conn(0x7f92280cf890 msgr2=0x7f92280cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- 172.21.15.181:0/1706766544 shutdown_connections 2020-11-16T22:34:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.778+0000 7f922f617700 1 -- 172.21.15.181:0/1706766544 wait complete. 2020-11-16T22:34:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 Processor -- start 2020-11-16T22:34:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 -- start start 2020-11-16T22:34:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228066420 0x7f922813bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280d1c40 0x7f922813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922816afe0 con 0x7f9228066420 2020-11-16T22:34:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922816b280 con 0x7f92280d1c40 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922f617700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922816b520 con 0x7f9228065fa0 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43696/0 (socket says 172.21.15.181:43696) 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922dbb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280d1c40 0x7f922813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 -- 172.21.15.181:0/3988288211 learned_addr learned my addr 172.21.15.181:0/3988288211 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922cbb2700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228066420 0x7f922813bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 -- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280d1c40 msgr2=0x7f922813c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280d1c40 0x7f922813c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 -- 172.21.15.181:0/3988288211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228066420 msgr2=0x7f922813bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228066420 0x7f922813bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 -- 172.21.15.181:0/3988288211 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922816b7c0 con 0x7f9228065fa0 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f922d3b3700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f922400b1b0 tx=0x7f9224017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.782+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224020070 con 0x7f9228065fa0 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9224010e70 con 0x7f9228065fa0 2020-11-16T22:34:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922401a690 con 0x7f9228065fa0 2020-11-16T22:34:53.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f922f617700 1 -- 172.21.15.181:0/3988288211 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922816ba60 con 0x7f9228065fa0 2020-11-16T22:34:53.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f922f617700 1 -- 172.21.15.181:0/3988288211 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922815f340 con 0x7f9228065fa0 2020-11-16T22:34:53.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f8004f90 con 0x7f9228065fa0 2020-11-16T22:34:53.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f922401aa80 con 0x7f9228065fa0 2020-11-16T22:34:53.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f92157fa700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:53.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9224071300 con 0x7f9228065fa0 2020-11-16T22:34:53.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f922cbb2700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:53.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.786+0000 7f922cbb2700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f92280d22d0 tx=0x7f9218007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:53.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:53.790+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f92240475e0 con 0x7f9228065fa0 2020-11-16T22:34:53.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f91f8001660 con 0x7f9228065fa0 2020-11-16T22:34:53.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f92157fa700 1 -- 172.21.15.181:0/3988288211 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1710 (secure 0 0 0) 0x7f922404ac00 con 0x7f9228065fa0 2020-11-16T22:34:53.992 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:53.992 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":112410,"bytes_used":61120512,"bytes_avail":773032992768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5374,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":106,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"i","status":"up:active","gid":7396},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7399},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7405}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2020-11-16T22:34:50.825308+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:34:53.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 msgr2=0x7f9200057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f92280d22d0 tx=0x7f9218007a20).stop 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 msgr2=0x7f92280d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f922400b1b0 tx=0x7f9224017000).stop 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 shutdown_connections 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9200055060 0x7f9200057520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92280d1c40 0x7f922813c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9228065fa0 0x7f92280d1750 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 --2- 172.21.15.181:0/3988288211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9228066420 0x7f922813bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:53.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 >> 172.21.15.181:0/3988288211 conn(0x7f92280cf890 msgr2=0x7f9228153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:53.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 shutdown_connections 2020-11-16T22:34:53.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:54.002+0000 7f9206ffd700 1 -- 172.21.15.181:0/3988288211 wait complete. 2020-11-16T22:34:54.006 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:34:55.008 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 Processor -- start 2020-11-16T22:34:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 -- start start 2020-11-16T22:34:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac167ac0 0x7fb7ac167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ac168390 con 0x7fb7ac1672c0 2020-11-16T22:34:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ac0b2060 con 0x7fb7ac0b3220 2020-11-16T22:34:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b3079700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac0b5f00 con 0x7fb7ac167ac0 2020-11-16T22:34:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac167ac0 0x7fb7ac167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac167ac0 0x7fb7ac167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35862/0 (socket says 172.21.15.181:35862) 2020-11-16T22:34:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 -- 172.21.15.181:0/1147925584 learned_addr learned my addr 172.21.15.181:0/1147925584 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 -- 172.21.15.181:0/1147925584 >> v1:172.21.15.181:6789/0 conn(0x7fb7ac0b3220 legacy=0x7fb7ac166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 -- 172.21.15.181:0/1147925584 >> v1:172.21.15.175:6789/0 conn(0x7fb7ac1672c0 legacy=0x7fb7ac161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 -- 172.21.15.181:0/1147925584 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac0b2300 con 0x7fb7ac167ac0 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7b0e15700 1 --2- 172.21.15.181:0/1147925584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac167ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fb794006fd0 tx=0x7fb794006eb0).ready entity=mon.1 client_cookie=906fc4dbf20ddb4b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7ab7fe700 1 -- 172.21.15.181:0/1147925584 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940098d0 con 0x7fb7ac167ac0 2020-11-16T22:34:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7ab7fe700 1 -- 172.21.15.181:0/1147925584 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794009a30 con 0x7fb7ac167ac0 2020-11-16T22:34:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.150+0000 7fb7ab7fe700 1 -- 172.21.15.181:0/1147925584 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794009db0 con 0x7fb7ac167ac0 2020-11-16T22:34:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 -- 172.21.15.181:0/1147925584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac167ac0 msgr2=0x7fb7ac167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 --2- 172.21.15.181:0/1147925584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac167ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fb794006fd0 tx=0x7fb794006eb0).stop 2020-11-16T22:34:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 -- 172.21.15.181:0/1147925584 shutdown_connections 2020-11-16T22:34:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 --2- 172.21.15.181:0/1147925584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac167ea0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 -- 172.21.15.181:0/1147925584 >> 172.21.15.181:0/1147925584 conn(0x7fb7ac060bc0 msgr2=0x7fb7ac0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 -- 172.21.15.181:0/1147925584 shutdown_connections 2020-11-16T22:34:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 -- 172.21.15.181:0/1147925584 wait complete. 2020-11-16T22:34:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.154+0000 7fb7b3079700 1 Processor -- start 2020-11-16T22:34:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- start start 2020-11-16T22:34:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac065500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac160fd0 con 0x7fb7ac167ac0 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0659f0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0b7810 con 0x7fb7ac0b3220 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b0e15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b0e15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43712/0 (socket says 172.21.15.181:43712) 2020-11-16T22:34:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35868/0 (socket says 172.21.15.181:35868) 2020-11-16T22:34:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b0e15700 1 -- 172.21.15.181:0/2573090129 learned_addr learned my addr 172.21.15.181:0/2573090129 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b1616700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac065500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 -- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 msgr2=0x7fb7ac0b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 -- 172.21.15.181:0/2573090129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ac167ac0 msgr2=0x7fb7ac065500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac065500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 -- 172.21.15.181:0/2573090129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac0b7ab0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b0e15700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7abfff700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fb79c00c710 tx=0x7fb79c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c013410 con 0x7fb7ac1672c0 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c013570 con 0x7fb7ac1672c0 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac0b7d50 con 0x7fb7ac1672c0 2020-11-16T22:34:55.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c013910 con 0x7fb7ac1672c0 2020-11-16T22:34:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.158+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1614a0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac069eb0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb79c010030 con 0x7fb7ac1672c0 2020-11-16T22:34:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7a8ff9700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 0x7fb788057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7b0e15700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 0x7fb788057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb79c06d940 con 0x7fb7ac1672c0 2020-11-16T22:34:55.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.162+0000 7fb7b0e15700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 0x7fb788057550 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb79400bf60 tx=0x7fb794015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.170+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb79c043ba0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.370+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7ac067070 con 0x7fb7ac1672c0 2020-11-16T22:34:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.370+0000 7fb7a8ff9700 1 -- 172.21.15.181:0/2573090129 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fb79c0471c0 con 0x7fb7ac1672c0 2020-11-16T22:34:55.363 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:55.363 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 msgr2=0x7fb788057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 0x7fb788057550 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb79400bf60 tx=0x7fb794015000).stop 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 msgr2=0x7fb7ac161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fb79c00c710 tx=0x7fb79c007b50).stop 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 shutdown_connections 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb788055090 0x7fb788057550 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7ac167ac0 0x7fb7ac065500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b3220 0x7fb7ac0b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 --2- 172.21.15.181:0/2573090129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac1672c0 0x7fb7ac161940 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 >> 172.21.15.181:0/2573090129 conn(0x7fb7ac060bc0 msgr2=0x7fb7ac0d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 shutdown_connections 2020-11-16T22:34:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.374+0000 7fb7b3079700 1 -- 172.21.15.181:0/2573090129 wait complete. 2020-11-16T22:34:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:34:55.378 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'b', 'i'] 2020-11-16T22:34:55.378 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:34:55.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.506+0000 7f47b0542700 1 Processor -- start 2020-11-16T22:34:55.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.506+0000 7f47b0542700 1 -- start start 2020-11-16T22:34:55.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47a8165880 0x7f47a80b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80b3310 con 0x7f47a80600f0 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a80b3120 con 0x7f47a8165880 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8060be0 con 0x7f47a80b44d0 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ae2de700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f47a80600f0 0x7f47a80604d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50182/0 (socket says 172.21.15.181:50182) 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43722/0 (socket says 172.21.15.181:43722) 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ae2de700 1 -- 172.21.15.181:0/2388914157 learned_addr learned my addr 172.21.15.181:0/2388914157 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:55.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47adadd700 1 --2- 172.21.15.181:0/2388914157 >> v2:172.21.15.175:3300/0 conn(0x7f47a8165880 0x7f47a80b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 -- 172.21.15.181:0/2388914157 >> v2:172.21.15.175:3300/0 conn(0x7f47a8165880 msgr2=0x7f47a80b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 --2- 172.21.15.181:0/2388914157 >> v2:172.21.15.175:3300/0 conn(0x7f47a8165880 0x7f47a80b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 -- 172.21.15.181:0/2388914157 >> v1:172.21.15.181:6789/0 conn(0x7f47a80600f0 legacy=0x7f47a80604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 -- 172.21.15.181:0/2388914157 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80b35b0 con 0x7f47a80b44d0 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ad2dc700 1 -- 172.21.15.181:0/2388914157 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666292219 0 0) 0x7f47a80b3310 con 0x7f47a80600f0 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47aeadf700 1 --2- 172.21.15.181:0/2388914157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f47a0009cc0 tx=0x7f47a00091d0).ready entity=mon.2 client_cookie=4b199db1511a7bcb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ad2dc700 1 -- 172.21.15.181:0/2388914157 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000bb20 con 0x7f47a80b44d0 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ad2dc700 1 -- 172.21.15.181:0/2388914157 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a000bc80 con 0x7f47a80b44d0 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47ad2dc700 1 -- 172.21.15.181:0/2388914157 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0011400 con 0x7f47a80b44d0 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- 172.21.15.181:0/2388914157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 msgr2=0x7f47a80b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 --2- 172.21.15.181:0/2388914157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f47a0009cc0 tx=0x7f47a00091d0).stop 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- 172.21.15.181:0/2388914157 shutdown_connections 2020-11-16T22:34:55.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 --2- 172.21.15.181:0/2388914157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80b44d0 0x7f47a80b4960 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 --2- 172.21.15.181:0/2388914157 >> v2:172.21.15.175:3300/0 conn(0x7f47a8165880 0x7f47a80b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- 172.21.15.181:0/2388914157 >> 172.21.15.181:0/2388914157 conn(0x7f47a80d3680 msgr2=0x7f47a80d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- 172.21.15.181:0/2388914157 shutdown_connections 2020-11-16T22:34:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.510+0000 7f47b0542700 1 -- 172.21.15.181:0/2388914157 wait complete. 2020-11-16T22:34:55.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 Processor -- start 2020-11-16T22:34:55.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 -- start start 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a80b44d0 0x7f47a81636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 0x7f47a8176a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a814e080 con 0x7f47a8165880 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8174950 con 0x7f47a80b44d0 2020-11-16T22:34:55.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47b0542700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8177540 con 0x7f47a80600f0 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43724/0 (socket says 172.21.15.181:43724) 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 -- 172.21.15.181:0/4213752387 learned_addr learned my addr 172.21.15.181:0/4213752387 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47adadd700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a80b44d0 0x7f47a81636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47aeadf700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 0x7f47a8176a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 -- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a80b44d0 msgr2=0x7f47a81636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a80b44d0 0x7f47a81636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 -- 172.21.15.181:0/4213752387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 msgr2=0x7f47a8176a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 0x7f47a8176a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 -- 172.21.15.181:0/4213752387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a81777e0 con 0x7f47a80600f0 2020-11-16T22:34:55.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47aeadf700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 0x7f47a8176a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:55.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.514+0000 7f47ae2de700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f47a400c740 tx=0x7f47a4007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40072e0 con 0x7f47a80600f0 2020-11-16T22:34:55.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4007440 con 0x7f47a80600f0 2020-11-16T22:34:55.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a8174b80 con 0x7f47a80600f0 2020-11-16T22:34:55.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4005800 con 0x7f47a80600f0 2020-11-16T22:34:55.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a8176f40 con 0x7f47a80600f0 2020-11-16T22:34:55.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8069eb0 con 0x7f47a80600f0 2020-11-16T22:34:55.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f47a4010070 con 0x7f47a80600f0 2020-11-16T22:34:55.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.518+0000 7f47967fc700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 0x7f4784057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:55.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.522+0000 7f47adadd700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 0x7f4784057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:55.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.522+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f47a406d640 con 0x7f47a80600f0 2020-11-16T22:34:55.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.522+0000 7f47adadd700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 0x7f4784057520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f47a80b6fc0 tx=0x7f4798006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.530+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f47a4043790 con 0x7f47a80600f0 2020-11-16T22:34:55.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.738+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f47a80d44b0 con 0x7f47a80600f0 2020-11-16T22:34:55.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.738+0000 7f47967fc700 1 -- 172.21.15.181:0/4213752387 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1710 (secure 0 0 0) 0x7f47a4007c00 con 0x7f47a80600f0 2020-11-16T22:34:55.729 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:55.730 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":112410,"bytes_used":61120512,"bytes_avail":773032992768,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5374,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":107,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"i","status":"up:active","gid":7396},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7399},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7405}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2020-11-16T22:34:50.825308+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:34:55.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.738+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 msgr2=0x7f4784057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 0x7f4784057520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f47a80b6fc0 tx=0x7f4798006df0).stop 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 msgr2=0x7f47a81631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f47a400c740 tx=0x7f47a4007ac0).stop 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 shutdown_connections 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4784055060 0x7f4784057520 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8165880 0x7f47a8176a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47a80600f0 0x7f47a81631b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 --2- 172.21.15.181:0/4213752387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a80b44d0 0x7f47a81636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:55.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 >> 172.21.15.181:0/4213752387 conn(0x7f47a80d3680 msgr2=0x7f47a80d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:55.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 shutdown_connections 2020-11-16T22:34:55.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:55.742+0000 7f47b0542700 1 -- 172.21.15.181:0/4213752387 wait complete. 2020-11-16T22:34:55.743 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:34:56.745 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:56.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.881+0000 7fe5b1846700 1 Processor -- start 2020-11-16T22:34:56.875 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.881+0000 7fe5b1846700 1 -- start start 2020-11-16T22:34:56.877 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5b1846700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5ac160430 0x7fe5ac161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5b1846700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5b1846700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac160ca0 con 0x7fe5ac0c85e0 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5b1846700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c8a40 con 0x7fe5ac160430 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5b1846700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0cf030 con 0x7fe5ac160810 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:56.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43736/0 (socket says 172.21.15.181:43736) 2020-11-16T22:34:56.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/4232047487 learned_addr learned my addr 172.21.15.181:0/4232047487 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:56.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5aaffd700 1 --2- 172.21.15.181:0/4232047487 >> v2:172.21.15.175:3300/0 conn(0x7fe5ac160430 0x7fe5ac161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:56.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/4232047487 >> v2:172.21.15.175:3300/0 conn(0x7fe5ac160430 msgr2=0x7fe5ac161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:56.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 --2- 172.21.15.181:0/4232047487 >> v2:172.21.15.175:3300/0 conn(0x7fe5ac160430 0x7fe5ac161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:56.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/4232047487 >> v1:172.21.15.175:6789/0 conn(0x7fe5ac0c85e0 legacy=0x7fe5ac162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:56.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/4232047487 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac161a10 con 0x7fe5ac160810 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5ab7fe700 1 --2- 172.21.15.181:0/4232047487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe5a4006fd0 tx=0x7fe5a4006eb0).ready entity=mon.2 client_cookie=b26e137267c7fad0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5a9ffb700 1 -- 172.21.15.181:0/4232047487 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40098d0 con 0x7fe5ac160810 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5a9ffb700 1 -- 172.21.15.181:0/4232047487 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4009a30 con 0x7fe5ac160810 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.885+0000 7fe5a9ffb700 1 -- 172.21.15.181:0/4232047487 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4009db0 con 0x7fe5ac160810 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- 172.21.15.181:0/4232047487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 msgr2=0x7fe5ac0600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- 172.21.15.181:0/4232047487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe5a4006fd0 tx=0x7fe5a4006eb0).stop 2020-11-16T22:34:56.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- 172.21.15.181:0/4232047487 shutdown_connections 2020-11-16T22:34:56.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- 172.21.15.181:0/4232047487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:56.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- 172.21.15.181:0/4232047487 >> v2:172.21.15.175:3300/0 conn(0x7fe5ac160430 0x7fe5ac161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:56.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- 172.21.15.181:0/4232047487 >> 172.21.15.181:0/4232047487 conn(0x7fe5ac0d3680 msgr2=0x7fe5ac0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:56.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- 172.21.15.181:0/4232047487 shutdown_connections 2020-11-16T22:34:56.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- 172.21.15.181:0/4232047487 wait complete. 2020-11-16T22:34:56.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 Processor -- start 2020-11-16T22:34:56.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- start start 2020-11-16T22:34:56.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 0x7fe5ac0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:56.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac160430 0x7fe5ac1662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:56.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1706f0 con 0x7fe5ac160430 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac170990 con 0x7fe5ac160810 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5b1846700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac170c30 con 0x7fe5ac0c85e0 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 0x7fe5ac0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.889+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 0x7fe5ac0c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43738/0 (socket says 172.21.15.181:43738) 2020-11-16T22:34:56.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5aaffd700 1 -- 172.21.15.181:0/1963816071 learned_addr learned my addr 172.21.15.181:0/1963816071 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:56.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5aa7fc700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac160430 0x7fe5ac1662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:56.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 msgr2=0x7fe5ac0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:56.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 0x7fe5ac0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/1963816071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac160430 msgr2=0x7fe5ac1662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:57.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac160430 0x7fe5ac1662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 -- 172.21.15.181:0/1963816071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac170ed0 con 0x7fe5ac160810 2020-11-16T22:34:57.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5ab7fe700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fe5a4006fd0 tx=0x7fe5a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:57.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a401d5d0 con 0x7fe5ac160810 2020-11-16T22:34:57.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a401d730 con 0x7fe5ac160810 2020-11-16T22:34:57.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac171170 con 0x7fe5ac160810 2020-11-16T22:34:57.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a401dad0 con 0x7fe5ac160810 2020-11-16T22:34:57.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0c99a0 con 0x7fe5ac160810 2020-11-16T22:34:57.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe5a401a040 con 0x7fe5ac160810 2020-11-16T22:34:57.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac069eb0 con 0x7fe5ac160810 2020-11-16T22:34:57.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 0x7fe588057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe5a406f340 con 0x7fe5ac160810 2020-11-16T22:34:57.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5aaffd700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 0x7fe588057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:57.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.893+0000 7fe5aaffd700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 0x7fe588057520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fe5ac15d3c0 tx=0x7fe59c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:57.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:56.897+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe5a40456d0 con 0x7fe5ac160810 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5ac160e80 con 0x7fe5ac160810 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe592ffd700 1 -- 172.21.15.181:0/1963816071 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fe5a4048cf0 con 0x7fe5ac160810 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 msgr2=0x7fe588057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 0x7fe588057520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fe5ac15d3c0 tx=0x7fe59c008040).stop 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 msgr2=0x7fe5ac0600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fe5a4006fd0 tx=0x7fe5a4015000).stop 2020-11-16T22:34:57.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 shutdown_connections 2020-11-16T22:34:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe588055060 0x7fe588057520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5ac160810 0x7fe5ac0600f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5ac0c85e0 0x7fe5ac0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 --2- 172.21.15.181:0/1963816071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac160430 0x7fe5ac1662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 >> 172.21.15.181:0/1963816071 conn(0x7fe5ac0d3680 msgr2=0x7fe5ac0cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:57.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 shutdown_connections 2020-11-16T22:34:57.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.097+0000 7fe5b1846700 1 -- 172.21.15.181:0/1963816071 wait complete. 2020-11-16T22:34:57.499 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:34:57.499 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:57.499 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:57.703 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'b', 'i'] 2020-11-16T22:34:57.704 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:34:57.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 Processor -- start 2020-11-16T22:34:57.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- start start 2020-11-16T22:34:57.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6774161860 0x7f6774062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.836 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740c8db0 con 0x7f6774161480 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6774165030 con 0x7f6774161860 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67740c8330 con 0x7f6774062740 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43750/0 (socket says 172.21.15.181:43750) 2020-11-16T22:34:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 -- 172.21.15.181:0/2951443314 learned_addr learned my addr 172.21.15.181:0/2951443314 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:57.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f67737fe700 1 --2- 172.21.15.181:0/2951443314 >> v2:172.21.15.175:3300/0 conn(0x7f6774161860 0x7f6774062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:57.838 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 -- 172.21.15.181:0/2951443314 >> v2:172.21.15.175:3300/0 conn(0x7f6774161860 msgr2=0x7f6774062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 --2- 172.21.15.181:0/2951443314 >> v2:172.21.15.175:3300/0 conn(0x7f6774161860 0x7f6774062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 -- 172.21.15.181:0/2951443314 >> v1:172.21.15.181:6789/0 conn(0x7f6774161480 legacy=0x7f6774164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 -- 172.21.15.181:0/2951443314 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740c85d0 con 0x7f6774062740 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6778814700 1 --2- 172.21.15.181:0/2951443314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f676c009d50 tx=0x7f676c009290).ready entity=mon.2 client_cookie=19a38ae419592ddb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6772ffd700 1 -- 172.21.15.181:0/2951443314 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00bc80 con 0x7f6774062740 2020-11-16T22:34:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6772ffd700 1 -- 172.21.15.181:0/2951443314 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c00bde0 con 0x7f6774062740 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6772ffd700 1 -- 172.21.15.181:0/2951443314 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011610 con 0x7f6774062740 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- 172.21.15.181:0/2951443314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 msgr2=0x7f67740644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 --2- 172.21.15.181:0/2951443314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f676c009d50 tx=0x7f676c009290).stop 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- 172.21.15.181:0/2951443314 shutdown_connections 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 --2- 172.21.15.181:0/2951443314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67740644e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 --2- 172.21.15.181:0/2951443314 >> v2:172.21.15.175:3300/0 conn(0x7f6774161860 0x7f6774062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- 172.21.15.181:0/2951443314 >> 172.21.15.181:0/2951443314 conn(0x7f67740d2dc0 msgr2=0x7f67740ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- 172.21.15.181:0/2951443314 shutdown_connections 2020-11-16T22:34:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.845+0000 7f6779015700 1 -- 172.21.15.181:0/2951443314 wait complete. 2020-11-16T22:34:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 Processor -- start 2020-11-16T22:34:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- start start 2020-11-16T22:34:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67741736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 0x7f67740c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 0x7f67740c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774173fa0 con 0x7f6774161860 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774174240 con 0x7f6774161480 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67741744e0 con 0x7f6774062740 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6773fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67741736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:57.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f67737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 0x7f67740c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6773fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67741736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43752/0 (socket says 172.21.15.181:43752) 2020-11-16T22:34:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f67737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 0x7f67740c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35912/0 (socket says 172.21.15.181:35912) 2020-11-16T22:34:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6773fff700 1 -- 172.21.15.181:0/4118783227 learned_addr learned my addr 172.21.15.181:0/4118783227 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 0x7f67740c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:58.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 -- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 msgr2=0x7f67741736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67741736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 -- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 msgr2=0x7f67740c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 0x7f67740c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 -- 172.21.15.181:0/4118783227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774161e40 con 0x7f6774161860 2020-11-16T22:34:58.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6778814700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 0x7f67740c78e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f676c009d50 tx=0x7f676c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:58.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00bc80 con 0x7f6774161860 2020-11-16T22:34:58.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c009e90 con 0x7f6774161860 2020-11-16T22:34:58.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67741620e0 con 0x7f6774161860 2020-11-16T22:34:58.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c010cb0 con 0x7f6774161860 2020-11-16T22:34:58.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.849+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6774162580 con 0x7f6774161860 2020-11-16T22:34:58.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6774173e00 con 0x7f6774161860 2020-11-16T22:34:58.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f676c017070 con 0x7f6774161860 2020-11-16T22:34:58.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6753fff700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 0x7f674c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:58.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6773fff700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 0x7f674c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:58.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f676c06fee0 con 0x7f6774161860 2020-11-16T22:34:58.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.853+0000 7f6773fff700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 0x7f674c057520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f67640033a0 tx=0x7f6764007be0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:58.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:57.861+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6774173e00 con 0x7f6774161860 2020-11-16T22:34:58.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6774173e00 con 0x7f6774161860 2020-11-16T22:34:58.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6753fff700 1 -- 172.21.15.181:0/4118783227 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1813 (secure 0 0 0) 0x7f6774173e00 con 0x7f6774161860 2020-11-16T22:34:58.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 msgr2=0x7f674c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 0x7f674c057520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f67640033a0 tx=0x7f6764007be0).stop 2020-11-16T22:34:58.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 msgr2=0x7f67740c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 0x7f67740c78e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f676c009d50 tx=0x7f676c016000).stop 2020-11-16T22:34:58.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 shutdown_connections 2020-11-16T22:34:58.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f674c055060 0x7f674c057520 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6774161860 0x7f67740c78e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6774062740 0x7f67741736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 --2- 172.21.15.181:0/4118783227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6774161480 0x7f67740c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 >> 172.21.15.181:0/4118783227 conn(0x7f67740d2dc0 msgr2=0x7f6774063140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:58.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 shutdown_connections 2020-11-16T22:34:58.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.073+0000 7f6779015700 1 -- 172.21.15.181:0/4118783227 wait complete. 2020-11-16T22:34:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:58.455 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":112410,"bytes_used":61120512,"bytes_avail":773032992768,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":107,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"i","status":"up:active","gid":7396},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7399},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7405}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2020-11-16T22:34:56.827363+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2020-11-16T22:34:58.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]', 'i': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:34:58.456 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:58.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 Processor -- start 2020-11-16T22:34:58.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- start start 2020-11-16T22:34:58.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c85e0 con 0x7f4c0815d520 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c8840 con 0x7f4c0815d140 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815ff70 con 0x7f4c080656e0 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43764/0 (socket says 172.21.15.181:43764) 2020-11-16T22:34:58.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 -- 172.21.15.181:0/2940749201 learned_addr learned my addr 172.21.15.181:0/2940749201 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:58.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c077fe700 1 --1- 172.21.15.181:0/2940749201 >> v1:172.21.15.181:6789/0 conn(0x7f4c0815d140 0x7f4c080cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50224/0 (socket says 172.21.15.181:50224) 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c067fc700 1 -- 172.21.15.181:0/2940749201 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541844395 0 0) 0x7f4c080c8840 con 0x7f4c0815d140 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c067fc700 1 -- 172.21.15.181:0/2940749201 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf4003240 con 0x7f4c0815d140 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 -- 172.21.15.181:0/2940749201 >> v1:172.21.15.181:6789/0 conn(0x7f4c0815d140 legacy=0x7f4c080cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 -- 172.21.15.181:0/2940749201 >> v1:172.21.15.175:6789/0 conn(0x7f4c0815d520 legacy=0x7f4c08064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 -- 172.21.15.181:0/2940749201 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080c8ae0 con 0x7f4c080656e0 2020-11-16T22:34:58.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c07fff700 1 --2- 172.21.15.181:0/2940749201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f4c00009d80 tx=0x7f4c00009290).ready entity=mon.2 client_cookie=5a0825763187dab4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:58.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c067fc700 1 -- 172.21.15.181:0/2940749201 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000bbc0 con 0x7f4c080656e0 2020-11-16T22:34:58.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c067fc700 1 -- 172.21.15.181:0/2940749201 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0000bd20 con 0x7f4c080656e0 2020-11-16T22:34:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c067fc700 1 -- 172.21.15.181:0/2940749201 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00011610 con 0x7f4c080656e0 2020-11-16T22:34:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2940749201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 msgr2=0x7f4c08160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2940749201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f4c00009d80 tx=0x7f4c00009290).stop 2020-11-16T22:34:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2940749201 shutdown_connections 2020-11-16T22:34:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2940749201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c08160360 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2940749201 >> 172.21.15.181:0/2940749201 conn(0x7f4c080d2dc0 msgr2=0x7f4c080ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2940749201 shutdown_connections 2020-11-16T22:34:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.597+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2940749201 wait complete. 2020-11-16T22:34:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 Processor -- start 2020-11-16T22:34:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 -- start start 2020-11-16T22:34:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c080c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 0x7f4c080c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815c180 con 0x7f4c0815d140 2020-11-16T22:34:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08142db0 con 0x7f4c0815d520 2020-11-16T22:34:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c0ca1f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080ccee0 con 0x7f4c080656e0 2020-11-16T22:34:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c077fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c080c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35928/0 (socket says 172.21.15.181:35928) 2020-11-16T22:34:59.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.601+0000 7f4c07fff700 1 -- 172.21.15.181:0/2635293746 learned_addr learned my addr 172.21.15.181:0/2635293746 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c06ffd700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 0x7f4c080c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 -- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 msgr2=0x7f4c080c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c080c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 -- 172.21.15.181:0/2635293746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 msgr2=0x7f4c080c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 0x7f4c080c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 -- 172.21.15.181:0/2635293746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080cd180 con 0x7f4c0815d520 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c06ffd700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 0x7f4c080c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c07fff700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f4c00009cd0 tx=0x7f4c00016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00011650 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c00011e70 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c080cd420 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00010da0 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c080cd960 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c08044f60 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4c00017070 con 0x7f4c0815d520 2020-11-16T22:34:59.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 0x7f4be0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4c077fe700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 0x7f4be0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.605+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c0006ff40 con 0x7f4c0815d520 2020-11-16T22:34:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.609+0000 7f4c077fe700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 0x7f4be0057520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f4bf8007f60 tx=0x7f4bf8007940).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.613+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4c00046220 con 0x7f4c0815d520 2020-11-16T22:34:59.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.817+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c080c8250 con 0x7f4c0815d520 2020-11-16T22:34:59.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.817+0000 7f4be77fe700 1 -- 172.21.15.181:0/2635293746 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7f4c00015070 con 0x7f4c0815d520 2020-11-16T22:34:59.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 msgr2=0x7f4be0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 0x7f4be0057520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f4bf8007f60 tx=0x7f4bf8007940).stop 2020-11-16T22:34:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 msgr2=0x7f4c080ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f4c00009cd0 tx=0x7f4c00016000).stop 2020-11-16T22:34:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 shutdown_connections 2020-11-16T22:34:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4be0055060 0x7f4be0057520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0815d520 0x7f4c080ce0a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c080656e0 0x7f4c080c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 --2- 172.21.15.181:0/2635293746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c0815d140 0x7f4c080c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 >> 172.21.15.181:0/2635293746 conn(0x7f4c080d2dc0 msgr2=0x7f4c080d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 shutdown_connections 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:58.821+0000 7f4c0ca1f700 1 -- 172.21.15.181:0/2635293746 wait complete. 2020-11-16T22:34:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:34:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:59.212 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:59.414 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub resume 2020-11-16T22:34:59.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.561+0000 7f4433fff700 1 Processor -- start 2020-11-16T22:34:59.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4433fff700 1 -- start start 2020-11-16T22:34:59.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4433fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f443415d080 0x7f44340cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4433fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4434065150 con 0x7f44340600f0 2020-11-16T22:34:59.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4433fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44340653f0 con 0x7f443415d460 2020-11-16T22:34:59.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4433fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4434066a20 con 0x7f443415d080 2020-11-16T22:34:59.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44327fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f443415d460 0x7f4434066310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50236/0 (socket says 172.21.15.181:50236) 2020-11-16T22:34:59.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44327fc700 1 -- 172.21.15.181:0/1839124008 learned_addr learned my addr 172.21.15.181:0/1839124008 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 --2- 172.21.15.181:0/1839124008 >> v2:172.21.15.175:3300/0 conn(0x7f443415d080 0x7f44340cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716270088 0 0) 0x7f44340653f0 con 0x7f443415d460 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4420003240 con 0x7f443415d460 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821002047 0 0) 0x7f4434065150 con 0x7f44340600f0 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44340653f0 con 0x7f44340600f0 2020-11-16T22:34:59.559 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3283442130 0 0) 0x7f4420003240 con 0x7f443415d460 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 >> v2:172.21.15.175:3300/0 conn(0x7f443415d080 msgr2=0x7f44340cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 --2- 172.21.15.181:0/1839124008 >> v2:172.21.15.175:3300/0 conn(0x7f443415d080 0x7f44340cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 >> v1:172.21.15.175:6789/0 conn(0x7f44340600f0 legacy=0x7f4434060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4434065690 con 0x7f443415d460 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f44280014e0 con 0x7f443415d460 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 >> v1:172.21.15.181:6789/0 conn(0x7f443415d460 legacy=0x7f4434066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f4420003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44200044e0 0x7f4420004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4420004e50 0x7f442000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434066a20 con 0x7f44200044e0 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200053b0 con 0x7f44200038c0 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442000bca0 con 0x7f4420004e50 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f4420003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44327fc700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44200044e0 0x7f4420004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44337fe700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4420004e50 0x7f442000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 -- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4420004e50 msgr2=0x7f442000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4420004e50 0x7f442000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 -- 172.21.15.181:0/1839124008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44200044e0 msgr2=0x7f4420004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44200044e0 0x7f4420004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 -- 172.21.15.181:0/1839124008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420003240 con 0x7f44200038c0 2020-11-16T22:34:59.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44337fe700 1 -- 172.21.15.181:0/1839124008 reap_dead start 2020-11-16T22:34:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f44327fc700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44200044e0 0x7f4420004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4432ffd700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f4420003cc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4424009ea0 tx=0x7f44240091d0).ready entity=mon.1 client_cookie=e55bd25949321620 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400ec90 con 0x7f44200038c0 2020-11-16T22:34:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400edf0 con 0x7f44200038c0 2020-11-16T22:34:59.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.565+0000 7f4431ffb700 1 -- 172.21.15.181:0/1839124008 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424012600 con 0x7f44200038c0 2020-11-16T22:34:59.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 msgr2=0x7f4420003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f4420003cc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4424009ea0 tx=0x7f44240091d0).stop 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- 172.21.15.181:0/1839124008 shutdown_connections 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 --2- 172.21.15.181:0/1839124008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f4420003cc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- 172.21.15.181:0/1839124008 >> 172.21.15.181:0/1839124008 conn(0x7f44340cfe90 msgr2=0x7f44340d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- 172.21.15.181:0/1839124008 shutdown_connections 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- 172.21.15.181:0/1839124008 wait complete. 2020-11-16T22:34:59.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 Processor -- start 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.569+0000 7f4433fff700 1 -- start start 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 0x7f44340b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44340b5270 0x7f4434156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340b56d0 con 0x7f44340c89c0 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443416e3f0 con 0x7f44200038c0 2020-11-16T22:34:59.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44340c8e40 con 0x7f44340b5270 2020-11-16T22:34:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44340b5270 0x7f4434156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35944/0 (socket says 172.21.15.181:35944) 2020-11-16T22:34:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 -- 172.21.15.181:0/3958744929 learned_addr learned my addr 172.21.15.181:0/3958744929 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f44327fc700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 0x7f44340b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 -- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44340b5270 msgr2=0x7f4434156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44340b5270 0x7f4434156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 -- 172.21.15.181:0/3958744929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 msgr2=0x7f44340b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 0x7f44340b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 -- 172.21.15.181:0/3958744929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443416d550 con 0x7f44200038c0 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f44327fc700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 0x7f44340b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4432ffd700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f4424001150 tx=0x7f442401a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f441affd700 1 -- 172.21.15.181:0/3958744929 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424012570 con 0x7f44200038c0 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f441affd700 1 -- 172.21.15.181:0/3958744929 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442401b630 con 0x7f44200038c0 2020-11-16T22:34:59.569 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f441affd700 1 -- 172.21.15.181:0/3958744929 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401b9f0 con 0x7f44200038c0 2020-11-16T22:34:59.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443416d7f0 con 0x7f44200038c0 2020-11-16T22:34:59.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.573+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443416dd30 con 0x7f44200038c0 2020-11-16T22:34:59.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.577+0000 7f441affd700 1 -- 172.21.15.181:0/3958744929 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f442402a020 con 0x7f44200038c0 2020-11-16T22:34:59.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.577+0000 7f441affd700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 0x7f4410057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.577+0000 7f441affd700 1 -- 172.21.15.181:0/3958744929 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f442406fa30 con 0x7f44200038c0 2020-11-16T22:34:59.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.577+0000 7f44327fc700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 0x7f4410057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.577+0000 7f44327fc700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 0x7f4410057520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f4428000fb0 tx=0x7f4428009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 Processor -- start 2020-11-16T22:34:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 -- start start 2020-11-16T22:34:59.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x27e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 0x27e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 0x27dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x266e4b0 con 0x25c47b0 2020-11-16T22:34:59.575 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2722930 con 0x27e2fd0 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25e26f0 con 0x2556bb0 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44327fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 0x27e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 0x27dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44327fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 0x27e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43796/0 (socket says 172.21.15.181:43796) 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f44327fc700 1 -- 172.21.15.181:0/996520912 learned_addr learned my addr 172.21.15.181:0/996520912 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.581+0000 7f4432ffd700 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x27e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 -- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 msgr2=0x27e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 0x27e3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 -- 172.21.15.181:0/996520912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 msgr2=0x27e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x27e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 -- 172.21.15.181:0/996520912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25dd5a0 con 0x27e2fd0 2020-11-16T22:34:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44337fe700 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 0x27dc700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f44340b23a0 tx=0x7f442c007b00).ready entity=mon.1 client_cookie=f7371fab2b27b9cb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f440bfff700 1 -- 172.21.15.181:0/996520912 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c007240 con 0x27e2fd0 2020-11-16T22:34:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f440bfff700 1 -- 172.21.15.181:0/996520912 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c0073a0 con 0x27e2fd0 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f440bfff700 1 -- 172.21.15.181:0/996520912 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0029f0 con 0x27e2fd0 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 msgr2=0x27dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 0x27dc700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f44340b23a0 tx=0x7f442c007b00).stop 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- 172.21.15.181:0/996520912 shutdown_connections 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x27e2fd0 0x27dc700 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x27e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 --2- 172.21.15.181:0/996520912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2556bb0 0x27e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- 172.21.15.181:0/996520912 >> 172.21.15.181:0/996520912 conn(0x2792e40 msgr2=0x27df3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- 172.21.15.181:0/996520912 shutdown_connections 2020-11-16T22:34:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- 172.21.15.181:0/996520912 wait complete. 2020-11-16T22:34:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 -- start start 2020-11-16T22:34:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.585+0000 7f44482ac740 1 Processor -- start 2020-11-16T22:34:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x28422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 0x27df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x270f6e0 con 0x2556bb0 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x270f820 con 0x25c47b0 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2710a90 con 0x27e2fd0 2020-11-16T22:34:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 10 client.0 ll_register_callbacks cb 0x2705ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44327fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 0x27df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44327fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 0x27df4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35960/0 (socket says 172.21.15.181:35960) 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43802/0 (socket says 172.21.15.181:43802) 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44327fc700 1 -- 172.21.15.181:0/64946425 learned_addr learned my addr 172.21.15.181:0/64946425 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f4432ffd700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x28422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 -- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 msgr2=0x27df4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 0x27df4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 -- 172.21.15.181:0/64946425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 msgr2=0x28422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x28422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 -- 172.21.15.181:0/64946425 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2700280 con 0x27e2fd0 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44337fe700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f442c007ad0 tx=0x7f442c01b000).ready entity=mon.2 client_cookie=2454c857e307585b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:34:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0037b0 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27c36c0 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f442c018870 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442c018c40 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 10 client.7193 fetch_fsmap learned FSMap version 107 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x2535fd0 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27c36c0 con 0x27e2fd0 2020-11-16T22:34:59.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f442c01f040 con 0x27e2fd0 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 10 client.7193 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 10 client.7193 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 20 client.7193 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x28308c0 0x27e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 4 client.7193 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2600c50 con 0x28308c0 2020-11-16T22:34:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f4432ffd700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x28308c0 0x27e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 10 client.7193 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f4432ffd700 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x28308c0 0x27e4a70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8153edeb55045721 server_cookie=ca2488f720a95c3c in_seq=0 out_seq=0 2020-11-16T22:34:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 1 -- 172.21.15.181:0/64946425 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2600c50 con 0x28308c0 2020-11-16T22:34:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44097fa700 10 client.7193 handle_command_reply: tid=0 2020-11-16T22:34:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.589+0000 7f44482ac740 1 client.7193 shutdown 2020-11-16T22:34:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 msgr2=0x28364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f442c007ad0 tx=0x7f442c01b000).stop 2020-11-16T22:34:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 shutdown_connections 2020-11-16T22:34:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x28308c0 0x27e4a70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x27e2fd0 0x28364b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44097fa700 0 client.7193 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x28422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 --2- 172.21.15.181:0/64946425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25c47b0 0x27df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 >> 172.21.15.181:0/64946425 conn(0x2792e40 msgr2=0x2818a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 shutdown_connections 2020-11-16T22:34:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 1 -- 172.21.15.181:0/64946425 wait complete. 2020-11-16T22:34:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.593+0000 7f44482ac740 20 client.7193 trim_cache size 0 max 16384 2020-11-16T22:34:59.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 Processor -- start 2020-11-16T22:34:59.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- start start 2020-11-16T22:34:59.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 0x2836fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x282cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 0x28129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26e7e40 con 0x2556bb0 2020-11-16T22:34:59.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2836690 con 0x2792e40 2020-11-16T22:34:59.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x27ddb50 con 0x25c47b0 2020-11-16T22:34:59.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 0x2836fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 0x28129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 0x2836fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35966/0 (socket says 172.21.15.181:35966) 2020-11-16T22:34:59.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4432ffd700 1 -- 172.21.15.181:0/3933089114 learned_addr learned my addr 172.21.15.181:0/3933089114 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44327fc700 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x282cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 -- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 msgr2=0x2836fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 0x2836fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 -- 172.21.15.181:0/3933089114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 msgr2=0x282cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x282cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 -- 172.21.15.181:0/3933089114 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000081d0 con 0x25c47b0 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44337fe700 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 0x28129d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f442c003ed0 tx=0x7f442c021000).ready entity=mon.2 client_cookie=d1fe5d685d56e199 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4418ff9700 1 -- 172.21.15.181:0/3933089114 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c003800 con 0x25c47b0 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4418ff9700 1 -- 172.21.15.181:0/3933089114 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c0180c0 con 0x25c47b0 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f4418ff9700 1 -- 172.21.15.181:0/3933089114 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c018460 con 0x25c47b0 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 msgr2=0x28129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 0x28129d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f442c003ed0 tx=0x7f442c021000).stop 2020-11-16T22:34:59.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- 172.21.15.181:0/3933089114 shutdown_connections 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25c47b0 0x28129d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2792e40 0x2836fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 --2- 172.21.15.181:0/3933089114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2556bb0 0x282cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- 172.21.15.181:0/3933089114 >> 172.21.15.181:0/3933089114 conn(0x7f4400002320 msgr2=0x2831340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- 172.21.15.181:0/3933089114 shutdown_connections 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- 172.21.15.181:0/3933089114 wait complete. 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 -- start start 2020-11-16T22:34:59.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.613+0000 7f44482ac740 1 Processor -- start 2020-11-16T22:34:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x2851860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2792e40 0x282cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x28303f0 con 0x25c47b0 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2812a70 con 0x2556bb0 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2812d10 con 0x2792e40 2020-11-16T22:34:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 10 client.0 ll_register_callbacks cb 0x27e3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44337fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2792e40 0x282cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35972/0 (socket says 172.21.15.181:35972) 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 -- 172.21.15.181:0/2541174284 learned_addr learned my addr 172.21.15.181:0/2541174284 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44327fc700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x2851860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 -- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2792e40 msgr2=0x282cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2792e40 0x282cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 -- 172.21.15.181:0/2541174284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 msgr2=0x2851860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x2851860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 -- 172.21.15.181:0/2541174284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2574720 con 0x2556bb0 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f4432ffd700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f44240094d0 tx=0x7f442402f040).ready entity=mon.1 client_cookie=51e3a7dc2ab57430 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424033040 con 0x2556bb0 2020-11-16T22:34:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f442406db90 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f442c0073a0 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442406dd50 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 10 client.7876 fetch_fsmap learned FSMap version 107 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x257fe00 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c0073a0 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f4424040070 con 0x2556bb0 2020-11-16T22:34:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 10 client.7876 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 10 client.7876 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 20 client.7876 populate_metadata read hostname 'smithi181' 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x27e2fd0 0x28a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 4 client.7876 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,r,e,s,u,m,e,",}] 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " s c r u b r e s u m e " }) v1 -- 0x2607f50 con 0x27e2fd0 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44327fc700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x27e2fd0 0x28a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 10 client.7876 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44327fc700 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x27e2fd0 0x28a15d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=473c629bfdeba4c3 server_cookie=e9e2e586b70dee6c in_seq=0 out_seq=0 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 1 -- 172.21.15.181:0/2541174284 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x2607f50 con 0x27e2fd0 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 10 client.7876 handle_command_reply: tid=0 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 client.7876 shutdown 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 msgr2=0x289d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f44240094d0 tx=0x7f442402f040).stop 2020-11-16T22:34:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 shutdown_connections 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x27e2fd0 0x28a15d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2792e40 0x282cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44097fa700 0 client.7876 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2556bb0 0x289d150 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 --2- 172.21.15.181:0/2541174284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x25c47b0 0x2851860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.617+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 >> 172.21.15.181:0/2541174284 conn(0x7f4400002320 msgr2=0x282f520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 shutdown_connections 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f44482ac740 1 -- 172.21.15.181:0/2541174284 wait complete. 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f44482ac740 20 client.7876 trim_cache size 0 max 16384 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 msgr2=0x7f4410057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 0x7f4410057520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f4428000fb0 tx=0x7f4428009040).stop 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 msgr2=0x7f443416cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f4424001150 tx=0x7f442401a000).stop 2020-11-16T22:34:59.620 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 shutdown_connections 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4410055060 0x7f4410057520 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44340b5270 0x7f4434156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44200038c0 0x7f443416cf60 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 --2- 172.21.15.181:0/3958744929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44340c89c0 0x7f44340b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 >> 172.21.15.181:0/3958744929 conn(0x7f44340cfe90 msgr2=0x7f44340d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 shutdown_connections 2020-11-16T22:34:59.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.621+0000 7f4433fff700 1 -- 172.21.15.181:0/3958744929 wait complete. 2020-11-16T22:34:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:34:59.622 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:34:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:34:59.630 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:34:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.765+0000 7fb759522700 1 Processor -- start 2020-11-16T22:34:59.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb759522700 1 -- start start 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb759522700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb75415d080 0x7fb7540cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb759522700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb754065150 con 0x7fb75415d460 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb759522700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7540653f0 con 0x7fb7540600f0 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb759522700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb754066a20 con 0x7fb75415d080 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb75415d080 0x7fb7540cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb75415d080 0x7fb7540cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35980/0 (socket says 172.21.15.181:35980) 2020-11-16T22:34:59.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 -- 172.21.15.181:0/1808735238 learned_addr learned my addr 172.21.15.181:0/1808735238 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 -- 172.21.15.181:0/1808735238 >> v1:172.21.15.181:6789/0 conn(0x7fb7540600f0 legacy=0x7fb754060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 -- 172.21.15.181:0/1808735238 >> v1:172.21.15.175:6789/0 conn(0x7fb75415d460 legacy=0x7fb754066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 -- 172.21.15.181:0/1808735238 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb754065690 con 0x7fb75415d080 2020-11-16T22:34:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb753fff700 1 --2- 172.21.15.181:0/1808735238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75415d080 0x7fb7540cdd90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fb74400ba20 tx=0x7fb744006eb0).ready entity=mon.1 client_cookie=6e4ccb4cae3ca2ff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.769+0000 7fb752ffd700 1 -- 172.21.15.181:0/1808735238 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744009890 con 0x7fb75415d080 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb752ffd700 1 -- 172.21.15.181:0/1808735238 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7440099f0 con 0x7fb75415d080 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb752ffd700 1 -- 172.21.15.181:0/1808735238 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744009d70 con 0x7fb75415d080 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- 172.21.15.181:0/1808735238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75415d080 msgr2=0x7fb7540cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 --2- 172.21.15.181:0/1808735238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75415d080 0x7fb7540cdd90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fb74400ba20 tx=0x7fb744006eb0).stop 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- 172.21.15.181:0/1808735238 shutdown_connections 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 --2- 172.21.15.181:0/1808735238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75415d080 0x7fb7540cdd90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- 172.21.15.181:0/1808735238 >> 172.21.15.181:0/1808735238 conn(0x7fb7540cfe90 msgr2=0x7fb7540d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- 172.21.15.181:0/1808735238 shutdown_connections 2020-11-16T22:34:59.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- 172.21.15.181:0/1808735238 wait complete. 2020-11-16T22:34:59.766 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:34:59.769+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:34:59.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 Processor -- start 2020-11-16T22:34:59.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.773+0000 7fb759522700 1 -- start start 2020-11-16T22:34:59.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 0x7fb7541712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540b7dc0 con 0x7fb75415d460 2020-11-16T22:34:59.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540b8060 con 0x7fb754154fc0 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb759522700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb754154340 con 0x7fb7540600f0 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb758d21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb758d21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35990/0 (socket says 172.21.15.181:35990) 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43828/0 (socket says 172.21.15.181:43828) 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb758d21700 1 -- 172.21.15.181:0/4263319515 learned_addr learned my addr 172.21.15.181:0/4263319515 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:34:59.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb7537fe700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 0x7fb7541712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 -- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 msgr2=0x7fb754155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.770 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 -- 172.21.15.181:0/4263319515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 msgr2=0x7fb7541712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 0x7fb7541712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 -- 172.21.15.181:0/4263319515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7541545e0 con 0x7fb7540600f0 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb758d21700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb7537fe700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 0x7fb7541712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:34:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.777+0000 7fb753fff700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fb74400ba20 tx=0x7fb744015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744011e00 con 0x7fb7540600f0 2020-11-16T22:34:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74401d450 con 0x7fb7540600f0 2020-11-16T22:34:59.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb754154880 con 0x7fb7540600f0 2020-11-16T22:34:59.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7540cfa60 con 0x7fb7540600f0 2020-11-16T22:34:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74401d810 con 0x7fb7540600f0 2020-11-16T22:34:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.781+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754044f60 con 0x7fb7540600f0 2020-11-16T22:34:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.785+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb74400f070 con 0x7fb7540600f0 2020-11-16T22:34:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.785+0000 7fb733fff700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 0x7fb72c05b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:34:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.785+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb74406fd80 con 0x7fb7540600f0 2020-11-16T22:34:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.785+0000 7fb7537fe700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 0x7fb72c05b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:34:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.785+0000 7fb7537fe700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 0x7fb72c05b930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fb7540cf750 tx=0x7fb748009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:34:59.781 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.789+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb744045e50 con 0x7fb7540600f0 2020-11-16T22:34:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.989+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb754158150 con 0x7fb7540600f0 2020-11-16T22:34:59.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.989+0000 7fb733fff700 1 -- 172.21.15.181:0/4263319515 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fb744049470 con 0x7fb7540600f0 2020-11-16T22:34:59.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:34:59.982 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:34:59.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 msgr2=0x7fb72c05b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 0x7fb72c05b930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fb7540cf750 tx=0x7fb748009040).stop 2020-11-16T22:34:59.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 msgr2=0x7fb754170db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:34:59.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fb74400ba20 tx=0x7fb744015000).stop 2020-11-16T22:34:59.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 shutdown_connections 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb72c059470 0x7fb72c05b930 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb754154fc0 0x7fb754155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7540600f0 0x7fb754170db0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 --2- 172.21.15.181:0/4263319515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb75415d460 0x7fb7541712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 >> 172.21.15.181:0/4263319515 conn(0x7fb7540cfe90 msgr2=0x7fb7540cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:34:59.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 shutdown_connections 2020-11-16T22:34:59.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:34:59.993+0000 7fb759522700 1 -- 172.21.15.181:0/4263319515 wait complete. 2020-11-16T22:34:59.987 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:34:59.999 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:35:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 Processor -- start 2020-11-16T22:35:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 -- start start 2020-11-16T22:35:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 0x7f20280600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20280605e0 0x7f20280660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280cfbf0 con 0x7f20280cf390 2020-11-16T22:35:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2028065190 con 0x7f20280605e0 2020-11-16T22:35:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2027fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2028065430 con 0x7f20280cf770 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20280605e0 0x7f20280660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20267fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 0x7f20280600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20280605e0 0x7f20280660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35998/0 (socket says 172.21.15.181:35998) 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 -- 172.21.15.181:0/2773066804 learned_addr learned my addr 172.21.15.181:0/2773066804 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 -- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 msgr2=0x7f20280600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 0x7f20280600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 -- 172.21.15.181:0/2773066804 >> v1:172.21.15.181:6789/0 conn(0x7f20280cf390 legacy=0x7f2028163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 -- 172.21.15.181:0/2773066804 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280656d0 con 0x7f20280605e0 2020-11-16T22:35:00.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2025ffb700 1 -- 172.21.15.181:0/2773066804 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884814462 0 0) 0x7f20280cfbf0 con 0x7f20280cf390 2020-11-16T22:35:00.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20267fc700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 0x7f20280600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f20277fe700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280605e0 0x7f20280660b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f2020009d80 tx=0x7f2020009290).ready entity=mon.1 client_cookie=fe61a1f87e1ed266 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2025ffb700 1 -- 172.21.15.181:0/2773066804 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bb80 con 0x7f20280605e0 2020-11-16T22:35:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2025ffb700 1 -- 172.21.15.181:0/2773066804 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000bce0 con 0x7f20280605e0 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.133+0000 7f2025ffb700 1 -- 172.21.15.181:0/2773066804 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020011610 con 0x7f20280605e0 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280605e0 msgr2=0x7f20280660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280605e0 0x7f20280660b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f2020009d80 tx=0x7f2020009290).stop 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- 172.21.15.181:0/2773066804 shutdown_connections 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280605e0 0x7f20280660b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- 172.21.15.181:0/2773066804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280cf770 0x7f20280600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- 172.21.15.181:0/2773066804 >> 172.21.15.181:0/2773066804 conn(0x7f20280cfe90 msgr2=0x7f20280d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- 172.21.15.181:0/2773066804 shutdown_connections 2020-11-16T22:35:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- 172.21.15.181:0/2773066804 wait complete. 2020-11-16T22:35:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 Processor -- start 2020-11-16T22:35:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- start start 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280605e0 0x7f20280d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20280cf390 0x7f2028171df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280ca470 con 0x7f20280cf390 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cb380 con 0x7f20280cf770 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.137+0000 7f2027fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20281707e0 con 0x7f20280605e0 2020-11-16T22:35:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36004/0 (socket says 172.21.15.181:36004) 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280605e0 0x7f20280d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 -- 172.21.15.181:0/1321848777 learned_addr learned my addr 172.21.15.181:0/1321848777 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20267fc700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20280cf390 0x7f2028171df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 -- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280605e0 msgr2=0x7f20280d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280605e0 0x7f20280d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 -- 172.21.15.181:0/1321848777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20280cf390 msgr2=0x7f2028171df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20280cf390 0x7f2028171df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 -- 172.21.15.181:0/1321848777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280ca660 con 0x7f20280cf770 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f20277fe700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f2028067b50 tx=0x7f2020016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f201effd700 1 -- 172.21.15.181:0/1321848777 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020020070 con 0x7f20280cf770 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f201effd700 1 -- 172.21.15.181:0/1321848777 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020011e30 con 0x7f20280cf770 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280ca900 con 0x7f20280cf770 2020-11-16T22:35:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f201effd700 1 -- 172.21.15.181:0/1321848777 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bb80 con 0x7f20280cf770 2020-11-16T22:35:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280cae40 con 0x7f20280cf770 2020-11-16T22:35:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.141+0000 7f201effd700 1 -- 172.21.15.181:0/1321848777 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2020017070 con 0x7f20280cf770 2020-11-16T22:35:00.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f201effd700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 0x7f2004057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f201effd700 1 -- 172.21.15.181:0/1321848777 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f202006f640 con 0x7f20280cf770 2020-11-16T22:35:00.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f2026ffd700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 0x7f2004057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f2026ffd700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 0x7f2004057520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f20100011a0 tx=0x7f20100079b0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f203c00e740 1 Processor -- start 2020-11-16T22:35:00.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.145+0000 7f203c00e740 1 -- start start 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x22b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x213e4b0 con 0x22b2fd0 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f2930 con 0x20947b0 2020-11-16T22:35:00.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x20b26f0 con 0x2026bb0 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x22b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x22b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36008/0 (socket says 172.21.15.181:36008) 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43850/0 (socket says 172.21.15.181:43850) 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f2026ffd700 1 -- 172.21.15.181:0/1375108248 learned_addr learned my addr 172.21.15.181:0/1375108248 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20277fe700 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 -- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 msgr2=0x22b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x22b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 -- 172.21.15.181:0/1375108248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 msgr2=0x22ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 -- 172.21.15.181:0/1375108248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20ad5a0 con 0x2026bb0 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f20267fc700 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f202813fea0 tx=0x7f2018006df0).ready entity=mon.2 client_cookie=e569c953f29debbd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f1ffffff700 1 -- 172.21.15.181:0/1375108248 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018011e00 con 0x2026bb0 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f1ffffff700 1 -- 172.21.15.181:0/1375108248 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018014440 con 0x2026bb0 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f1ffffff700 1 -- 172.21.15.181:0/1375108248 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180147c0 con 0x2026bb0 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 msgr2=0x22b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f202813fea0 tx=0x7f2018006df0).stop 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- 172.21.15.181:0/1375108248 shutdown_connections 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x22b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 --2- 172.21.15.181:0/1375108248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b3670 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- 172.21.15.181:0/1375108248 >> 172.21.15.181:0/1375108248 conn(0x2262e40 msgr2=0x22af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.149+0000 7f203c00e740 1 -- 172.21.15.181:0/1375108248 shutdown_connections 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- 172.21.15.181:0/1375108248 wait complete. 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- start start 2020-11-16T22:35:00.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 Processor -- start 2020-11-16T22:35:00.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x2307700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x21df6e0 con 0x22b2fd0 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x21df820 con 0x20947b0 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21e0a90 con 0x2026bb0 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 10 client.0 ll_register_callbacks cb 0x21d5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43856/0 (socket says 172.21.15.181:43856) 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 -- 172.21.15.181:0/535702089 learned_addr learned my addr 172.21.15.181:0/535702089 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f20267fc700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x2307700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f20277fe700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 -- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 msgr2=0x2307700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x2307700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 -- 172.21.15.181:0/535702089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 msgr2=0x22aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 -- 172.21.15.181:0/535702089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21d0280 con 0x2026bb0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f2026ffd700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2010017b80 tx=0x7f2010019040).ready entity=mon.2 client_cookie=22fcc13973f3c08 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f1ffd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201001c040 con 0x2026bb0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20100129c0 con 0x2026bb0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22936c0 con 0x2026bb0 2020-11-16T22:35:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2010012d90 con 0x2026bb0 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 10 client.7214 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.153+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x2005fd0 con 0x2026bb0 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22936c0 con 0x2026bb0 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f20100203e0 con 0x2026bb0 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 10 client.7214 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 10 client.7214 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 20 client.7214 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:00.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x22b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 4 client.7214 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x20d0c50 con 0x2302ab0 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f20267fc700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x22b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 10 client.7214 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f20267fc700 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x22b4a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=60d700be03891c80 server_cookie=7ecf3de83497a907 in_seq=0 out_seq=0 2020-11-16T22:35:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/535702089 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20d0c50 con 0x2302ab0 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 10 client.7214 handle_command_reply: tid=0 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 client.7214 shutdown 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 msgr2=0x23122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2010017b80 tx=0x7f2010019040).stop 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 shutdown_connections 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x22b4a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22b2fd0 0x22aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x23122a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 --2- 172.21.15.181:0/535702089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x20947b0 0x2307700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f1ffd7fa700 0 client.7214 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 >> 172.21.15.181:0/535702089 conn(0x2262e40 msgr2=0x22e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 shutdown_connections 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 1 -- 172.21.15.181:0/535702089 wait complete. 2020-11-16T22:35:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.157+0000 7f203c00e740 20 client.7214 trim_cache size 0 max 16384 2020-11-16T22:35:00.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 Processor -- start 2020-11-16T22:35:00.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 -- start start 2020-11-16T22:35:00.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x2310b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b7e40 con 0x20947b0 2020-11-16T22:35:00.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x23045d0 con 0x2262e40 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22aecb0 con 0x2026bb0 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.177+0000 7f20267fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36022/0 (socket says 172.21.15.181:36022) 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 -- 172.21.15.181:0/2939092977 learned_addr learned my addr 172.21.15.181:0/2939092977 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f20277fe700 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x2310b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 -- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 msgr2=0x22b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 -- 172.21.15.181:0/2939092977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 msgr2=0x2310b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x2310b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 -- 172.21.15.181:0/2939092977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22afb30 con 0x2262e40 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f20277fe700 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x2310b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f2026ffd700 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f2010012fd0 tx=0x7f2010020000).ready entity=mon.1 client_cookie=60f936a10effd941 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f201cff9700 1 -- 172.21.15.181:0/2939092977 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201001c040 con 0x2262e40 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f201cff9700 1 -- 172.21.15.181:0/2939092977 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010012630 con 0x2262e40 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f201cff9700 1 -- 172.21.15.181:0/2939092977 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100129c0 con 0x2262e40 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 msgr2=0x22b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f2010012fd0 tx=0x7f2010020000).stop 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- 172.21.15.181:0/2939092977 shutdown_connections 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x2310b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2262e40 0x22b1da0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 --2- 172.21.15.181:0/2939092977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2026bb0 0x22b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- 172.21.15.181:0/2939092977 >> 172.21.15.181:0/2939092977 conn(0x7f1ff4002320 msgr2=0x22e42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- 172.21.15.181:0/2939092977 shutdown_connections 2020-11-16T22:35:00.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- 172.21.15.181:0/2939092977 wait complete. 2020-11-16T22:35:00.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 -- start start 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.181+0000 7f203c00e740 1 Processor -- start 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x22e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 0x2301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2369160 con 0x20947b0 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2344710 con 0x2026bb0 2020-11-16T22:35:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x22e4500 con 0x22b2fd0 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 10 client.0 ll_register_callbacks cb 0x22b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 0x2301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36028/0 (socket says 172.21.15.181:36028) 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f2026ffd700 1 -- 172.21.15.181:0/1246460542 learned_addr learned my addr 172.21.15.181:0/1246460542 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20267fc700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x22e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 -- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 msgr2=0x2344220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 -- 172.21.15.181:0/1246460542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 msgr2=0x22e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x22e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 -- 172.21.15.181:0/1246460542 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff40081d0 con 0x22b2fd0 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f2026ffd700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f20277fe700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 0x2301ee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f2020049490 tx=0x7f202006f040).ready entity=mon.2 client_cookie=edbf50019f41aaf0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:35:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020048ec0 con 0x22b2fd0 2020-11-16T22:35:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2020045350 con 0x22b2fd0 2020-11-16T22:35:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20d7f50 con 0x22b2fd0 2020-11-16T22:35:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20200484c0 con 0x22b2fd0 2020-11-16T22:35:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 10 client.7220 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x2044720 con 0x22b2fd0 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20d7f50 con 0x22b2fd0 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2020048620 con 0x22b2fd0 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 10 client.7220 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 10 client.7220 resolve_mds: resolved ID 'b' to GID 7399 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 20 client.7220 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x23023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 4 client.7220 mds_command: new command op to 7399 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 --> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x20b51d0 con 0x2302ab0 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f2026ffd700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x23023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.185+0000 7f1ffd7fa700 10 client.7220 ms_handle_connect on v2:172.21.15.181:6836/1406664890 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2026ffd700 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x23023d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=ebb1f0832f8652c4 server_cookie=7e13c628101fcac7 in_seq=0 out_seq=0 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f1ffd7fa700 1 -- 172.21.15.181:0/1246460542 <== mds.1 v2:172.21.15.181:6836/1406664890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x20b51d0 con 0x2302ab0 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f1ffd7fa700 10 client.7220 handle_command_reply: tid=0 2020-11-16T22:35:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 client.7220 shutdown 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 msgr2=0x2301ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 0x2301ee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f2020049490 tx=0x7f202006f040).stop 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 shutdown_connections 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:6836/1406664890,v1:172.21.15.181:6837/1406664890] conn(0x2302ab0 0x23023d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x22b2fd0 0x2301ee0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2026bb0 0x2344220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 --2- 172.21.15.181:0/1246460542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x20947b0 0x22e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f1ffd7fa700 0 client.7220 ms_handle_reset on v2:172.21.15.181:6836/1406664890 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 >> 172.21.15.181:0/1246460542 conn(0x7f1ff4002320 msgr2=0x2311d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 shutdown_connections 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 1 -- 172.21.15.181:0/1246460542 wait complete. 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f203c00e740 20 client.7220 trim_cache size 0 max 16384 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 msgr2=0x7f2004057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 0x7f2004057520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f20100011a0 tx=0x7f20100079b0).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 msgr2=0x7f20281722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f2028067b50 tx=0x7f2020016000).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 shutdown_connections 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2004055060 0x7f2004057520 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20280cf770 0x7f20281722e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20280605e0 0x7f20280d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 --2- 172.21.15.181:0/1321848777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20280cf390 0x7f2028171df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 >> 172.21.15.181:0/1321848777 conn(0x7f20280cfe90 msgr2=0x7f2028140a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 shutdown_connections 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.189+0000 7f2027fff700 1 -- 172.21.15.181:0/1321848777 wait complete. 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:35:00.189 INFO:teuthology.orchestra.run.smithi181.stdout: "e8986808-291a-4767-bae7-a1c0041da596": { 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:00.190 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:35:00.197 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.325+0000 7fa2df161700 1 Processor -- start 2020-11-16T22:35:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- start start 2020-11-16T22:35:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa2d815d080 0x7fa2d80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d8065150 con 0x7fa2d815d460 2020-11-16T22:35:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80653f0 con 0x7fa2d80600f0 2020-11-16T22:35:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d8066a20 con 0x7fa2d815d080 2020-11-16T22:35:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de960700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa2d80600f0 0x7fa2d8060560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50342/0 (socket says 172.21.15.181:50342) 2020-11-16T22:35:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de960700 1 -- 172.21.15.181:0/3420329308 learned_addr learned my addr 172.21.15.181:0/3420329308 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de15f700 1 --2- 172.21.15.181:0/3420329308 >> v2:172.21.15.175:3300/0 conn(0x7fa2d815d080 0x7fa2d80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981452740 0 0) 0x7fa2d80653f0 con 0x7fa2d80600f0 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c0003240 con 0x7fa2d80600f0 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019705617 0 0) 0x7fa2d8065150 con 0x7fa2d815d460 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d80653f0 con 0x7fa2d815d460 2020-11-16T22:35:00.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3190358100 0 0) 0x7fa2c0003240 con 0x7fa2d80600f0 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de15f700 1 -- 172.21.15.181:0/3420329308 >> v1:172.21.15.181:6789/0 conn(0x7fa2d80600f0 legacy=0x7fa2d8060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de15f700 1 -- 172.21.15.181:0/3420329308 >> v1:172.21.15.175:6789/0 conn(0x7fa2d815d460 legacy=0x7fa2d8066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de15f700 1 -- 172.21.15.181:0/3420329308 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8065690 con 0x7fa2d815d080 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2de15f700 1 --2- 172.21.15.181:0/3420329308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d080 0x7fa2d80cdd90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fa2c800b950 tx=0x7fa2c8006df0).ready entity=mon.0 client_cookie=339b807afdb2ccca server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0001860 con 0x7fa2d815d080 2020-11-16T22:35:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8011400 con 0x7fa2d815d080 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2dd15d700 1 -- 172.21.15.181:0/3420329308 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8011780 con 0x7fa2d815d080 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- 172.21.15.181:0/3420329308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d080 msgr2=0x7fa2d80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 --2- 172.21.15.181:0/3420329308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d080 0x7fa2d80cdd90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fa2c800b950 tx=0x7fa2c8006df0).stop 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- 172.21.15.181:0/3420329308 shutdown_connections 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 --2- 172.21.15.181:0/3420329308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d080 0x7fa2d80cdd90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- 172.21.15.181:0/3420329308 >> 172.21.15.181:0/3420329308 conn(0x7fa2d80cfe90 msgr2=0x7fa2d80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- 172.21.15.181:0/3420329308 shutdown_connections 2020-11-16T22:35:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.329+0000 7fa2df161700 1 -- 172.21.15.181:0/3420329308 wait complete. 2020-11-16T22:35:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 Processor -- start 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- start start 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 0x7fa2d80b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d460 0x7fa2d8163b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8157a40 con 0x7fa2d815d460 2020-11-16T22:35:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d816f620 con 0x7fa2d815d080 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d816f8c0 con 0x7fa2d80600f0 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2de15f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 0x7fa2d80b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36042/0 (socket says 172.21.15.181:36042) 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2de15f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 0x7fa2d80b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43886/0 (socket says 172.21.15.181:43886) 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 -- 172.21.15.181:0/3162806404 learned_addr learned my addr 172.21.15.181:0/3162806404 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2de960700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d460 0x7fa2d8163b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 -- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 msgr2=0x7fa2d80b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 0x7fa2d80b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 -- 172.21.15.181:0/3162806404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d460 msgr2=0x7fa2d8163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d460 0x7fa2d8163b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 -- 172.21.15.181:0/3162806404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d816fb60 con 0x7fa2d815d080 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2dd95e700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa2d400c6c0 tx=0x7fa2d4007ae0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400e490 con 0x7fa2d815d080 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400e5f0 con 0x7fa2d815d080 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400e990 con 0x7fa2d815d080 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d816fe00 con 0x7fa2d815d080 2020-11-16T22:35:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.333+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8164020 con 0x7fa2d815d080 2020-11-16T22:35:00.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d8044f60 con 0x7fa2d815d080 2020-11-16T22:35:00.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa2d4019030 con 0x7fa2d815d080 2020-11-16T22:35:00.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2c67fc700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 0x7fa2b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:00.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa2d406d210 con 0x7fa2d815d080 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2de15f700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 0x7fa2b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.337+0000 7fa2de15f700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 0x7fa2b0057520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fa2c800b150 tx=0x7fa2c8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.345+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa2d4043470 con 0x7fa2d815d080 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.541+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d805c4a0 con 0x7fa2d815d080 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2c67fc700 1 -- 172.21.15.181:0/3162806404 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fa2d4046a90 con 0x7fa2d815d080 2020-11-16T22:35:00.538 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:00.539 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:35:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 msgr2=0x7fa2b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 0x7fa2b0057520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fa2c800b150 tx=0x7fa2c8015000).stop 2020-11-16T22:35:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 msgr2=0x7fa2d8163640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fa2d400c6c0 tx=0x7fa2d4007ae0).stop 2020-11-16T22:35:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 shutdown_connections 2020-11-16T22:35:00.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa2b0055060 0x7fa2b0057520 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d815d460 0x7fa2d8163b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2d80600f0 0x7fa2d80b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 --2- 172.21.15.181:0/3162806404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2d815d080 0x7fa2d8163640 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 >> 172.21.15.181:0/3162806404 conn(0x7fa2d80cfe90 msgr2=0x7fa2d80d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 shutdown_connections 2020-11-16T22:35:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:00.545+0000 7fa2df161700 1 -- 172.21.15.181:0/3162806404 wait complete. 2020-11-16T22:35:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:35:00.947 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:35:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 Processor -- start 2020-11-16T22:35:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 -- start start 2020-11-16T22:35:01.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f156c0cf6b0 0x7f156c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f156c15d2e0 0x7f156c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0c8910 con 0x7f156c15d2e0 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0b3460 con 0x7f156c0cf6b0 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.085+0000 7f1572335700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0b31d0 con 0x7f156c0b2900 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f156c0cf6b0 0x7f156c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1571333700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f156c0cf6b0 0x7f156c15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36052/0 (socket says 172.21.15.181:36052) 2020-11-16T22:35:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1571333700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43892/0 (socket says 172.21.15.181:43892) 2020-11-16T22:35:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 -- 172.21.15.181:0/3371536824 learned_addr learned my addr 172.21.15.181:0/3371536824 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1571b34700 1 --2- 172.21.15.181:0/3371536824 >> v2:172.21.15.175:3300/0 conn(0x7f156c15d2e0 0x7f156c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 -- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 msgr2=0x7f156c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 -- 172.21.15.181:0/3371536824 >> v2:172.21.15.175:3300/0 conn(0x7f156c15d2e0 msgr2=0x7f156c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 --2- 172.21.15.181:0/3371536824 >> v2:172.21.15.175:3300/0 conn(0x7f156c15d2e0 0x7f156c0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 -- 172.21.15.181:0/3371536824 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0c8130 con 0x7f156c0cf6b0 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1571333700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1570b32700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c0cf6b0 0x7f156c15cdf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f155c009d50 tx=0x7f155c009290).ready entity=mon.1 client_cookie=cf3de8e096139776 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f156bfff700 1 -- 172.21.15.181:0/3371536824 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00bad0 con 0x7f156c0cf6b0 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f156bfff700 1 -- 172.21.15.181:0/3371536824 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00bc30 con 0x7f156c0cf6b0 2020-11-16T22:35:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f156bfff700 1 -- 172.21.15.181:0/3371536824 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c011400 con 0x7f156c0cf6b0 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 -- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c0cf6b0 msgr2=0x7f156c15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c0cf6b0 0x7f156c15cdf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f155c009d50 tx=0x7f155c009290).stop 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 -- 172.21.15.181:0/3371536824 shutdown_connections 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 --2- 172.21.15.181:0/3371536824 >> v2:172.21.15.175:3300/0 conn(0x7f156c15d2e0 0x7f156c0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 --2- 172.21.15.181:0/3371536824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c0cf6b0 0x7f156c15cdf0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 -- 172.21.15.181:0/3371536824 >> 172.21.15.181:0/3371536824 conn(0x7f156c0d2dc0 msgr2=0x7f156c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 -- 172.21.15.181:0/3371536824 shutdown_connections 2020-11-16T22:35:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.089+0000 7f1572335700 1 -- 172.21.15.181:0/3371536824 wait complete. 2020-11-16T22:35:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 Processor -- start 2020-11-16T22:35:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- start start 2020-11-16T22:35:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c0cf6b0 0x7f156c063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0d1370 con 0x7f156c0cf6b0 2020-11-16T22:35:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0d1610 con 0x7f156c15d2e0 2020-11-16T22:35:01.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0d18b0 con 0x7f156c0b2900 2020-11-16T22:35:01.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36056/0 (socket says 172.21.15.181:36056) 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 -- 172.21.15.181:0/4132022828 learned_addr learned my addr 172.21.15.181:0/4132022828 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571333700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1570b32700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c0cf6b0 0x7f156c063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 -- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 msgr2=0x7f156c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 -- 172.21.15.181:0/4132022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c0cf6b0 msgr2=0x7f156c063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:01.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c0cf6b0 0x7f156c063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 -- 172.21.15.181:0/4132022828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0d1b50 con 0x7f156c15d2e0 2020-11-16T22:35:01.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1571b34700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f155400eba0 tx=0x7f1554006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:01.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f15697fa700 1 -- 172.21.15.181:0/4132022828 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400c820 con 0x7f156c15d2e0 2020-11-16T22:35:01.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f15697fa700 1 -- 172.21.15.181:0/4132022828 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155400c980 con 0x7f156c15d2e0 2020-11-16T22:35:01.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0d1df0 con 0x7f156c15d2e0 2020-11-16T22:35:01.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f15697fa700 1 -- 172.21.15.181:0/4132022828 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400cd40 con 0x7f156c15d2e0 2020-11-16T22:35:01.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.093+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c0643b0 con 0x7f156c15d2e0 2020-11-16T22:35:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.097+0000 7f15697fa700 1 -- 172.21.15.181:0/4132022828 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1554012600 con 0x7f156c15d2e0 2020-11-16T22:35:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.097+0000 7f15697fa700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 0x7f1548057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.097+0000 7f15697fa700 1 -- 172.21.15.181:0/4132022828 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f155406dfb0 con 0x7f156c15d2e0 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.097+0000 7f1571333700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 0x7f1548057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.097+0000 7f1571333700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 0x7f1548057520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f1564006fd0 tx=0x7f1564006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 Processor -- start 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- start start 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1b92720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1b4b0 con 0x19717b0 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1acf930 con 0x1903bb0 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x198f6f0 con 0x1b8ffd0 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43908/0 (socket says 172.21.15.181:43908) 2020-11-16T22:35:01.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36066/0 (socket says 172.21.15.181:36066) 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1571b34700 1 -- 172.21.15.181:0/3743126921 learned_addr learned my addr 172.21.15.181:0/3743126921 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1571333700 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1b92720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 -- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 msgr2=0x1b89700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 -- 172.21.15.181:0/3743126921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 msgr2=0x1b92720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1b92720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 -- 172.21.15.181:0/3743126921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x198a5a0 con 0x1903bb0 2020-11-16T22:35:01.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1571b34700 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:01.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1570b32700 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f155c009d50 tx=0x7f155c016000).ready entity=mon.1 client_cookie=cfbdd519b0ef0b73 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f15627fc700 1 -- 172.21.15.181:0/3743126921 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c009950 con 0x1903bb0 2020-11-16T22:35:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f15627fc700 1 -- 172.21.15.181:0/3743126921 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0094a0 con 0x1903bb0 2020-11-16T22:35:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f15627fc700 1 -- 172.21.15.181:0/3743126921 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c009610 con 0x1903bb0 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 msgr2=0x1b90670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f155c009d50 tx=0x7f155c016000).stop 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- 172.21.15.181:0/3743126921 shutdown_connections 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b8ffd0 0x1b89700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1b92720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 --2- 172.21.15.181:0/3743126921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1b90670 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- 172.21.15.181:0/3743126921 >> 172.21.15.181:0/3743126921 conn(0x1b3fe40 msgr2=0x1b8c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- 172.21.15.181:0/3743126921 shutdown_connections 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.101+0000 7f1582028740 1 -- 172.21.15.181:0/3743126921 wait complete. 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- start start 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 Processor -- start 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 0x1be2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b8ffd0 0x1be2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1abc6e0 con 0x1b8ffd0 2020-11-16T22:35:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1abc820 con 0x1903bb0 2020-11-16T22:35:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1abda90 con 0x19717b0 2020-11-16T22:35:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 10 client.0 ll_register_callbacks cb 0x1ab2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36070/0 (socket says 172.21.15.181:36070) 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 -- 172.21.15.181:0/149083753 learned_addr learned my addr 172.21.15.181:0/149083753 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1570b32700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 0x1be2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571b34700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b8ffd0 0x1be2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 -- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 msgr2=0x1be2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 0x1be2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 -- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b8ffd0 msgr2=0x1be2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b8ffd0 0x1be2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 -- 172.21.15.181:0/149083753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1aad280 con 0x1903bb0 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1570b32700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 0x1be2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1571333700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f1564014d70 tx=0x7f1564013000).ready entity=mon.1 client_cookie=586691ab0a31fdb4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:35:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564012a30 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1b706c0 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1564012b90 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1564021450 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 10 client.7924 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x18e2fd0 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1b706c0 con 0x1903bb0 2020-11-16T22:35:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f15640215b0 con 0x1903bb0 2020-11-16T22:35:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 10 client.7924 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 10 client.7924 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:35:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 20 client.7924 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1bdd1f0 0x1b8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 4 client.7924 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:35:02.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.105+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x19adc50 con 0x1bdd1f0 2020-11-16T22:35:02.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1570b32700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1bdd1f0 0x1b8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f153b7fe700 10 client.7924 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:35:02.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1570b32700 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1bdd1f0 0x1b8a550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=6aa44280e184f1ec server_cookie=1b72b808a3562b69 in_seq=0 out_seq=0 2020-11-16T22:35:02.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f153b7fe700 1 -- 172.21.15.181:0/149083753 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x19adc50 con 0x1bdd1f0 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f153b7fe700 10 client.7924 handle_command_reply: tid=0 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 client.7924 shutdown 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 msgr2=0x1bb2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f1564014d70 tx=0x7f1564013000).stop 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 shutdown_connections 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1bdd1f0 0x1b8a550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b8ffd0 0x1be2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1903bb0 0x1bb2720 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f153b7fe700 0 client.7924 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:35:02.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 --2- 172.21.15.181:0/149083753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x19717b0 0x1be2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 >> 172.21.15.181:0/149083753 conn(0x1b3fe40 msgr2=0x1b90fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:02.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 shutdown_connections 2020-11-16T22:35:02.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 1 -- 172.21.15.181:0/149083753 wait complete. 2020-11-16T22:35:02.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.109+0000 7f1582028740 20 client.7924 trim_cache size 0 max 16384 2020-11-16T22:35:02.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.129+0000 7f1582028740 1 Processor -- start 2020-11-16T22:35:02.326 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.129+0000 7f1582028740 1 -- start start 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1baedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a94e40 con 0x19717b0 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b8ef70 con 0x1b3fe40 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1bd9920 con 0x1903bb0 2020-11-16T22:35:02.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43922/0 (socket says 172.21.15.181:43922) 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 -- 172.21.15.181:0/948292588 learned_addr learned my addr 172.21.15.181:0/948292588 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1571333700 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1571b34700 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1baedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 -- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 msgr2=0x1bc3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 -- 172.21.15.181:0/948292588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 msgr2=0x1baedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1baedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 -- 172.21.15.181:0/948292588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400081c0 con 0x1903bb0 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1570b32700 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f156c045680 tx=0x7f155c030280).ready entity=mon.2 client_cookie=be371b542eecf43b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1562ffd700 1 -- 172.21.15.181:0/948292588 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c009610 con 0x1903bb0 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1562ffd700 1 -- 172.21.15.181:0/948292588 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00b8c0 con 0x1903bb0 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1562ffd700 1 -- 172.21.15.181:0/948292588 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c011c50 con 0x1903bb0 2020-11-16T22:35:02.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 msgr2=0x1bdc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f156c045680 tx=0x7f155c030280).stop 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- 172.21.15.181:0/948292588 shutdown_connections 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1baedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 --2- 172.21.15.181:0/948292588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1bdc810 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- 172.21.15.181:0/948292588 >> 172.21.15.181:0/948292588 conn(0x7f1540002320 msgr2=0x1bc2680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:02.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- 172.21.15.181:0/948292588 shutdown_connections 2020-11-16T22:35:02.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- 172.21.15.181:0/948292588 wait complete. 2020-11-16T22:35:02.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 -- start start 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.133+0000 7f1582028740 1 Processor -- start 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1befd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1bda6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c2aed0 con 0x19717b0 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bdabc0 con 0x1b3fe40 2020-11-16T22:35:02.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c2a450 con 0x1903bb0 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 10 client.0 ll_register_callbacks cb 0x1b907e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1befd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36084/0 (socket says 172.21.15.181:36084) 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571b34700 1 -- 172.21.15.181:0/2484256485 learned_addr learned my addr 172.21.15.181:0/2484256485 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1570b32700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1bda6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 -- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 msgr2=0x1bc0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:02.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 -- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 msgr2=0x1bda6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:02.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1bda6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 -- 172.21.15.181:0/2484256485 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1921720 con 0x1903bb0 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571b34700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1571333700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1befd20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1564014d70 tx=0x7f1564021000).ready entity=mon.2 client_cookie=272a157f5b2adf2e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564012a30 con 0x1903bb0 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f155c0094a0 con 0x1903bb0 2020-11-16T22:35:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1564012b90 con 0x1903bb0 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15640148d0 con 0x1903bb0 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 10 client.7241 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x192ce00 con 0x1903bb0 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0094a0 con 0x1903bb0 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f156401fa80 con 0x1903bb0 2020-11-16T22:35:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 10 client.7241 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 10 client.7241 resolve_mds: resolved ID 'i' to GID 7396 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 20 client.7241 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1b8ffd0 0x1c29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 4 client.7241 mds_command: new command op to 7396 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 --> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x19b4f50 con 0x1b8ffd0 2020-11-16T22:35:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1570b32700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1b8ffd0 0x1c29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 10 client.7241 ms_handle_connect on v2:172.21.15.175:6837/3389989681 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1570b32700 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1b8ffd0 0x1c29ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=840ff7cfd86f237c server_cookie=b6c4f2839d0a6ee in_seq=0 out_seq=0 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 1 -- 172.21.15.181:0/2484256485 <== mds.2 v2:172.21.15.175:6837/3389989681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x19b4f50 con 0x1b8ffd0 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f153b7fe700 10 client.7241 handle_command_reply: tid=0 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 client.7241 shutdown 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 msgr2=0x1befd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.137+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1befd20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1564014d70 tx=0x7f1564021000).stop 2020-11-16T22:35:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 shutdown_connections 2020-11-16T22:35:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:6837/3389989681,v1:172.21.15.175:6839/3389989681] conn(0x1b8ffd0 0x1c29ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b3fe40 0x1bc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f153b7fe700 0 client.7241 ms_handle_reset on v2:172.21.15.175:6837/3389989681 2020-11-16T22:35:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1903bb0 0x1befd20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 --2- 172.21.15.181:0/2484256485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x19717b0 0x1bda6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 >> 172.21.15.181:0/2484256485 conn(0x7f1540002320 msgr2=0x1b848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 shutdown_connections 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 1 -- 172.21.15.181:0/2484256485 wait complete. 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1582028740 20 client.7241 trim_cache size 0 max 16384 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 msgr2=0x7f1548057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:02.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 0x7f1548057520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f1564006fd0 tx=0x7f1564006df0).stop 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 msgr2=0x7f156c0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f155400eba0 tx=0x7f1554006df0).stop 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 shutdown_connections 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1548055060 0x7f1548057520 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c15d2e0 0x7f156c0b4360 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:02.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f156c0b2900 0x7f156c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 --2- 172.21.15.181:0/4132022828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c0cf6b0 0x7f156c063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 >> 172.21.15.181:0/4132022828 conn(0x7f156c0d2dc0 msgr2=0x7f156c15cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:03.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 shutdown_connections 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:01.141+0000 7f1572335700 1 -- 172.21.15.181:0/4132022828 wait complete. 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stdout: "e8986808-291a-4767-bae7-a1c0041da596": { 2020-11-16T22:35:03.151 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:35:03.152 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "e8986808-291a-4767-bae7-a1c0041da596", 2020-11-16T22:35:03.152 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:35:03.152 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:03.152 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:03.152 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:35:03.153 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:03.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.293+0000 7fc1a4a35700 1 Processor -- start 2020-11-16T22:35:03.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.293+0000 7fc1a4a35700 1 -- start start 2020-11-16T22:35:03.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1a015d510 0x7fc1a015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc1a00b7c10 0x7fc1a00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a00ceb10 con 0x7fc1a00ce020 2020-11-16T22:35:03.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00cedb0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00cf050 con 0x7fc1a015d510 2020-11-16T22:35:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1a015d510 0x7fc1a015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1a015d510 0x7fc1a015d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36092/0 (socket says 172.21.15.181:36092) 2020-11-16T22:35:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 -- 172.21.15.181:0/1901129801 learned_addr learned my addr 172.21.15.181:0/1901129801 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19ed9d700 1 --2- 172.21.15.181:0/1901129801 >> v2:172.21.15.175:3300/0 conn(0x7fc1a00b7c10 0x7fc1a00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 -- 172.21.15.181:0/1901129801 >> v2:172.21.15.175:3300/0 conn(0x7fc1a00b7c10 msgr2=0x7fc1a00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:03.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 --2- 172.21.15.181:0/1901129801 >> v2:172.21.15.175:3300/0 conn(0x7fc1a00b7c10 0x7fc1a00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 -- 172.21.15.181:0/1901129801 >> v1:172.21.15.175:6789/0 conn(0x7fc1a00ce020 legacy=0x7fc1a00b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 -- 172.21.15.181:0/1901129801 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00cf2f0 con 0x7fc1a015d510 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19e59c700 1 --2- 172.21.15.181:0/1901129801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a015d8f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fc1a00681b0 tx=0x7fc190006eb0).ready entity=mon.1 client_cookie=dbe46938f1e20785 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19d59a700 1 -- 172.21.15.181:0/1901129801 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190009810 con 0x7fc1a015d510 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19d59a700 1 -- 172.21.15.181:0/1901129801 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc190009970 con 0x7fc1a015d510 2020-11-16T22:35:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc19d59a700 1 -- 172.21.15.181:0/1901129801 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190009cf0 con 0x7fc1a015d510 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1901129801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 msgr2=0x7fc1a015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1901129801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a015d8f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fc1a00681b0 tx=0x7fc190006eb0).stop 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1901129801 shutdown_connections 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1901129801 >> v2:172.21.15.175:3300/0 conn(0x7fc1a00b7c10 0x7fc1a00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1901129801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a015d8f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1901129801 >> 172.21.15.181:0/1901129801 conn(0x7fc1a00d3680 msgr2=0x7fc1a00d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1901129801 shutdown_connections 2020-11-16T22:35:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.297+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1901129801 wait complete. 2020-11-16T22:35:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 Processor -- start 2020-11-16T22:35:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 -- start start 2020-11-16T22:35:03.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a00ce020 0x7fc1a015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0171cc0 con 0x7fc1a00ce020 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01714e0 con 0x7fc1a015d510 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc1a4a35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0171780 con 0x7fc1a00b7c10 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43940/0 (socket says 172.21.15.181:43940) 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36100/0 (socket says 172.21.15.181:36100) 2020-11-16T22:35:03.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 -- 172.21.15.181:0/1812720164 learned_addr learned my addr 172.21.15.181:0/1812720164 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:03.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19dd9b700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a00ce020 0x7fc1a015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 -- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 msgr2=0x7fc1a016e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 -- 172.21.15.181:0/1812720164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a00ce020 msgr2=0x7fc1a015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a00ce020 0x7fc1a015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19e59c700 1 -- 172.21.15.181:0/1812720164 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0171a20 con 0x7fc1a00b7c10 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.301+0000 7fc19ed9d700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:03.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc19e59c700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fc19000b1b0 tx=0x7fc190015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190011d50 con 0x7fc1a00b7c10 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19001e460 con 0x7fc1a00b7c10 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a016da50 con 0x7fc1a00b7c10 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19001e7e0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a016e680 con 0x7fc1a00b7c10 2020-11-16T22:35:03.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a016def0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.305+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc190025030 con 0x7fc1a00b7c10 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.309+0000 7fc18e7fc700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 0x7fc174057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.309+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc19006f760 con 0x7fc1a00b7c10 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.309+0000 7fc19dd9b700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 0x7fc174057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.309+0000 7fc19dd9b700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 0x7fc174057550 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fc1a00d2d60 tx=0x7fc194007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:03.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.309+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc1900459e0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1a016def0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc18e7fc700 1 -- 172.21.15.181:0/1812720164 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fc1a016def0 con 0x7fc1a00b7c10 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 msgr2=0x7fc174057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 0x7fc174057550 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fc1a00d2d60 tx=0x7fc194007b90).stop 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 msgr2=0x7fc1a0170e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fc19000b1b0 tx=0x7fc190015000).stop 2020-11-16T22:35:03.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 shutdown_connections 2020-11-16T22:35:03.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc174055090 0x7fc174057550 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1a015d510 0x7fc1a016e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1a00b7c10 0x7fc1a0170e20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 --2- 172.21.15.181:0/1812720164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a00ce020 0x7fc1a015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:03.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 >> 172.21.15.181:0/1812720164 conn(0x7fc1a00d3680 msgr2=0x7fc1a015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:03.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 shutdown_connections 2020-11-16T22:35:03.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:03.505+0000 7fc1a4a35700 1 -- 172.21.15.181:0/1812720164 wait complete. 2020-11-16T22:35:03.911 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:35:03.911 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:03.911 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:35:03.912 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:03.913 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:35:03.919 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:03.921 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:04.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.057+0000 7f1201763700 1 Processor -- start 2020-11-16T22:35:04.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.057+0000 7f1201763700 1 -- start start 2020-11-16T22:35:04.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 0x7f11fc062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc065930 con 0x7f11fc0615f0 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc1621b0 con 0x7f11fc0637b0 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc064970 con 0x7f11fc0633d0 2020-11-16T22:35:04.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11f3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11f3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36106/0 (socket says 172.21.15.181:36106) 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43946/0 (socket says 172.21.15.181:43946) 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11f3fff700 1 -- 172.21.15.181:0/847988774 learned_addr learned my addr 172.21.15.181:0/847988774 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 0x7f11fc062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 -- 172.21.15.181:0/847988774 >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 msgr2=0x7f11fc061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 -- 172.21.15.181:0/847988774 >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 msgr2=0x7f11fc062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 0x7f11fc062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 -- 172.21.15.181:0/847988774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc064c10 con 0x7f11fc0633d0 2020-11-16T22:35:04.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 0x7f11fc062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:04.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11f3fff700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11faffd700 1 --2- 172.21.15.181:0/847988774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f11fc0676d0 tx=0x7f11ec006df0).ready entity=mon.2 client_cookie=7fdc80f64bbd3627 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11fa7fc700 1 -- 172.21.15.181:0/847988774 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0097b0 con 0x7f11fc0633d0 2020-11-16T22:35:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11fa7fc700 1 -- 172.21.15.181:0/847988774 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec009910 con 0x7f11fc0633d0 2020-11-16T22:35:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f11fa7fc700 1 -- 172.21.15.181:0/847988774 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec009c90 con 0x7f11fc0633d0 2020-11-16T22:35:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- 172.21.15.181:0/847988774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 msgr2=0x7f11fc161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- 172.21.15.181:0/847988774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f11fc0676d0 tx=0x7f11ec006df0).stop 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- 172.21.15.181:0/847988774 shutdown_connections 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.175:3300/0 conn(0x7f11fc0615f0 0x7f11fc062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- 172.21.15.181:0/847988774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0633d0 0x7f11fc161cc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 --2- 172.21.15.181:0/847988774 >> v2:172.21.15.181:3300/0 conn(0x7f11fc0637b0 0x7f11fc061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- 172.21.15.181:0/847988774 >> 172.21.15.181:0/847988774 conn(0x7f11fc0cf890 msgr2=0x7f11fc0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- 172.21.15.181:0/847988774 shutdown_connections 2020-11-16T22:35:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.061+0000 7f1201763700 1 -- 172.21.15.181:0/847988774 wait complete. 2020-11-16T22:35:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 Processor -- start 2020-11-16T22:35:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- start start 2020-11-16T22:35:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 0x7f11fc0b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11fc0633d0 0x7f11fc0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0cacd0 con 0x7f11fc0633d0 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0caf70 con 0x7f11fc0615f0 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0cb210 con 0x7f11fc0637b0 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 0x7f11fc0b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43956/0 (socket says 172.21.15.181:43956) 2020-11-16T22:35:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11fb7fe700 1 -- 172.21.15.181:0/2234319081 learned_addr learned my addr 172.21.15.181:0/2234319081 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11f3fff700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11fc0633d0 0x7f11fc0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 -- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 msgr2=0x7f11fc16e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 -- 172.21.15.181:0/2234319081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11fc0633d0 msgr2=0x7f11fc0ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11fc0633d0 0x7f11fc0ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 -- 172.21.15.181:0/2234319081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0cb4b0 con 0x7f11fc0615f0 2020-11-16T22:35:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11faffd700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 0x7f11fc0b86a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f11ec012000 tx=0x7f11ec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11f2ffd700 1 -- 172.21.15.181:0/2234319081 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec011c50 con 0x7f11fc0615f0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11f2ffd700 1 -- 172.21.15.181:0/2234319081 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec011db0 con 0x7f11fc0615f0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0cb750 con 0x7f11fc0615f0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11f2ffd700 1 -- 172.21.15.181:0/2234319081 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec01e690 con 0x7f11fc0615f0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0cbe90 con 0x7f11fc0615f0 2020-11-16T22:35:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.065+0000 7f11f2ffd700 1 -- 172.21.15.181:0/2234319081 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f11ec025030 con 0x7f11fc0615f0 2020-11-16T22:35:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.069+0000 7f11f2ffd700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.069+0000 7f11f2ffd700 1 -- 172.21.15.181:0/2234319081 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11ec06f150 con 0x7f11fc0615f0 2020-11-16T22:35:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.069+0000 7f11f3fff700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.069+0000 7f11f3fff700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 0x7f11d8057520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f11fc0b8b90 tx=0x7f11e4006eb0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:05.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 Processor -- start 2020-11-16T22:35:05.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- start start 2020-11-16T22:35:05.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 0x1f44c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d7b4b0 con 0x1cd17b0 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e2f930 con 0x1ef07b0 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cef6f0 con 0x1c63bb0 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43962/0 (socket says 172.21.15.181:43962) 2020-11-16T22:35:05.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11f3fff700 1 -- 172.21.15.181:0/2201092171 learned_addr learned my addr 172.21.15.181:0/2201092171 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:05.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36120/0 (socket says 172.21.15.181:36120) 2020-11-16T22:35:05.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11faffd700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 0x1f44c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 -- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 msgr2=0x1f45c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 -- 172.21.15.181:0/2201092171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 msgr2=0x1f44c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 0x1f44c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 -- 172.21.15.181:0/2201092171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cea5a0 con 0x1ef07b0 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11f3fff700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11faffd700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 0x1f44c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f11f40114c0 tx=0x7f11f4008040).ready entity=mon.1 client_cookie=c09a85e9bf34880c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11d3fff700 1 -- 172.21.15.181:0/2201092171 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4011e80 con 0x1ef07b0 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11d3fff700 1 -- 172.21.15.181:0/2201092171 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4015690 con 0x1ef07b0 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f11d3fff700 1 -- 172.21.15.181:0/2201092171 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4015a50 con 0x1ef07b0 2020-11-16T22:35:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 msgr2=0x1f46160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f11f40114c0 tx=0x7f11f4008040).stop 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- 172.21.15.181:0/2201092171 shutdown_connections 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ef07b0 0x1f46160 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1cd17b0 0x1f44c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 --2- 172.21.15.181:0/2201092171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f45c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- 172.21.15.181:0/2201092171 >> 172.21.15.181:0/2201092171 conn(0x1e9fe40 msgr2=0x1f44580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- 172.21.15.181:0/2201092171 shutdown_connections 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.073+0000 7f12101f4740 1 -- 172.21.15.181:0/2201092171 wait complete. 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- start start 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 Processor -- start 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 0x1f40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e1c6e0 con 0x1ef07b0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e1c820 con 0x1cd17b0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e1da90 con 0x1c63bb0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 10 client.0 ll_register_callbacks cb 0x1e12ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f29c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43966/0 (socket says 172.21.15.181:43966) 2020-11-16T22:35:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11faffd700 1 -- 172.21.15.181:0/1043590468 learned_addr learned my addr 172.21.15.181:0/1043590468 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 0x1f40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 -- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 msgr2=0x1f29c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f29c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 -- 172.21.15.181:0/1043590468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 msgr2=0x1f40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 0x1f40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 -- 172.21.15.181:0/1043590468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e0d280 con 0x1cd17b0 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 0x1f40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11f3fff700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f40050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x1f2a770 tx=0x7f11e4014000).ready entity=mon.1 client_cookie=8ce3b4a4bf44c11c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4012b00 con 0x1cd17b0 2020-11-16T22:35:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11e4012c60 con 0x1cd17b0 2020-11-16T22:35:05.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ee4e10 con 0x1cd17b0 2020-11-16T22:35:05.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e4012e20 con 0x1cd17b0 2020-11-16T22:35:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 10 client.7948 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x1c42fd0 con 0x1cd17b0 2020-11-16T22:35:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ee4e10 con 0x1cd17b0 2020-11-16T22:35:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f11e401f600 con 0x1cd17b0 2020-11-16T22:35:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 10 client.7948 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 10 client.7948 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 20 client.7948 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f43150 0x1f43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 4 client.7948 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1d0dc50 con 0x1f43150 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11faffd700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f43150 0x1f43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11faffd700 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f43150 0x1f43530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4c5c9db4f77a8a3f server_cookie=7b5c6e0c7509fbe5 in_seq=0 out_seq=0 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 10 client.7948 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 1 -- 172.21.15.181:0/1043590468 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d0dc50 con 0x1f43150 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f11d17fa700 10 client.7948 handle_command_reply: tid=0 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.077+0000 7f12101f4740 1 client.7948 shutdown 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 msgr2=0x1f40050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f40050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x1f2a770 tx=0x7f11e4014000).stop 2020-11-16T22:35:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 shutdown_connections 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1f43150 0x1f43530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ef07b0 0x1f40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f11d17fa700 0 client.7948 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 --2- 172.21.15.181:0/1043590468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f40050 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 >> 172.21.15.181:0/1043590468 conn(0x1e9fe40 msgr2=0x1ee9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 shutdown_connections 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 1 -- 172.21.15.181:0/1043590468 wait complete. 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.081+0000 7f12101f4740 20 client.7948 trim_cache size 0 max 16384 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 Processor -- start 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 -- start start 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c63bb0 0x1f4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cd17b0 0x1ef9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df4e40 con 0x1c63bb0 2020-11-16T22:35:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ee80c0 con 0x1e9fe40 2020-11-16T22:35:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eea3e0 con 0x1cd17b0 2020-11-16T22:35:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cd17b0 0x1ef9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36132/0 (socket says 172.21.15.181:36132) 2020-11-16T22:35:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 -- 172.21.15.181:0/551371351 learned_addr learned my addr 172.21.15.181:0/551371351 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11f3fff700 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c63bb0 0x1f4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 -- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cd17b0 msgr2=0x1ef9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cd17b0 0x1ef9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 -- 172.21.15.181:0/551371351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c63bb0 msgr2=0x1f4cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c63bb0 0x1f4cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.101+0000 7f11faffd700 1 -- 172.21.15.181:0/551371351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80081d0 con 0x1e9fe40 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f11ec048d70 tx=0x7f11ec011000).ready entity=mon.1 client_cookie=cdab664e28e8e000 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11f0ff9700 1 -- 172.21.15.181:0/551371351 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec006920 con 0x1e9fe40 2020-11-16T22:35:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11f0ff9700 1 -- 172.21.15.181:0/551371351 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec0471b0 con 0x1e9fe40 2020-11-16T22:35:06.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11f0ff9700 1 -- 172.21.15.181:0/551371351 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec047410 con 0x1e9fe40 2020-11-16T22:35:06.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 msgr2=0x1f45530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f11ec048d70 tx=0x7f11ec011000).stop 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- 172.21.15.181:0/551371351 shutdown_connections 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1cd17b0 0x1ef9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e9fe40 0x1f45530 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- 172.21.15.181:0/551371351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c63bb0 0x1f4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- 172.21.15.181:0/551371351 >> 172.21.15.181:0/551371351 conn(0x7f11c8002320 msgr2=0x1f0fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- 172.21.15.181:0/551371351 shutdown_connections 2020-11-16T22:35:06.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- 172.21.15.181:0/551371351 wait complete. 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- start start 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 Processor -- start 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f9f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e9fe40 0x1f9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f291e0 con 0x1e9fe40 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f29c10 con 0x1cd17b0 2020-11-16T22:35:06.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f40790 con 0x1c63bb0 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f12101f4740 10 client.0 ll_register_callbacks cb 0x1f290a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f9f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43978/0 (socket says 172.21.15.181:43978) 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 -- 172.21.15.181:0/3314510327 learned_addr learned my addr 172.21.15.181:0/3314510327 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:06.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11fb7fe700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e9fe40 0x1f9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 -- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 msgr2=0x1f9f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f9f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 -- 172.21.15.181:0/3314510327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e9fe40 msgr2=0x1f9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e9fe40 0x1f9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 -- 172.21.15.181:0/3314510327 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c81720 con 0x1c63bb0 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11faffd700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f11ec048d70 tx=0x7f11ec035000).ready entity=mon.2 client_cookie=a0520cc05442ae6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:06.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:35:06.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.105+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec045070 con 0x1c63bb0 2020-11-16T22:35:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11ec040c50 con 0x1c63bb0 2020-11-16T22:35:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11f4015690 con 0x1c63bb0 2020-11-16T22:35:06.322 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec0451d0 con 0x1c63bb0 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 10 client.7262 fetch_fsmap learned FSMap version 107 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x1c8ce00 con 0x1c63bb0 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4015690 con 0x1c63bb0 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 107) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f11ec047fd0 con 0x1c63bb0 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 10 client.7262 fetch_fsmap finished waiting for FSMap version 107 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 10 client.7262 resolve_mds: resolved ID 'd' to GID 7405 2020-11-16T22:35:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 20 client.7262 populate_metadata read hostname 'smithi181' 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1ef07b0 0x1fa5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 4 client.7262 mds_command: new command op to 7405 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 --> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1d14f50 con 0x1ef07b0 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11f3fff700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1ef07b0 0x1fa5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 10 client.7262 ms_handle_connect on v2:172.21.15.181:6838/1296416553 2020-11-16T22:35:06.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11f3fff700 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1ef07b0 0x1fa5a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=74858e3e066697b0 server_cookie=553ab5faae196994 in_seq=0 out_seq=0 2020-11-16T22:35:06.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 1 -- 172.21.15.181:0/3314510327 <== mds.0 v2:172.21.15.181:6838/1296416553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1d14f50 con 0x1ef07b0 2020-11-16T22:35:06.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 10 client.7262 handle_command_reply: tid=0 2020-11-16T22:35:06.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 client.7262 shutdown 2020-11-16T22:35:06.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 msgr2=0x1f3ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f11ec048d70 tx=0x7f11ec035000).stop 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 shutdown_connections 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:6838/1296416553,v1:172.21.15.181:6839/1296416553] conn(0x1ef07b0 0x1fa5a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e9fe40 0x1f9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c63bb0 0x1f3ec00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 --2- 172.21.15.181:0/3314510327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1cd17b0 0x1f9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 >> 172.21.15.181:0/3314510327 conn(0x7f11c8002320 msgr2=0x1ef9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f11d17fa700 0 client.7262 ms_handle_reset on v2:172.21.15.181:6838/1296416553 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 shutdown_connections 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 1 -- 172.21.15.181:0/3314510327 wait complete. 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f12101f4740 20 client.7262 trim_cache size 0 max 16384 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 msgr2=0x7f11d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 0x7f11d8057520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f11fc0b8b90 tx=0x7f11e4006eb0).stop 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 msgr2=0x7f11fc0b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 0x7f11fc0b86a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f11ec012000 tx=0x7f11ec015000).stop 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 shutdown_connections 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc0637b0 0x7f11fc16e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0615f0 0x7f11fc0b86a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 --2- 172.21.15.181:0/2234319081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11fc0633d0 0x7f11fc0ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 >> 172.21.15.181:0/2234319081 conn(0x7f11fc0cf890 msgr2=0x7f11fc0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 shutdown_connections 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:04.109+0000 7f1201763700 1 -- 172.21.15.181:0/2234319081 wait complete. 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:35:06.530 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:35:06.531 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:35:06.531 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:35:06.531 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:06.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 Processor -- start 2020-11-16T22:35:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- start start 2020-11-16T22:35:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9cf8161860 0x7f9cf8062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80c8db0 con 0x7f9cf8164e10 2020-11-16T22:35:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80c8330 con 0x7f9cf8161860 2020-11-16T22:35:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80c85d0 con 0x7f9cf8161480 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:43986/0 (socket says 172.21.15.181:43986) 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 -- 172.21.15.181:0/526870557 learned_addr learned my addr 172.21.15.181:0/526870557 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfe5b3700 1 --2- 172.21.15.181:0/526870557 >> v2:172.21.15.175:3300/0 conn(0x7f9cf8161860 0x7f9cf8062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 -- 172.21.15.181:0/526870557 >> v2:172.21.15.175:3300/0 conn(0x7f9cf8161860 msgr2=0x7f9cf8062960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 --2- 172.21.15.181:0/526870557 >> v2:172.21.15.175:3300/0 conn(0x7f9cf8161860 0x7f9cf8062960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 -- 172.21.15.181:0/526870557 >> v1:172.21.15.181:6789/0 conn(0x7f9cf8164e10 legacy=0x7f9cf8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 -- 172.21.15.181:0/526870557 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80c8870 con 0x7f9cf8161480 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfddb2700 1 --2- 172.21.15.181:0/526870557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f9ce8009d50 tx=0x7f9ce8009290).ready entity=mon.2 client_cookie=78a7ea4acd6fdeb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfcdb0700 1 -- 172.21.15.181:0/526870557 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800bc80 con 0x7f9cf8161480 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfcdb0700 1 -- 172.21.15.181:0/526870557 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800bde0 con 0x7f9cf8161480 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfcdb0700 1 -- 172.21.15.181:0/526870557 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8011610 con 0x7f9cf8161480 2020-11-16T22:35:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- 172.21.15.181:0/526870557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 msgr2=0x7f9cf8164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/526870557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f9ce8009d50 tx=0x7f9ce8009290).stop 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- 172.21.15.181:0/526870557 shutdown_connections 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/526870557 >> v2:172.21.15.175:3300/0 conn(0x7f9cf8161860 0x7f9cf8062960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/526870557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161480 0x7f9cf8164920 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- 172.21.15.181:0/526870557 >> 172.21.15.181:0/526870557 conn(0x7f9cf80d2dc0 msgr2=0x7f9cf80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- 172.21.15.181:0/526870557 shutdown_connections 2020-11-16T22:35:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.673+0000 7f9cfedb4700 1 -- 172.21.15.181:0/526870557 wait complete. 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.677+0000 7f9cfedb4700 1 Processor -- start 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.677+0000 7f9cfedb4700 1 -- start start 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 0x7f9cf80b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 0x7f9cf80b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b6fa0 con 0x7f9cf80b8160 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b7240 con 0x7f9cf8161480 2020-11-16T22:35:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b74e0 con 0x7f9cf8161860 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfddb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 0x7f9cf80b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfddb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36150/0 (socket says 172.21.15.181:36150) 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfddb2700 1 -- 172.21.15.181:0/3003442954 learned_addr learned my addr 172.21.15.181:0/3003442954 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfe5b3700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 0x7f9cf80b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 -- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 msgr2=0x7f9cf8173bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 -- 172.21.15.181:0/3003442954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 msgr2=0x7f9cf80b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 0x7f9cf80b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 -- 172.21.15.181:0/3003442954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80b7780 con 0x7f9cf8161860 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfe5b3700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 0x7f9cf80b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfddb2700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfd5b1700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 0x7f9cf80b5280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9cf8061220 tx=0x7f9cec007b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec007020 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec007180 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0064f0 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80b7a20 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.681+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80cac10 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf80c8a50 con 0x7f9cf8161860 2020-11-16T22:35:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9cec010030 con 0x7f9cf8161860 2020-11-16T22:35:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cf5ffb700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 0x7f9cd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cfddb2700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 0x7f9cd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cec06d6a0 con 0x7f9cf8161860 2020-11-16T22:35:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.685+0000 7f9cfddb2700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 0x7f9cd0057520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f9ce80092d0 tx=0x7f9ce8016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.693+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9cec0437f0 con 0x7f9cf8161860 2020-11-16T22:35:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.897+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cf8044f60 con 0x7f9cf8161860 2020-11-16T22:35:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.897+0000 7f9cf5ffb700 1 -- 172.21.15.181:0/3003442954 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7f9cec046e10 con 0x7f9cf8161860 2020-11-16T22:35:06.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:06.889 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 msgr2=0x7f9cd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 0x7f9cd0057520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f9ce80092d0 tx=0x7f9ce8016000).stop 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 msgr2=0x7f9cf80b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 0x7f9cf80b5280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9cf8061220 tx=0x7f9cec007b00).stop 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 shutdown_connections 2020-11-16T22:35:06.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9cd0055060 0x7f9cd0057520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cf80b8160 0x7f9cf80b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf8161480 0x7f9cf8173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 --2- 172.21.15.181:0/3003442954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf8161860 0x7f9cf80b5280 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 >> 172.21.15.181:0/3003442954 conn(0x7f9cf80d2dc0 msgr2=0x7f9cf80ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 shutdown_connections 2020-11-16T22:35:06.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:06.901+0000 7f9cfedb4700 1 -- 172.21.15.181:0/3003442954 wait complete. 2020-11-16T22:35:06.894 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:35:06.908 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'b', 'i'] 2020-11-16T22:35:06.908 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:35:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 Processor -- start 2020-11-16T22:35:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- start start 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3380b2190 0x7fb33815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 0x7fb3380cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 0x7fb338160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380ce460 con 0x7fb33815ff80 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380ce700 con 0x7fb3380b2190 2020-11-16T22:35:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380ce9a0 con 0x7fb3380b2570 2020-11-16T22:35:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3380b2190 0x7fb33815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3380b2190 0x7fb33815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36158/0 (socket says 172.21.15.181:36158) 2020-11-16T22:35:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 -- 172.21.15.181:0/1171196068 learned_addr learned my addr 172.21.15.181:0/1171196068 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33caee700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 0x7fb3380cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 -- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 msgr2=0x7fb3380cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 0x7fb3380cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 -- 172.21.15.181:0/1171196068 >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 msgr2=0x7fb338160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33daf0700 1 --2- 172.21.15.181:0/1171196068 >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 0x7fb338160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/1171196068 >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 0x7fb338160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 -- 172.21.15.181:0/1171196068 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380cec40 con 0x7fb3380b2190 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33caee700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 0x7fb3380cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33daf0700 1 --2- 172.21.15.181:0/1171196068 >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 0x7fb338160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:35:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2190 0x7fb33815de80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fb328009d50 tx=0x7fb328009290).ready entity=mon.1 client_cookie=64791db785392cd9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb337fff700 1 -- 172.21.15.181:0/1171196068 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800bc40 con 0x7fb3380b2190 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb337fff700 1 -- 172.21.15.181:0/1171196068 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800bda0 con 0x7fb3380b2190 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb337fff700 1 -- 172.21.15.181:0/1171196068 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328011610 con 0x7fb3380b2190 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2190 msgr2=0x7fb33815de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2190 0x7fb33815de80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fb328009d50 tx=0x7fb328009290).stop 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- 172.21.15.181:0/1171196068 shutdown_connections 2020-11-16T22:35:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/1171196068 >> v2:172.21.15.175:3300/0 conn(0x7fb33815ff80 0x7fb338160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2190 0x7fb33815de80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/1171196068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2570 0x7fb3380cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- 172.21.15.181:0/1171196068 >> 172.21.15.181:0/1171196068 conn(0x7fb3380cfe90 msgr2=0x7fb3380d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:07.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- 172.21.15.181:0/1171196068 shutdown_connections 2020-11-16T22:35:07.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.033+0000 7fb33e2f1700 1 -- 172.21.15.181:0/1171196068 wait complete. 2020-11-16T22:35:07.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 Processor -- start 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- start start 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2190 0x7fb3380d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 0x7fb338170a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380d2b70 con 0x7fb33815ff80 2020-11-16T22:35:07.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33815a010 con 0x7fb3380b2570 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380d1220 con 0x7fb3380b2190 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36164/0 (socket says 172.21.15.181:36164) 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 -- 172.21.15.181:0/54672339 learned_addr learned my addr 172.21.15.181:0/54672339 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2190 0x7fb3380d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33daf0700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 0x7fb338170a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 -- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2190 msgr2=0x7fb3380d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2190 0x7fb3380d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 -- 172.21.15.181:0/54672339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 msgr2=0x7fb338170a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 0x7fb338170a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 -- 172.21.15.181:0/54672339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380d14c0 con 0x7fb3380b2570 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33daf0700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 0x7fb338170a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33caee700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fb32c00db60 tx=0x7fb32c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c009790 con 0x7fb3380b2570 2020-11-16T22:35:07.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0098f0 con 0x7fb3380b2570 2020-11-16T22:35:07.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3380d1760 con 0x7fb3380b2570 2020-11-16T22:35:07.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c009cb0 con 0x7fb3380b2570 2020-11-16T22:35:07.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.037+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3380d1ca0 con 0x7fb3380b2570 2020-11-16T22:35:07.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb338044f60 con 0x7fb3380b2570 2020-11-16T22:35:07.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb32c013690 con 0x7fb3380b2570 2020-11-16T22:35:07.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb3357fa700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 0x7fb310057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb32c06e380 con 0x7fb3380b2570 2020-11-16T22:35:07.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 0x7fb310057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.041+0000 7fb33d2ef700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 0x7fb310057570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb3280090f0 tx=0x7fb328016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.045+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb32c044600 con 0x7fb3380b2570 2020-11-16T22:35:07.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb33805c4a0 con 0x7fb3380b2570 2020-11-16T22:35:07.244 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb3357fa700 1 -- 172.21.15.181:0/54672339 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1690 (secure 0 0 0) 0x7fb32c047c20 con 0x7fb3380b2570 2020-11-16T22:35:07.244 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:07.244 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":512,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":115880,"bytes_used":61128704,"bytes_avail":773032984576,"bytes_total":773094113280,"write_bytes_sec":426,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":107,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"i","status":"up:active","gid":7396},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7399},{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7405}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":34,"modified":"2020-11-16T22:35:06.830544+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:35:07.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 msgr2=0x7fb310057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 0x7fb310057570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb3280090f0 tx=0x7fb328016000).stop 2020-11-16T22:35:07.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 msgr2=0x7fb338159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fb32c00db60 tx=0x7fb32c008040).stop 2020-11-16T22:35:07.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 shutdown_connections 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb3100550b0 0x7fb310057570 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb33815ff80 0x7fb338170a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b2190 0x7fb3380d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 --2- 172.21.15.181:0/54672339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3380b2570 0x7fb338159b20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 >> 172.21.15.181:0/54672339 conn(0x7fb3380cfe90 msgr2=0x7fb3380600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 shutdown_connections 2020-11-16T22:35:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.253+0000 7fb33e2f1700 1 -- 172.21.15.181:0/54672339 wait complete. 2020-11-16T22:35:07.264 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:35:07.264 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:35:07.264 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:07.301 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:07.319 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:07.335 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:35:07.335 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:35:07.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:07.336 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:35:07.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:35:07.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:07.420 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:35:07.436 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:35:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 Processor -- start 2020-11-16T22:35:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 -- start start 2020-11-16T22:35:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3260161480 0x7f3260164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3260161860 0x7f3260062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600c8db0 con 0x7f3260161480 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600c8330 con 0x7f3260161860 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266b47700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c85d0 con 0x7f3260164e10 2020-11-16T22:35:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44014/0 (socket says 172.21.15.181:44014) 2020-11-16T22:35:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 -- 172.21.15.181:0/610483598 learned_addr learned my addr 172.21.15.181:0/610483598 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3266346700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.181:3300/0 conn(0x7f3260161860 0x7f3260062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265b45700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.175:3300/0 conn(0x7f3260161480 0x7f3260164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 -- 172.21.15.181:0/610483598 >> v2:172.21.15.181:3300/0 conn(0x7f3260161860 msgr2=0x7f3260062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.181:3300/0 conn(0x7f3260161860 0x7f3260062740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 -- 172.21.15.181:0/610483598 >> v2:172.21.15.175:3300/0 conn(0x7f3260161480 msgr2=0x7f3260164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.175:3300/0 conn(0x7f3260161480 0x7f3260164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 -- 172.21.15.181:0/610483598 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600c8870 con 0x7f3260164e10 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3265344700 1 --2- 172.21.15.181:0/610483598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f32600b8580 tx=0x7f3254006eb0).ready entity=mon.2 client_cookie=c0070d991e028eb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3264b43700 1 -- 172.21.15.181:0/610483598 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254009810 con 0x7f3260164e10 2020-11-16T22:35:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3264b43700 1 -- 172.21.15.181:0/610483598 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254009970 con 0x7f3260164e10 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.589+0000 7f3264b43700 1 -- 172.21.15.181:0/610483598 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254009cf0 con 0x7f3260164e10 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- 172.21.15.181:0/610483598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 msgr2=0x7f3260062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- 172.21.15.181:0/610483598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f32600b8580 tx=0x7f3254006eb0).stop 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- 172.21.15.181:0/610483598 shutdown_connections 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.181:3300/0 conn(0x7f3260161860 0x7f3260062740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- 172.21.15.181:0/610483598 >> v2:172.21.15.175:3300/0 conn(0x7f3260161480 0x7f3260164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- 172.21.15.181:0/610483598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260164e10 0x7f3260062250 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- 172.21.15.181:0/610483598 >> 172.21.15.181:0/610483598 conn(0x7f32600d2dc0 msgr2=0x7f32600ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:07.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- 172.21.15.181:0/610483598 shutdown_connections 2020-11-16T22:35:07.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- 172.21.15.181:0/610483598 wait complete. 2020-11-16T22:35:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 Processor -- start 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- start start 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 0x7f3260147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32600ce8d0 0x7f3260146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260146ad0 con 0x7f3260161480 2020-11-16T22:35:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260146d70 con 0x7f32600ce8d0 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.593+0000 7f3266b47700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3260147010 con 0x7f3260161860 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44022/0 (socket says 172.21.15.181:44022) 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 -- 172.21.15.181:0/393201322 learned_addr learned my addr 172.21.15.181:0/393201322 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3266346700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32600ce8d0 0x7f3260146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265b45700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 0x7f3260147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 -- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32600ce8d0 msgr2=0x7f3260146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32600ce8d0 0x7f3260146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 -- 172.21.15.181:0/393201322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 msgr2=0x7f3260147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 0x7f3260147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 -- 172.21.15.181:0/393201322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260173af0 con 0x7f3260161860 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265b45700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 0x7f3260147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265344700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f3254009fd0 tx=0x7f3254015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254011d50 con 0x7f3260161860 2020-11-16T22:35:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325401e460 con 0x7f3260161860 2020-11-16T22:35:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3260173d90 con 0x7f3260161860 2020-11-16T22:35:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325401e820 con 0x7f3260161860 2020-11-16T22:35:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32601484d0 con 0x7f3260161860 2020-11-16T22:35:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3260173f50 con 0x7f3260161860 2020-11-16T22:35:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f325401a040 con 0x7f3260161860 2020-11-16T22:35:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 0x7f3238057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f325406f0d0 con 0x7f3260161860 2020-11-16T22:35:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.597+0000 7f3265b45700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 0x7f3238057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.601+0000 7f3265b45700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 0x7f3238057520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3250002820 tx=0x7f3250008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.605+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f32540453e0 con 0x7f3260161860 2020-11-16T22:35:07.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.757+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3260044f60 con 0x7f3260161860 2020-11-16T22:35:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f325dffb700 1 -- 172.21.15.181:0/393201322 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3254023020 con 0x7f3260161860 2020-11-16T22:35:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 msgr2=0x7f3238057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 0x7f3238057520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3250002820 tx=0x7f3250008040).stop 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 msgr2=0x7f3260174230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f3254009fd0 tx=0x7f3254015000).stop 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 shutdown_connections 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3238055060 0x7f3238057520 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32600ce8d0 0x7f3260146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3260161480 0x7f3260147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 --2- 172.21.15.181:0/393201322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3260161860 0x7f3260174230 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 >> 172.21.15.181:0/393201322 conn(0x7f32600d2dc0 msgr2=0x7f32600609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 shutdown_connections 2020-11-16T22:35:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.769+0000 7f3266b47700 1 -- 172.21.15.181:0/393201322 wait complete. 2020-11-16T22:35:07.773 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2020-11-16T22:35:07.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.901+0000 7f92d6ce0700 1 Processor -- start 2020-11-16T22:35:07.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.901+0000 7f92d6ce0700 1 -- start start 2020-11-16T22:35:07.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92d00615f0 0x7f92d00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d0160e40 con 0x7f92d015f5b0 2020-11-16T22:35:07.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d0160ba0 con 0x7f92d0062790 2020-11-16T22:35:07.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d01609a0 con 0x7f92d00615f0 2020-11-16T22:35:07.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f92d0062790 0x7f92d0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50494/0 (socket says 172.21.15.181:50494) 2020-11-16T22:35:07.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 -- 172.21.15.181:0/1542002274 learned_addr learned my addr 172.21.15.181:0/1542002274 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:07.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cffff700 1 --2- 172.21.15.181:0/1542002274 >> v2:172.21.15.175:3300/0 conn(0x7f92d00615f0 0x7f92d00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497810827 0 0) 0x7f92d0160ba0 con 0x7f92d0062790 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92b0003240 con 0x7f92d0062790 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1828270884 0 0) 0x7f92b0003240 con 0x7f92d0062790 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 >> v2:172.21.15.175:3300/0 conn(0x7f92d00615f0 msgr2=0x7f92d00622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/1542002274 >> v2:172.21.15.175:3300/0 conn(0x7f92d00615f0 0x7f92d00622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 >> v1:172.21.15.175:6789/0 conn(0x7f92d015f5b0 legacy=0x7f92d00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0060970 con 0x7f92d0062790 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f92c0001860 con 0x7f92d0062790 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 >> v1:172.21.15.181:6789/0 conn(0x7f92d0062790 legacy=0x7f92d0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b0003730 0x7f92b0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b00044e0 0x7f92b0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 0x7f92b000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01609a0 con 0x7f92b00044e0 2020-11-16T22:35:07.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b0004350 con 0x7f92b0004e50 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b000bc50 con 0x7f92b0003730 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 0x7f92b000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d4a7c700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b0003730 0x7f92b0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cffff700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b00044e0 0x7f92b0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 -- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b0003730 msgr2=0x7f92b0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92b0003730 0x7f92b0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 -- 172.21.15.181:0/1542002274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b00044e0 msgr2=0x7f92b0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b00044e0 0x7f92b0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 -- 172.21.15.181:0/1542002274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0003240 con 0x7f92b0004e50 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cffff700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b00044e0 0x7f92b0004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 -- 172.21.15.181:0/1542002274 reap_dead start 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d527d700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 0x7f92b000b730 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f92c000c6a0 tx=0x7f92d00b8580).ready entity=mon.1 client_cookie=872ac5087532a426 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c0005b80 con 0x7f92b0004e50 2020-11-16T22:35:07.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c0005ce0 con 0x7f92b0004e50 2020-11-16T22:35:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92cf7fe700 1 -- 172.21.15.181:0/1542002274 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c00072e0 con 0x7f92b0004e50 2020-11-16T22:35:07.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 msgr2=0x7f92b000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 0x7f92b000b730 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f92c000c6a0 tx=0x7f92d00b8580).stop 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/1542002274 shutdown_connections 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/1542002274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92b0004e50 0x7f92b000b730 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/1542002274 >> 172.21.15.181:0/1542002274 conn(0x7f92d00d69a0 msgr2=0x7f92d00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/1542002274 shutdown_connections 2020-11-16T22:35:07.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/1542002274 wait complete. 2020-11-16T22:35:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.909+0000 7f92d6ce0700 1 Processor -- start 2020-11-16T22:35:07.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.909+0000 7f92d6ce0700 1 -- start start 2020-11-16T22:35:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b0004e50 0x7f92d0148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 0x7f92d0172690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01735a0 con 0x7f92b0004e50 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0173300 con 0x7f92d0148c30 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d016f240 con 0x7f92d0170a80 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d527d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 0x7f92d0172690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36200/0 (socket says 172.21.15.181:36200) 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 -- 172.21.15.181:0/3826830149 learned_addr learned my addr 172.21.15.181:0/3826830149 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d4a7c700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b0004e50 0x7f92d0148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 -- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 msgr2=0x7f92d0172690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 0x7f92d0172690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 -- 172.21.15.181:0/3826830149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b0004e50 msgr2=0x7f92d0148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:07.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b0004e50 0x7f92d0148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 -- 172.21.15.181:0/3826830149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d016f4e0 con 0x7f92d0148c30 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d527d700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 0x7f92d0172690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92cffff700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f92b800e970 tx=0x7f92b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80126d0 con 0x7f92d0148c30 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8012830 con 0x7f92d0148c30 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d016f780 con 0x7f92d0148c30 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8012bb0 con 0x7f92d0148c30 2020-11-16T22:35:07.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d0170ee0 con 0x7f92d0148c30 2020-11-16T22:35:07.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.913+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d0069eb0 con 0x7f92d0148c30 2020-11-16T22:35:07.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.917+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f92b8014430 con 0x7f92d0148c30 2020-11-16T22:35:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.917+0000 7f92ccff9700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 0x7f92ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.917+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f92b806e270 con 0x7f92d0148c30 2020-11-16T22:35:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.917+0000 7f92d4a7c700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 0x7f92ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:07.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.917+0000 7f92d4a7c700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 0x7f92ac057520 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f92c80010f0 tx=0x7f92c8007b50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:07.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:07.925+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f92b80444f0 con 0x7f92d0148c30 2020-11-16T22:35:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.129+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f92d00600f0 con 0x7f92d0148c30 2020-11-16T22:35:08.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.901+0000 7f92ccff9700 1 -- 172.21.15.181:0/3826830149 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v402) v1 ==== 154+0+0 (secure 0 0 0) 0x7f92b8047b10 con 0x7f92d0148c30 2020-11-16T22:35:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 msgr2=0x7f92ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 0x7f92ac057520 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f92c80010f0 tx=0x7f92c8007b50).stop 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 msgr2=0x7f92d0170590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f92b800e970 tx=0x7f92b8008040).stop 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 shutdown_connections 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f92ac055060 0x7f92ac057520 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92d0170a80 0x7f92d0172690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b0004e50 0x7f92d0148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 --2- 172.21.15.181:0/3826830149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0148c30 0x7f92d0170590 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 >> 172.21.15.181:0/3826830149 conn(0x7f92d00d69a0 msgr2=0x7f92d00a7630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 shutdown_connections 2020-11-16T22:35:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:08.905+0000 7f92d6ce0700 1 -- 172.21.15.181:0/3826830149 wait complete. 2020-11-16T22:35:08.913 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:35:08.913 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:35:08.914 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2020-11-16T22:35:09.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.057+0000 7f448130f700 1 Processor -- start 2020-11-16T22:35:09.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.057+0000 7f448130f700 1 -- start start 2020-11-16T22:35:09.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f447c0b2190 0x7f447c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0ce460 con 0x7f447c15ff80 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0ce700 con 0x7f447c0b2190 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0ce9a0 con 0x7f447c0b2570 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44050/0 (socket says 172.21.15.181:44050) 2020-11-16T22:35:09.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 -- 172.21.15.181:0/1183398081 learned_addr learned my addr 172.21.15.181:0/1183398081 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:09.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447bfff700 1 --2- 172.21.15.181:0/1183398081 >> v2:172.21.15.175:3300/0 conn(0x7f447c0b2190 0x7f447c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 -- 172.21.15.181:0/1183398081 >> v2:172.21.15.175:3300/0 conn(0x7f447c0b2190 msgr2=0x7f447c15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:09.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 --2- 172.21.15.181:0/1183398081 >> v2:172.21.15.175:3300/0 conn(0x7f447c0b2190 0x7f447c15de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 -- 172.21.15.181:0/1183398081 >> v1:172.21.15.181:6789/0 conn(0x7f447c15ff80 legacy=0x7f447c160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 -- 172.21.15.181:0/1183398081 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0cec40 con 0x7f447c0b2570 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447affd700 1 -- 172.21.15.181:0/1183398081 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539559870 0 0) 0x7f447c0ce460 con 0x7f447c15ff80 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447b7fe700 1 --2- 172.21.15.181:0/1183398081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f4470006fd0 tx=0x7f4470006eb0).ready entity=mon.2 client_cookie=46f04367f3f3036d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447affd700 1 -- 172.21.15.181:0/1183398081 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009810 con 0x7f447c0b2570 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447affd700 1 -- 172.21.15.181:0/1183398081 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470009970 con 0x7f447c0b2570 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f447affd700 1 -- 172.21.15.181:0/1183398081 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009cf0 con 0x7f447c0b2570 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- 172.21.15.181:0/1183398081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 msgr2=0x7f447c0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 --2- 172.21.15.181:0/1183398081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f4470006fd0 tx=0x7f4470006eb0).stop 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- 172.21.15.181:0/1183398081 shutdown_connections 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 --2- 172.21.15.181:0/1183398081 >> v2:172.21.15.175:3300/0 conn(0x7f447c0b2190 0x7f447c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 --2- 172.21.15.181:0/1183398081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c0cf620 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- 172.21.15.181:0/1183398081 >> 172.21.15.181:0/1183398081 conn(0x7f447c0cfe90 msgr2=0x7f447c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- 172.21.15.181:0/1183398081 shutdown_connections 2020-11-16T22:35:09.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.061+0000 7f448130f700 1 -- 172.21.15.181:0/1183398081 wait complete. 2020-11-16T22:35:09.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 Processor -- start 2020-11-16T22:35:09.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 -- start start 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447c0b2190 0x7f447c1616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447c15ff80 0x7f447c1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c161cf0 con 0x7f447c0b2190 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c161f90 con 0x7f447c15ff80 2020-11-16T22:35:09.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f448130f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c158dc0 con 0x7f447c0b2570 2020-11-16T22:35:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44056/0 (socket says 172.21.15.181:44056) 2020-11-16T22:35:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f447b7fe700 1 -- 172.21.15.181:0/1600642176 learned_addr learned my addr 172.21.15.181:0/1600642176 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f4480b0e700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447c15ff80 0x7f447c1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.065+0000 7f447bfff700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447c0b2190 0x7f447c1616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 -- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447c15ff80 msgr2=0x7f447c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447c15ff80 0x7f447c1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 -- 172.21.15.181:0/1600642176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447c0b2190 msgr2=0x7f447c1616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447c0b2190 0x7f447c1616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 -- 172.21.15.181:0/1600642176 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c159060 con 0x7f447c0b2570 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447b7fe700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f4470009e10 tx=0x7f4470015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470011c80 con 0x7f447c0b2570 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470011de0 con 0x7f447c0b2570 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c159300 con 0x7f447c0b2570 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447001e6b0 con 0x7f447c0b2570 2020-11-16T22:35:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c15a000 con 0x7f447c0b2570 2020-11-16T22:35:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c044f60 con 0x7f447c0b2570 2020-11-16T22:35:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4470025030 con 0x7f447c0b2570 2020-11-16T22:35:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 0x7f4454057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f447006f340 con 0x7f447c0b2570 2020-11-16T22:35:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447bfff700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 0x7f4454057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.069+0000 7f447bfff700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 0x7f4454057520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f446c006fd0 tx=0x7f446c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.073+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4470045650 con 0x7f447c0b2570 2020-11-16T22:35:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.285+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7f447c05c4a0 con 0x7f447c0b2570 2020-11-16T22:35:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.909+0000 7f445bfff700 1 -- 172.21.15.181:0/1600642176 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v403) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4470023020 con 0x7f447c0b2570 2020-11-16T22:35:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 msgr2=0x7f4454057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 0x7f4454057520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f446c006fd0 tx=0x7f446c006df0).stop 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 msgr2=0x7f447c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f4470009e10 tx=0x7f4470015000).stop 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 shutdown_connections 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4454055060 0x7f4454057520 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f447c15ff80 0x7f447c1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f447c0b2190 0x7f447c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 --2- 172.21.15.181:0/1600642176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f447c0b2570 0x7f447c159b10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 >> 172.21.15.181:0/1600642176 conn(0x7f447c0cfe90 msgr2=0x7f447c0d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 shutdown_connections 2020-11-16T22:35:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:09.913+0000 7f448130f700 1 -- 172.21.15.181:0/1600642176 wait complete. 2020-11-16T22:35:09.918 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:35:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 Processor -- start 2020-11-16T22:35:10.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 -- start start 2020-11-16T22:35:10.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf0145a30 0x7f4bf0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf0061b50 con 0x7f4bf01492c0 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00618b0 con 0x7f4bf0145a30 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf64d5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00cb930 con 0x7f4bf00816e0 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf54d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf0145a30 0x7f4bf0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf5cd4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf5cd4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44064/0 (socket says 172.21.15.181:44064) 2020-11-16T22:35:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf54d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf0145a30 0x7f4bf0145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36220/0 (socket says 172.21.15.181:36220) 2020-11-16T22:35:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.065+0000 7f4bf5cd4700 1 -- 172.21.15.181:0/1799776510 learned_addr learned my addr 172.21.15.181:0/1799776510 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4be7fff700 1 -- 172.21.15.181:0/1799776510 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231688321 0 0) 0x7f4bf0061b50 con 0x7f4bf01492c0 2020-11-16T22:35:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4be7fff700 1 -- 172.21.15.181:0/1799776510 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd4003240 con 0x7f4bf01492c0 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf54d3700 1 -- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 msgr2=0x7f4bf00cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf54d3700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf54d3700 1 -- 172.21.15.181:0/1799776510 >> v1:172.21.15.175:6789/0 conn(0x7f4bf01492c0 legacy=0x7f4bf0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf54d3700 1 -- 172.21.15.181:0/1799776510 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00cbbd0 con 0x7f4bf0145a30 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf5cd4700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf54d3700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf0145a30 0x7f4bf0145e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f4bec00b9a0 tx=0x7f4bec006df0).ready entity=mon.1 client_cookie=52f24db277ce28c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4be7fff700 1 -- 172.21.15.181:0/1799776510 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec009680 con 0x7f4bf0145a30 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4be7fff700 1 -- 172.21.15.181:0/1799776510 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec0097e0 con 0x7f4bf0145a30 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4be7fff700 1 -- 172.21.15.181:0/1799776510 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec009b60 con 0x7f4bf0145a30 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 -- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf0145a30 msgr2=0x7f4bf0145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf0145a30 0x7f4bf0145e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f4bec00b9a0 tx=0x7f4bec006df0).stop 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 -- 172.21.15.181:0/1799776510 shutdown_connections 2020-11-16T22:35:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/1799776510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf0145a30 0x7f4bf0145e10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 -- 172.21.15.181:0/1799776510 >> 172.21.15.181:0/1799776510 conn(0x7f4bf00d68c0 msgr2=0x7f4bf00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 -- 172.21.15.181:0/1799776510 shutdown_connections 2020-11-16T22:35:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.069+0000 7f4bf64d5700 1 -- 172.21.15.181:0/1799776510 wait complete. 2020-11-16T22:35:10.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 Processor -- start 2020-11-16T22:35:10.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- start start 2020-11-16T22:35:10.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf0145a30 0x7f4bf016f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf01492c0 0x7f4bf00d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf016fbb0 con 0x7f4bf0145a30 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf016fe50 con 0x7f4bf01492c0 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01700f0 con 0x7f4bf00816e0 2020-11-16T22:35:10.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44068/0 (socket says 172.21.15.181:44068) 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 -- 172.21.15.181:0/2446258605 learned_addr learned my addr 172.21.15.181:0/2446258605 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf5cd4700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf01492c0 0x7f4bf00d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf4cd2700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf0145a30 0x7f4bf016f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 -- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf01492c0 msgr2=0x7f4bf00d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf01492c0 0x7f4bf00d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 -- 172.21.15.181:0/2446258605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf0145a30 msgr2=0x7f4bf016f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf0145a30 0x7f4bf016f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 -- 172.21.15.181:0/2446258605 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00d47c0 con 0x7f4bf00816e0 2020-11-16T22:35:10.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf54d3700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f4bf00d0f10 tx=0x7f4bec015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0067c0 con 0x7f4bf00816e0 2020-11-16T22:35:10.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec011c40 con 0x7f4bf00816e0 2020-11-16T22:35:10.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf00d4a60 con 0x7f4bf00816e0 2020-11-16T22:35:10.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01e460 con 0x7f4bf00816e0 2020-11-16T22:35:10.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.073+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00d3900 con 0x7f4bf00816e0 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4bec025030 con 0x7f4bf00816e0 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0044f60 con 0x7f4bf00816e0 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4be57fa700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4bf4cd2700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4bec06f0e0 con 0x7f4bf00816e0 2020-11-16T22:35:10.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.077+0000 7f4bf4cd2700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f4bf0167aa0 tx=0x7f4be0007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.085+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4bec045470 con 0x7f4bf00816e0 2020-11-16T22:35:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.237+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f4bf0068e30 con 0x7f4bf00816e0 2020-11-16T22:35:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4be57fa700 1 -- 172.21.15.181:0/2446258605 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4bec023070 con 0x7f4bf00816e0 2020-11-16T22:35:10.237 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 msgr2=0x7f4bc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f4bf0167aa0 tx=0x7f4be0007a20).stop 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 msgr2=0x7f4bf0141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f4bf00d0f10 tx=0x7f4bec015000).stop 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 shutdown_connections 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4bc8055060 0x7f4bc8057520 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf01492c0 0x7f4bf00d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf00816e0 0x7f4bf0141e90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 --2- 172.21.15.181:0/2446258605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bf0145a30 0x7f4bf016f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 >> 172.21.15.181:0/2446258605 conn(0x7f4bf00d68c0 msgr2=0x7f4bf0080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:10.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 shutdown_connections 2020-11-16T22:35:10.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.249+0000 7f4bf64d5700 1 -- 172.21.15.181:0/2446258605 wait complete. 2020-11-16T22:35:10.252 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:35:10.252 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:10.289 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:10.306 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:35:10.324 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:35:10.324 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:35:10.324 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:10.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.509+0000 7f6ce2181700 1 Processor -- start 2020-11-16T22:35:10.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.509+0000 7f6ce2181700 1 -- start start 2020-11-16T22:35:10.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 0x7f6cdc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc061b50 con 0x7f6cdc0816e0 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc0618b0 con 0x7f6cdc1492c0 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0cb930 con 0x7f6cdc145a30 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce097e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6cdc1492c0 0x7f6cdc080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50538/0 (socket says 172.21.15.181:50538) 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce097e700 1 -- 172.21.15.181:0/184591683 learned_addr learned my addr 172.21.15.181:0/184591683 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:10.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce117f700 1 --2- 172.21.15.181:0/184591683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 0x7f6cdc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ccffff700 1 -- 172.21.15.181:0/184591683 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333360093 0 0) 0x7f6cdc0618b0 con 0x7f6cdc1492c0 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ccffff700 1 -- 172.21.15.181:0/184591683 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc0003240 con 0x7f6cdc1492c0 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce117f700 1 -- 172.21.15.181:0/184591683 >> v1:172.21.15.181:6789/0 conn(0x7f6cdc1492c0 legacy=0x7f6cdc080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce117f700 1 -- 172.21.15.181:0/184591683 >> v1:172.21.15.175:6789/0 conn(0x7f6cdc0816e0 legacy=0x7f6cdc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce117f700 1 -- 172.21.15.181:0/184591683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0cbbd0 con 0x7f6cdc145a30 2020-11-16T22:35:10.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce117f700 1 --2- 172.21.15.181:0/184591683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 0x7f6cdc145e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6cd800b9d0 tx=0x7f6cd8006df0).ready entity=mon.2 client_cookie=842aa1b005121ffd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ccffff700 1 -- 172.21.15.181:0/184591683 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80097b0 con 0x7f6cdc145a30 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ccffff700 1 -- 172.21.15.181:0/184591683 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd8009910 con 0x7f6cdc145a30 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ccffff700 1 -- 172.21.15.181:0/184591683 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8009c90 con 0x7f6cdc145a30 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- 172.21.15.181:0/184591683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 msgr2=0x7f6cdc145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 --2- 172.21.15.181:0/184591683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 0x7f6cdc145e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6cd800b9d0 tx=0x7f6cd8006df0).stop 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- 172.21.15.181:0/184591683 shutdown_connections 2020-11-16T22:35:10.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 --2- 172.21.15.181:0/184591683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc145a30 0x7f6cdc145e10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- 172.21.15.181:0/184591683 >> 172.21.15.181:0/184591683 conn(0x7f6cdc0d68c0 msgr2=0x7f6cdc0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:10.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- 172.21.15.181:0/184591683 shutdown_connections 2020-11-16T22:35:10.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 -- 172.21.15.181:0/184591683 wait complete. 2020-11-16T22:35:10.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.513+0000 7f6ce2181700 1 Processor -- start 2020-11-16T22:35:10.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- start start 2020-11-16T22:35:10.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 0x7f6cdc15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cdc1492c0 0x7f6cdc146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1736a0 con 0x7f6cdc1492c0 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0d33d0 con 0x7f6cdc145a30 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0d3670 con 0x7f6cdc0816e0 2020-11-16T22:35:10.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce097e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 0x7f6cdc15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44082/0 (socket says 172.21.15.181:44082) 2020-11-16T22:35:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 -- 172.21.15.181:0/611752060 learned_addr learned my addr 172.21.15.181:0/611752060 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce1980700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cdc1492c0 0x7f6cdc146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 -- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 msgr2=0x7f6cdc15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 0x7f6cdc15e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 -- 172.21.15.181:0/611752060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cdc1492c0 msgr2=0x7f6cdc146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cdc1492c0 0x7f6cdc146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 -- 172.21.15.181:0/611752060 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0d3910 con 0x7f6cdc0816e0 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce097e700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 0x7f6cdc15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce117f700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6cd800be90 tx=0x7f6cd8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8011d50 con 0x7f6cdc0816e0 2020-11-16T22:35:10.509 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd801d3c0 con 0x7f6cdc0816e0 2020-11-16T22:35:10.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0d3bb0 con 0x7f6cdc0816e0 2020-11-16T22:35:10.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd801d780 con 0x7f6cdc0816e0 2020-11-16T22:35:10.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.517+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc0d4050 con 0x7f6cdc0816e0 2020-11-16T22:35:10.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6cd801a040 con 0x7f6cdc0816e0 2020-11-16T22:35:10.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc0d4240 con 0x7f6cdc0816e0 2020-11-16T22:35:10.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ccd7fa700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 0x7f6cb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6cd806f330 con 0x7f6cdc0816e0 2020-11-16T22:35:10.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ce097e700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 0x7f6cb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.521+0000 7f6ce097e700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 0x7f6cb4057520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f6cdc141ad0 tx=0x7f6cd0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.529+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6cd8045480 con 0x7f6cdc0816e0 2020-11-16T22:35:10.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.729+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cdc044f60 con 0x7f6cdc0816e0 2020-11-16T22:35:10.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.729+0000 7f6ccd7fa700 1 -- 172.21.15.181:0/611752060 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7f6cd8024020 con 0x7f6cdc0816e0 2020-11-16T22:35:10.719 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:10.719 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:35:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 msgr2=0x7f6cb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 0x7f6cb4057520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f6cdc141ad0 tx=0x7f6cd0009040).stop 2020-11-16T22:35:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 msgr2=0x7f6cdc15df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6cd800be90 tx=0x7f6cd8015000).stop 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 shutdown_connections 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cb4055060 0x7f6cb4057520 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cdc1492c0 0x7f6cdc146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc0816e0 0x7f6cdc15df80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 --2- 172.21.15.181:0/611752060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc145a30 0x7f6cdc15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 >> 172.21.15.181:0/611752060 conn(0x7f6cdc0d68c0 msgr2=0x7f6cdc080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 shutdown_connections 2020-11-16T22:35:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.733+0000 7f6ce2181700 1 -- 172.21.15.181:0/611752060 wait complete. 2020-11-16T22:35:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:35:10.734 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:10.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 Processor -- start 2020-11-16T22:35:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 -- start start 2020-11-16T22:35:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c168390 con 0x7fea1c1672c0 2020-11-16T22:35:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c0b2060 con 0x7fea1c0b3220 2020-11-16T22:35:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea22acc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c167770 con 0x7fea1c167ac0 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44090/0 (socket says 172.21.15.181:44090) 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 -- 172.21.15.181:0/743977216 learned_addr learned my addr 172.21.15.181:0/743977216 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea1bfff700 1 --1- 172.21.15.181:0/743977216 >> v1:172.21.15.181:6789/0 conn(0x7fea1c0b3220 0x7fea1c166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50554/0 (socket says 172.21.15.181:50554) 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea1b7fe700 1 -- 172.21.15.181:0/743977216 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674087910 0 0) 0x7fea1c0b2060 con 0x7fea1c0b3220 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 -- 172.21.15.181:0/743977216 >> v1:172.21.15.181:6789/0 conn(0x7fea1c0b3220 legacy=0x7fea1c166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 -- 172.21.15.181:0/743977216 >> v1:172.21.15.175:6789/0 conn(0x7fea1c1672c0 legacy=0x7fea1c161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 -- 172.21.15.181:0/743977216 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0b2300 con 0x7fea1c167ac0 2020-11-16T22:35:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea20868700 1 --2- 172.21.15.181:0/743977216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fea04009d80 tx=0x7fea04009290).ready entity=mon.2 client_cookie=e941e0f5a32c68dd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea1b7fe700 1 -- 172.21.15.181:0/743977216 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0400bc80 con 0x7fea1c167ac0 2020-11-16T22:35:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea1b7fe700 1 -- 172.21.15.181:0/743977216 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0400bde0 con 0x7fea1c167ac0 2020-11-16T22:35:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.869+0000 7fea1b7fe700 1 -- 172.21.15.181:0/743977216 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011610 con 0x7fea1c167ac0 2020-11-16T22:35:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- 172.21.15.181:0/743977216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 msgr2=0x7fea1c167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 --2- 172.21.15.181:0/743977216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fea04009d80 tx=0x7fea04009290).stop 2020-11-16T22:35:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- 172.21.15.181:0/743977216 shutdown_connections 2020-11-16T22:35:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 --2- 172.21.15.181:0/743977216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c167ac0 0x7fea1c167ea0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- 172.21.15.181:0/743977216 >> 172.21.15.181:0/743977216 conn(0x7fea1c060bc0 msgr2=0x7fea1c0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- 172.21.15.181:0/743977216 shutdown_connections 2020-11-16T22:35:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- 172.21.15.181:0/743977216 wait complete. 2020-11-16T22:35:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 Processor -- start 2020-11-16T22:35:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- start start 2020-11-16T22:35:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea1c0b3220 0x7fea1c062950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 0x7fea1c174b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c061a30 con 0x7fea1c0b3220 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c061cd0 con 0x7fea1c167ac0 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea22acc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c061f70 con 0x7fea1c1672c0 2020-11-16T22:35:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea21069700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 0x7fea1c174b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44100/0 (socket says 172.21.15.181:44100) 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea1bfff700 1 -- 172.21.15.181:0/1004618402 learned_addr learned my addr 172.21.15.181:0/1004618402 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.873+0000 7fea20868700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea1c0b3220 0x7fea1c062950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 -- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 msgr2=0x7fea1c174b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 0x7fea1c174b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 -- 172.21.15.181:0/1004618402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea1c0b3220 msgr2=0x7fea1c062950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea1c0b3220 0x7fea1c062950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 -- 172.21.15.181:0/1004618402 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c062210 con 0x7fea1c1672c0 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea21069700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 0x7fea1c174b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea1bfff700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fea0c00c760 tx=0x7fea0c007a30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0072e0 con 0x7fea1c1672c0 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c005530 con 0x7fea1c1672c0 2020-11-16T22:35:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0058f0 con 0x7fea1c1672c0 2020-11-16T22:35:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c0624b0 con 0x7fea1c1672c0 2020-11-16T22:35:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c16a7c0 con 0x7fea1c1672c0 2020-11-16T22:35:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fea0c010070 con 0x7fea1c1672c0 2020-11-16T22:35:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c062640 con 0x7fea1c1672c0 2020-11-16T22:35:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 0x7fe9ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea0c06d2e0 con 0x7fea1c1672c0 2020-11-16T22:35:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.877+0000 7fea20868700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 0x7fe9ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.881+0000 7fea20868700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 0x7fe9ec057520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fea040092d0 tx=0x7fea04016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:10.885+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fea0c043540 con 0x7fea1c1672c0 2020-11-16T22:35:11.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea1c16a7c0 con 0x7fea1c1672c0 2020-11-16T22:35:11.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea18ff9700 1 -- 172.21.15.181:0/1004618402 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fea1c16a7c0 con 0x7fea1c1672c0 2020-11-16T22:35:11.029 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:11.029 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 msgr2=0x7fe9ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 0x7fe9ec057520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fea040092d0 tx=0x7fea04016000).stop 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 msgr2=0x7fea1c16a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fea0c00c760 tx=0x7fea0c007a30).stop 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 shutdown_connections 2020-11-16T22:35:11.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe9ec055060 0x7fe9ec057520 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea1c167ac0 0x7fea1c174b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea1c0b3220 0x7fea1c062950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 --2- 172.21.15.181:0/1004618402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea1c1672c0 0x7fea1c16a130 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 >> 172.21.15.181:0/1004618402 conn(0x7fea1c060bc0 msgr2=0x7fea1c162420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 shutdown_connections 2020-11-16T22:35:11.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.041+0000 7fea22acc700 1 -- 172.21.15.181:0/1004618402 wait complete. 2020-11-16T22:35:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:33:53.695421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:33:54.659875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:35:11.045 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:35:11.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.189+0000 7fe1606ff700 1 Processor -- start 2020-11-16T22:35:11.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.189+0000 7fe1606ff700 1 -- start start 2020-11-16T22:35:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1580ce750 0x7fe1580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe158061b70 con 0x7fe1580cf0a0 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15815cda0 con 0x7fe1580ce750 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815d040 con 0x7fe1580612a0 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44104/0 (socket says 172.21.15.181:44104) 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 -- 172.21.15.181:0/518820115 learned_addr learned my addr 172.21.15.181:0/518820115 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15dc9a700 1 --2- 172.21.15.181:0/518820115 >> v2:172.21.15.181:3300/0 conn(0x7fe1580ce750 0x7fe1580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 -- 172.21.15.181:0/518820115 >> v2:172.21.15.181:3300/0 conn(0x7fe1580ce750 msgr2=0x7fe1580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 --2- 172.21.15.181:0/518820115 >> v2:172.21.15.181:3300/0 conn(0x7fe1580ce750 0x7fe1580cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 -- 172.21.15.181:0/518820115 >> v1:172.21.15.181:6789/0 conn(0x7fe1580cf0a0 legacy=0x7fe15815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 -- 172.21.15.181:0/518820115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15815d2e0 con 0x7fe1580612a0 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15e49b700 1 --2- 172.21.15.181:0/518820115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fe148009d80 tx=0x7fe148009290).ready entity=mon.2 client_cookie=bc8d1320b1943923 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15d499700 1 -- 172.21.15.181:0/518820115 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800bb80 con 0x7fe1580612a0 2020-11-16T22:35:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15d499700 1 -- 172.21.15.181:0/518820115 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14800bce0 con 0x7fe1580612a0 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe15d499700 1 -- 172.21.15.181:0/518820115 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148011610 con 0x7fe1580612a0 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- 172.21.15.181:0/518820115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 msgr2=0x7fe158061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 --2- 172.21.15.181:0/518820115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fe148009d80 tx=0x7fe148009290).stop 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- 172.21.15.181:0/518820115 shutdown_connections 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 --2- 172.21.15.181:0/518820115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580612a0 0x7fe158061680 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 --2- 172.21.15.181:0/518820115 >> v2:172.21.15.181:3300/0 conn(0x7fe1580ce750 0x7fe1580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- 172.21.15.181:0/518820115 >> 172.21.15.181:0/518820115 conn(0x7fe1580cf890 msgr2=0x7fe1580cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- 172.21.15.181:0/518820115 shutdown_connections 2020-11-16T22:35:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.193+0000 7fe1606ff700 1 -- 172.21.15.181:0/518820115 wait complete. 2020-11-16T22:35:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 Processor -- start 2020-11-16T22:35:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 -- start start 2020-11-16T22:35:11.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580612a0 0x7fe1580b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1580ce750 0x7fe158145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580b69b0 con 0x7fe1580612a0 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580b6710 con 0x7fe1580ce750 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe1606ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe158170960 con 0x7fe1580cf0a0 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe15ec9c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe15ec9c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44112/0 (socket says 172.21.15.181:44112) 2020-11-16T22:35:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe15ec9c700 1 -- 172.21.15.181:0/1764714799 learned_addr learned my addr 172.21.15.181:0/1764714799 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe15dc9a700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1580ce750 0x7fe158145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.197+0000 7fe15e49b700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580612a0 0x7fe1580b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 -- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1580ce750 msgr2=0x7fe158145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1580ce750 0x7fe158145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 -- 172.21.15.181:0/1764714799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580612a0 msgr2=0x7fe1580b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580612a0 0x7fe1580b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 -- 172.21.15.181:0/1764714799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158170c00 con 0x7fe1580cf0a0 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe15ec9c700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe15000c740 tx=0x7fe150007b30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15000e490 con 0x7fe1580cf0a0 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15000e5f0 con 0x7fe1580cf0a0 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe158170ea0 con 0x7fe1580cf0a0 2020-11-16T22:35:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15000e970 con 0x7fe1580cf0a0 2020-11-16T22:35:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1580b8d20 con 0x7fe1580cf0a0 2020-11-16T22:35:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe158069eb0 con 0x7fe1580cf0a0 2020-11-16T22:35:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe150019030 con 0x7fe1580cf0a0 2020-11-16T22:35:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.201+0000 7fe1567fc700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 0x7fe134057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.205+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe15006d150 con 0x7fe1580cf0a0 2020-11-16T22:35:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.205+0000 7fe15e49b700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 0x7fe134057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.205+0000 7fe15e49b700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 0x7fe134057520 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fe148009eb0 tx=0x7fe148016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.209+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe150043350 con 0x7fe1580cf0a0 2020-11-16T22:35:11.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.361+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe158066280 con 0x7fe1580cf0a0 2020-11-16T22:35:11.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.361+0000 7fe1567fc700 1 -- 172.21.15.181:0/1764714799 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe150046970 con 0x7fe1580cf0a0 2020-11-16T22:35:11.350 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:35:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 msgr2=0x7fe134057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 0x7fe134057520 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fe148009eb0 tx=0x7fe148016000).stop 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 msgr2=0x7fe158145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe15000c740 tx=0x7fe150007b30).stop 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 shutdown_connections 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe134055060 0x7fe134057520 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580cf0a0 0x7fe158145c60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1580612a0 0x7fe1580b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 --2- 172.21.15.181:0/1764714799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1580ce750 0x7fe158145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 >> 172.21.15.181:0/1764714799 conn(0x7fe1580cf890 msgr2=0x7fe15805bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 shutdown_connections 2020-11-16T22:35:11.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.365+0000 7fe1606ff700 1 -- 172.21.15.181:0/1764714799 wait complete. 2020-11-16T22:35:11.364 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:35:11.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 Processor -- start 2020-11-16T22:35:11.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 -- start start 2020-11-16T22:35:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb4061b50 con 0x7f4fb41492c0 2020-11-16T22:35:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb40618b0 con 0x7f4fb40816e0 2020-11-16T22:35:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb8d0a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40cb930 con 0x7f4fb4145a30 2020-11-16T22:35:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44118/0 (socket says 172.21.15.181:44118) 2020-11-16T22:35:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 -- 172.21.15.181:0/513327748 learned_addr learned my addr 172.21.15.181:0/513327748 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4fb40816e0 0x7f4fb40cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50584/0 (socket says 172.21.15.181:50584) 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb27fc700 1 -- 172.21.15.181:0/513327748 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909634650 0 0) 0x7f4fb40618b0 con 0x7f4fb40816e0 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 -- 172.21.15.181:0/513327748 >> v1:172.21.15.181:6789/0 conn(0x7f4fb40816e0 legacy=0x7f4fb40cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 -- 172.21.15.181:0/513327748 >> v1:172.21.15.175:6789/0 conn(0x7f4fb41492c0 legacy=0x7f4fb4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 -- 172.21.15.181:0/513327748 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40cbbd0 con 0x7f4fb4145a30 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/513327748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f4f9c00ba20 tx=0x7f4f9c006eb0).ready entity=mon.2 client_cookie=cbf9d3cfcd9dca32 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb27fc700 1 -- 172.21.15.181:0/513327748 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0098d0 con 0x7f4fb4145a30 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb27fc700 1 -- 172.21.15.181:0/513327748 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c009a30 con 0x7f4fb4145a30 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.509+0000 7f4fb27fc700 1 -- 172.21.15.181:0/513327748 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c009db0 con 0x7f4fb4145a30 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/513327748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 msgr2=0x7f4fb4145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/513327748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f4f9c00ba20 tx=0x7f4f9c006eb0).stop 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/513327748 shutdown_connections 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/513327748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb4145e10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/513327748 >> 172.21.15.181:0/513327748 conn(0x7f4fb40d68c0 msgr2=0x7f4fb40d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/513327748 shutdown_connections 2020-11-16T22:35:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/513327748 wait complete. 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 Processor -- start 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- start start 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb412e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb41492c0 0x7f4fb412ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4173cb0 con 0x7f4fb41492c0 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb416fcb0 con 0x7f4fb40816e0 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb416ff50 con 0x7f4fb4145a30 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36282/0 (socket says 172.21.15.181:36282) 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 -- 172.21.15.181:0/1901723241 learned_addr learned my addr 172.21.15.181:0/1901723241 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb412e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb3fff700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb41492c0 0x7f4fb412ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 -- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 msgr2=0x7f4fb412e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb412e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 -- 172.21.15.181:0/1901723241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb41492c0 msgr2=0x7f4fb412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb41492c0 0x7f4fb412ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 -- 172.21.15.181:0/1901723241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb41701f0 con 0x7f4fb40816e0 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb37fe700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f4fb40d0f10 tx=0x7f4f9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011dd0 con 0x7f4fb40816e0 2020-11-16T22:35:11.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c01d450 con 0x7f4fb40816e0 2020-11-16T22:35:11.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb4170490 con 0x7f4fb40816e0 2020-11-16T22:35:11.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c01d810 con 0x7f4fb40816e0 2020-11-16T22:35:11.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.513+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb416f9a0 con 0x7f4fb40816e0 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb4170650 con 0x7f4fb40816e0 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4f9c00f070 con 0x7f4fb40816e0 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fab7fe700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 0x7f4f90057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f9c06fc10 con 0x7f4fb40816e0 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 0x7f4f90057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.517+0000 7f4fb2ffd700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 0x7f4f90057550 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f4fb415eeb0 tx=0x7f4fac007f10).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.525+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4f9c045e70 con 0x7f4fb40816e0 2020-11-16T22:35:11.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.677+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4fb4044f60 con 0x7f4fb40816e0 2020-11-16T22:35:11.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.677+0000 7f4fab7fe700 1 -- 172.21.15.181:0/1901723241 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f9c049490 con 0x7f4fb40816e0 2020-11-16T22:35:11.666 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 msgr2=0x7f4f90057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 0x7f4f90057550 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f4fb415eeb0 tx=0x7f4fac007f10).stop 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 msgr2=0x7f4fb415d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f4fb40d0f10 tx=0x7f4f9c015000).stop 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 shutdown_connections 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4f90055090 0x7f4f90057550 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb41492c0 0x7f4fb412ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb40816e0 0x7f4fb415d4b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 --2- 172.21.15.181:0/1901723241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb4145a30 0x7f4fb412e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 >> 172.21.15.181:0/1901723241 conn(0x7f4fb40d68c0 msgr2=0x7f4fb4080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 shutdown_connections 2020-11-16T22:35:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.681+0000 7f4fb8d0a700 1 -- 172.21.15.181:0/1901723241 wait complete. 2020-11-16T22:35:11.686 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:35:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.825+0000 7fc51207f700 1 Processor -- start 2020-11-16T22:35:11.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.825+0000 7fc51207f700 1 -- start start 2020-11-16T22:35:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc50c15f5b0 0x7fc50c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c160e40 con 0x7fc50c0615f0 2020-11-16T22:35:11.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c1609a0 con 0x7fc50c15f5b0 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c160ba0 con 0x7fc50c0d1830 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44136/0 (socket says 172.21.15.181:44136) 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 -- 172.21.15.181:0/2247654328 learned_addr learned my addr 172.21.15.181:0/2247654328 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50b7fe700 1 --2- 172.21.15.181:0/2247654328 >> v2:172.21.15.175:3300/0 conn(0x7fc50c15f5b0 0x7fc50c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 -- 172.21.15.181:0/2247654328 >> v2:172.21.15.175:3300/0 conn(0x7fc50c15f5b0 msgr2=0x7fc50c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:11.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 --2- 172.21.15.181:0/2247654328 >> v2:172.21.15.175:3300/0 conn(0x7fc50c15f5b0 0x7fc50c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 -- 172.21.15.181:0/2247654328 >> v1:172.21.15.175:6789/0 conn(0x7fc50c0615f0 legacy=0x7fc50c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:11.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 -- 172.21.15.181:0/2247654328 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c060970 con 0x7fc50c0d1830 2020-11-16T22:35:11.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50bfff700 1 --2- 172.21.15.181:0/2247654328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc504009d50 tx=0x7fc504009290).ready entity=mon.2 client_cookie=e17d93ab70e0e9dc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50a7fc700 1 -- 172.21.15.181:0/2247654328 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400bc40 con 0x7fc50c0d1830 2020-11-16T22:35:11.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50a7fc700 1 -- 172.21.15.181:0/2247654328 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50400bda0 con 0x7fc50c0d1830 2020-11-16T22:35:11.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc50a7fc700 1 -- 172.21.15.181:0/2247654328 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504011610 con 0x7fc50c0d1830 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- 172.21.15.181:0/2247654328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 msgr2=0x7fc50c0629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 --2- 172.21.15.181:0/2247654328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fc504009d50 tx=0x7fc504009290).stop 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- 172.21.15.181:0/2247654328 shutdown_connections 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 --2- 172.21.15.181:0/2247654328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0d1830 0x7fc50c0629b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 --2- 172.21.15.181:0/2247654328 >> v2:172.21.15.175:3300/0 conn(0x7fc50c15f5b0 0x7fc50c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- 172.21.15.181:0/2247654328 >> 172.21.15.181:0/2247654328 conn(0x7fc50c0d69a0 msgr2=0x7fc50c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- 172.21.15.181:0/2247654328 shutdown_connections 2020-11-16T22:35:11.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- 172.21.15.181:0/2247654328 wait complete. 2020-11-16T22:35:11.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 Processor -- start 2020-11-16T22:35:11.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.829+0000 7fc51207f700 1 -- start start 2020-11-16T22:35:11.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0d1830 0x7fc50c081660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50c15f5b0 0x7fc50c1632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c1726a0 con 0x7fc50c0d1830 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c173510 con 0x7fc50c15f5b0 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c164020 con 0x7fc50c0615f0 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50c15f5b0 0x7fc50c1632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44138/0 (socket says 172.21.15.181:44138) 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 -- 172.21.15.181:0/2963188891 learned_addr learned my addr 172.21.15.181:0/2963188891 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50affd700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0d1830 0x7fc50c081660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 -- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50c15f5b0 msgr2=0x7fc50c1632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50c15f5b0 0x7fc50c1632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 -- 172.21.15.181:0/2963188891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0d1830 msgr2=0x7fc50c081660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0d1830 0x7fc50c081660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 -- 172.21.15.181:0/2963188891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c172890 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc50b7fe700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc4fc006770 tx=0x7fc4fc008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00ded0 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc009790 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c172b30 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc009b10 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1637c0 con 0x7fc50c0615f0 2020-11-16T22:35:11.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c069eb0 con 0x7fc50c0615f0 2020-11-16T22:35:11.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.833+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc4fc013480 con 0x7fc50c0615f0 2020-11-16T22:35:11.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.837+0000 7fc4eb7fe700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 0x7fc4e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:11.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.837+0000 7fc50affd700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 0x7fc4e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:11.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.837+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4fc06e3b0 con 0x7fc50c0615f0 2020-11-16T22:35:11.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.837+0000 7fc50affd700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 0x7fc4e4057520 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc50c061a50 tx=0x7fc500007b70).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:11.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:11.845+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc4fc0445b0 con 0x7fc50c0615f0 2020-11-16T22:35:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc50c0600f0 con 0x7fc50c0615f0 2020-11-16T22:35:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc4eb7fe700 1 -- 172.21.15.181:0/2963188891 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc4fc047bd0 con 0x7fc50c0615f0 2020-11-16T22:35:11.988 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 msgr2=0x7fc4e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 0x7fc4e4057520 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fc50c061a50 tx=0x7fc500007b70).stop 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 msgr2=0x7fc50c081170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc4fc006770 tx=0x7fc4fc008040).stop 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 shutdown_connections 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4e4055060 0x7fc4e4057520 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50c15f5b0 0x7fc50c1632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc50c0615f0 0x7fc50c081170 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 --2- 172.21.15.181:0/2963188891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc50c0d1830 0x7fc50c081660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 >> 172.21.15.181:0/2963188891 conn(0x7fc50c0d69a0 msgr2=0x7fc50c061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 shutdown_connections 2020-11-16T22:35:11.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.001+0000 7fc51207f700 1 -- 172.21.15.181:0/2963188891 wait complete. 2020-11-16T22:35:12.011 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:35:12.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.149+0000 7f7ece8fe700 1 Processor -- start 2020-11-16T22:35:12.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.149+0000 7f7ece8fe700 1 -- start start 2020-11-16T22:35:12.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 0x7f7ec80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ec80615f0 0x7f7ec80d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec8160e40 con 0x7f7ec80622a0 2020-11-16T22:35:12.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec81609a0 con 0x7f7ec80615f0 2020-11-16T22:35:12.140 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8160ba0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 0x7f7ec80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ecce9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7ec80622a0 0x7f7ec8062720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50612/0 (socket says 172.21.15.181:50612) 2020-11-16T22:35:12.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ec80615f0 0x7f7ec80d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ecce9b700 1 -- 172.21.15.181:0/1196205046 learned_addr learned my addr 172.21.15.181:0/1196205046 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:12.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 -- 172.21.15.181:0/1196205046 >> v2:172.21.15.181:3300/0 conn(0x7f7ec80615f0 msgr2=0x7f7ec80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:12.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 --2- 172.21.15.181:0/1196205046 >> v2:172.21.15.181:3300/0 conn(0x7f7ec80615f0 0x7f7ec80d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 -- 172.21.15.181:0/1196205046 >> v1:172.21.15.181:6789/0 conn(0x7f7ec80622a0 legacy=0x7f7ec8062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 -- 172.21.15.181:0/1196205046 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8060970 con 0x7f7ec815f5b0 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec7fff700 1 --2- 172.21.15.181:0/1196205046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 0x7f7ec80d1340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f7eb000b990 tx=0x7f7eb0006d90).ready entity=mon.2 client_cookie=a35aa5be9b09eb8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec6ffd700 1 -- 172.21.15.181:0/1196205046 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0006030 con 0x7f7ec815f5b0 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec6ffd700 1 -- 172.21.15.181:0/1196205046 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0006190 con 0x7f7ec815f5b0 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ec6ffd700 1 -- 172.21.15.181:0/1196205046 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0006510 con 0x7f7ec815f5b0 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1196205046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 msgr2=0x7f7ec80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1196205046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 0x7f7ec80d1340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f7eb000b990 tx=0x7f7eb0006d90).stop 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1196205046 shutdown_connections 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1196205046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec815f5b0 0x7f7ec80d1340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1196205046 >> v2:172.21.15.181:3300/0 conn(0x7f7ec80615f0 0x7f7ec80d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1196205046 >> 172.21.15.181:0/1196205046 conn(0x7f7ec80d69a0 msgr2=0x7f7ec80d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1196205046 shutdown_connections 2020-11-16T22:35:12.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.153+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1196205046 wait complete. 2020-11-16T22:35:12.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 Processor -- start 2020-11-16T22:35:12.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- start start 2020-11-16T22:35:12.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 0x7f7ec80bae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec80622a0 0x7f7ec814b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8162c20 con 0x7f7ec80622a0 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80d3640 con 0x7f7ec815f5b0 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80bb680 con 0x7f7ec80615f0 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ec7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 0x7f7ec80bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36314/0 (socket says 172.21.15.181:36314) 2020-11-16T22:35:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ec7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 0x7f7ec80bae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44152/0 (socket says 172.21.15.181:44152) 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 -- 172.21.15.181:0/1418461512 learned_addr learned my addr 172.21.15.181:0/1418461512 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ec77fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec80622a0 0x7f7ec814b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 -- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 msgr2=0x7f7ec80bae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 0x7f7ec80bae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 -- 172.21.15.181:0/1418461512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec80622a0 msgr2=0x7f7ec814b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec80622a0 0x7f7ec814b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 -- 172.21.15.181:0/1418461512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80bb920 con 0x7f7ec815f5b0 2020-11-16T22:35:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ecce9b700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f7ec0005920 tx=0x7f7ec0007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0005b80 con 0x7f7ec815f5b0 2020-11-16T22:35:12.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0005ce0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec80bbbc0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007340 con 0x7f7ec815f5b0 2020-11-16T22:35:12.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.157+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec80bb320 con 0x7f7ec815f5b0 2020-11-16T22:35:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80bc1e0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7ec0010030 con 0x7f7ec815f5b0 2020-11-16T22:35:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ebffff700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 0x7f7ea4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ec7fff700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 0x7f7ea4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7ec006dc40 con 0x7f7ec815f5b0 2020-11-16T22:35:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.161+0000 7f7ec7fff700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 0x7f7ea4057520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f7eb000bf60 tx=0x7f7eb0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.165+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7ec0043ea0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ec8069eb0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ebffff700 1 -- 172.21.15.181:0/1418461512 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7ec00474c0 con 0x7f7ec815f5b0 2020-11-16T22:35:12.308 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 msgr2=0x7f7ea4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 0x7f7ea4057520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f7eb000bf60 tx=0x7f7eb0015000).stop 2020-11-16T22:35:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 msgr2=0x7f7ec814b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f7ec0005920 tx=0x7f7ec0007f20).stop 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 shutdown_connections 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7ea4055060 0x7f7ea4057520 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ec815f5b0 0x7f7ec814b7c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ec80615f0 0x7f7ec80bae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 --2- 172.21.15.181:0/1418461512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec80622a0 0x7f7ec814b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 >> 172.21.15.181:0/1418461512 conn(0x7f7ec80d69a0 msgr2=0x7f7ec8062720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 shutdown_connections 2020-11-16T22:35:12.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.321+0000 7f7ece8fe700 1 -- 172.21.15.181:0/1418461512 wait complete. 2020-11-16T22:35:12.322 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:12.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.457+0000 7feff85ce700 1 Processor -- start 2020-11-16T22:35:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.457+0000 7feff85ce700 1 -- start start 2020-11-16T22:35:12.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 0x7feff00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff0066480 con 0x7feff015d200 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff0066ee0 con 0x7feff015d5e0 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff0067180 con 0x7feff00667b0 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44164/0 (socket says 172.21.15.181:44164) 2020-11-16T22:35:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 -- 172.21.15.181:0/2552426595 learned_addr learned my addr 172.21.15.181:0/2552426595 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff5b69700 1 --2- 172.21.15.181:0/2552426595 >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 0x7feff00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 -- 172.21.15.181:0/2552426595 >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 msgr2=0x7feff00ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 --2- 172.21.15.181:0/2552426595 >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 0x7feff00ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 -- 172.21.15.181:0/2552426595 >> v1:172.21.15.175:6789/0 conn(0x7feff015d200 legacy=0x7feff00cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 -- 172.21.15.181:0/2552426595 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0067420 con 0x7feff00667b0 2020-11-16T22:35:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff5b69700 1 --2- 172.21.15.181:0/2552426595 >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 0x7feff00ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff6b6b700 1 --2- 172.21.15.181:0/2552426595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fefe8009d50 tx=0x7fefe8009290).ready entity=mon.2 client_cookie=6d561ccf2ba3f93b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff5368700 1 -- 172.21.15.181:0/2552426595 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800bc80 con 0x7feff00667b0 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff5368700 1 -- 172.21.15.181:0/2552426595 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe800bde0 con 0x7feff00667b0 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff5368700 1 -- 172.21.15.181:0/2552426595 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011610 con 0x7feff00667b0 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- 172.21.15.181:0/2552426595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 msgr2=0x7feff00cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 --2- 172.21.15.181:0/2552426595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fefe8009d50 tx=0x7fefe8009290).stop 2020-11-16T22:35:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- 172.21.15.181:0/2552426595 shutdown_connections 2020-11-16T22:35:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 --2- 172.21.15.181:0/2552426595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff00667b0 0x7feff00cf920 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 --2- 172.21.15.181:0/2552426595 >> v2:172.21.15.175:3300/0 conn(0x7feff015d5e0 0x7feff00ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- 172.21.15.181:0/2552426595 >> 172.21.15.181:0/2552426595 conn(0x7feff00d4790 msgr2=0x7feff00d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- 172.21.15.181:0/2552426595 shutdown_connections 2020-11-16T22:35:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.461+0000 7feff85ce700 1 -- 172.21.15.181:0/2552426595 wait complete. 2020-11-16T22:35:12.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 Processor -- start 2020-11-16T22:35:12.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 -- start start 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff00667b0 0x7feff016fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff015d200 0x7feff00b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00ba680 con 0x7feff015d200 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00d0830 con 0x7feff00667b0 2020-11-16T22:35:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00d0ad0 con 0x7feff015d5e0 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff636a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff00667b0 0x7feff016fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44170/0 (socket says 172.21.15.181:44170) 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 -- 172.21.15.181:0/1394369745 learned_addr learned my addr 172.21.15.181:0/1394369745 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff5b69700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff015d200 0x7feff00b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 -- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff00667b0 msgr2=0x7feff016fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff00667b0 0x7feff016fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 -- 172.21.15.181:0/1394369745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff015d200 msgr2=0x7feff00b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff015d200 0x7feff00b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 -- 172.21.15.181:0/1394369745 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00d0d70 con 0x7feff015d5e0 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff6b6b700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fefe80106d0 tx=0x7fefe8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800bc80 con 0x7feff015d5e0 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8009e90 con 0x7feff015d5e0 2020-11-16T22:35:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00d1010 con 0x7feff015d5e0 2020-11-16T22:35:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.465+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8010d00 con 0x7feff015d5e0 2020-11-16T22:35:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff0171a00 con 0x7feff015d5e0 2020-11-16T22:35:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff0170610 con 0x7feff015d5e0 2020-11-16T22:35:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fefe8017070 con 0x7feff015d5e0 2020-11-16T22:35:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7fefde7fc700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 0x7fefc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fefe806ffe0 con 0x7feff015d5e0 2020-11-16T22:35:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7feff636a700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 0x7fefc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.469+0000 7feff636a700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 0x7fefc8057520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fefec003300 tx=0x7fefec007b90).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.477+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fefe8046130 con 0x7feff015d5e0 2020-11-16T22:35:12.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feff00ba880 con 0x7feff015d5e0 2020-11-16T22:35:12.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7fefde7fc700 1 -- 172.21.15.181:0/1394369745 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4671 (secure 0 0 0) 0x7fefe8015070 con 0x7feff015d5e0 2020-11-16T22:35:12.664 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:12.665 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6835/206785056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":206785056},{"type":"v1","addr":"172.21.15.181:6835","nonce":206785056}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":6797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2019273904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2019273904},{"type":"v1","addr":"172.21.15.181:6841","nonce":2019273904}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7281,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3311990354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3311990354},{"type":"v1","addr":"172.21.15.175:6834","nonce":3311990354}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7284,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6835/3501106712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6833","nonce":3501106712},{"type":"v1","addr":"172.21.15.175:6835","nonce":3501106712}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":92},{"gid":7299,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6838/1115026546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1115026546},{"type":"v1","addr":"172.21.15.175:6838","nonce":1115026546}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91},{"gid":7311,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6841/3851083189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6840","nonce":3851083189},{"type":"v1","addr":"172.21.15.175:6841","nonce":3851083189}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":91}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:33:55.623227+0000","modified":"2020-11-16T22:34:54.919305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7405,"mds_1":7399,"mds_2":7396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7396":{"gid":7396,"name":"i","rank":2,"incarnation":100,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6839/3389989681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6837","nonce":3389989681},{"type":"v1","addr":"172.21.15.175:6839","nonce":3389989681}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7399":{"gid":7399,"name":"b","rank":1,"incarnation":98,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6837/1406664890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1406664890},{"type":"v1","addr":"172.21.15.181:6837","nonce":1406664890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7405":{"gid":7405,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":18,"addr":"172.21.15.181:6839/1296416553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1296416553},{"type":"v1","addr":"172.21.15.181:6839","nonce":1296416553}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:35:12.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 msgr2=0x7fefc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 0x7fefc8057520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fefec003300 tx=0x7fefec007b90).stop 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 msgr2=0x7feff00b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fefe80106d0 tx=0x7fefe8016000).stop 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 shutdown_connections 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fefc8055060 0x7fefc8057520 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feff015d5e0 0x7feff00b8de0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feff00667b0 0x7feff016fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 --2- 172.21.15.181:0/1394369745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff015d200 0x7feff00b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 >> 172.21.15.181:0/1394369745 conn(0x7feff00d4790 msgr2=0x7feff00cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 shutdown_connections 2020-11-16T22:35:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.677+0000 7feff85ce700 1 -- 172.21.15.181:0/1394369745 wait complete. 2020-11-16T22:35:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 107 2020-11-16T22:35:12.680 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 Processor -- start 2020-11-16T22:35:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 -- start start 2020-11-16T22:35:12.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00c00cac00 0x7f00c00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c0162500 con 0x7f00c0160780 2020-11-16T22:35:12.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c01627a0 con 0x7f00c0066c40 2020-11-16T22:35:12.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c5a6c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c0162a40 con 0x7f00c00cac00 2020-11-16T22:35:12.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c526b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f00c0066c40 0x7f00c00c9790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50638/0 (socket says 172.21.15.181:50638) 2020-11-16T22:35:12.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c526b700 1 -- 172.21.15.181:0/675089672 learned_addr learned my addr 172.21.15.181:0/675089672 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c4a6a700 1 --2- 172.21.15.181:0/675089672 >> v2:172.21.15.175:3300/0 conn(0x7f00c00cac00 0x7f00c00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878120605 0 0) 0x7f00c01627a0 con 0x7f00c0066c40 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ac003240 con 0x7f00c0066c40 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778686870 0 0) 0x7f00c0162500 con 0x7f00c0160780 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c01627a0 con 0x7f00c0160780 2020-11-16T22:35:12.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c4a6a700 1 -- 172.21.15.181:0/675089672 >> v1:172.21.15.181:6789/0 conn(0x7f00c0066c40 legacy=0x7f00c00c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1829173890 0 0) 0x7f00ac003240 con 0x7f00c0066c40 2020-11-16T22:35:12.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c4a6a700 1 -- 172.21.15.181:0/675089672 >> v1:172.21.15.175:6789/0 conn(0x7f00c0160780 legacy=0x7f00c0066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c4a6a700 1 -- 172.21.15.181:0/675089672 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0162ce0 con 0x7f00c00cac00 2020-11-16T22:35:12.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00c4a6a700 1 --2- 172.21.15.181:0/675089672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c00cafe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f00b000b920 tx=0x7f00b0006df0).ready entity=mon.0 client_cookie=4e72af475cd9983c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.817+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8001860 con 0x7f00c00cac00 2020-11-16T22:35:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b0011400 con 0x7f00c00cac00 2020-11-16T22:35:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00bf7fe700 1 -- 172.21.15.181:0/675089672 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0011780 con 0x7f00c00cac00 2020-11-16T22:35:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- 172.21.15.181:0/675089672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 msgr2=0x7f00c00cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/675089672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c00cafe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f00b000b920 tx=0x7f00b0006df0).stop 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- 172.21.15.181:0/675089672 shutdown_connections 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/675089672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c00cafe0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- 172.21.15.181:0/675089672 >> 172.21.15.181:0/675089672 conn(0x7f00c00d4590 msgr2=0x7f00c00d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- 172.21.15.181:0/675089672 shutdown_connections 2020-11-16T22:35:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- 172.21.15.181:0/675089672 wait complete. 2020-11-16T22:35:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 Processor -- start 2020-11-16T22:35:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.821+0000 7f00c5a6c700 1 -- start start 2020-11-16T22:35:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 0x7f00c01732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c0140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01737d0 con 0x7f00c00cac00 2020-11-16T22:35:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0165e30 con 0x7f00c0160780 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01660d0 con 0x7f00c0066c40 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c4a6a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 0x7f00c01732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36342/0 (socket says 172.21.15.181:36342) 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c4a6a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 0x7f00c01732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44182/0 (socket says 172.21.15.181:44182) 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 -- 172.21.15.181:0/1263724060 learned_addr learned my addr 172.21.15.181:0/1263724060 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:12.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00bffff700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c0140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 -- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 msgr2=0x7f00c01732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 0x7f00c01732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 -- 172.21.15.181:0/1263724060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 msgr2=0x7f00c0140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c0140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 -- 172.21.15.181:0/1263724060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0166370 con 0x7f00c0160780 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c526b700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f00b800ce10 tx=0x7f00b8001d70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800ce40 con 0x7f00c0160780 2020-11-16T22:35:12.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b800ab10 con 0x7f00c0160780 2020-11-16T22:35:12.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c015c0d0 con 0x7f00c0160780 2020-11-16T22:35:12.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8007170 con 0x7f00c0160780 2020-11-16T22:35:12.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c015a5b0 con 0x7f00c0160780 2020-11-16T22:35:12.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.825+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c0162ec0 con 0x7f00c0160780 2020-11-16T22:35:12.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.829+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f00b801b060 con 0x7f00c0160780 2020-11-16T22:35:12.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.829+0000 7f00bcff9700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 0x7f009c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:12.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.829+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00b806dcc0 con 0x7f00c0160780 2020-11-16T22:35:12.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.829+0000 7f00c4a6a700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 0x7f009c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:12.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.829+0000 7f00c4a6a700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 0x7f009c057550 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f00b000b150 tx=0x7f00b0013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:12.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.837+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f00b8074080 con 0x7f00c0160780 2020-11-16T22:35:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.989+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00c015c260 con 0x7f00c0160780 2020-11-16T22:35:12.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.993+0000 7f00bcff9700 1 -- 172.21.15.181:0/1263724060 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f00b8019070 con 0x7f00c0160780 2020-11-16T22:35:12.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:12.981 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":97,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:33:56.627315+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:33:53.695421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:33:54.659875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 msgr2=0x7f009c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 0x7f009c057550 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f00b000b150 tx=0x7f00b0013000).stop 2020-11-16T22:35:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 msgr2=0x7f00c0140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f00b800ce10 tx=0x7f00b8001d70).stop 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 shutdown_connections 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f009c055090 0x7f009c057550 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c0160780 0x7f00c0140ff0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c0066c40 0x7f00c01732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 --2- 172.21.15.181:0/1263724060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c00cac00 0x7f00c0140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 >> 172.21.15.181:0/1263724060 conn(0x7f00c00d4590 msgr2=0x7f00c00d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:12.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 shutdown_connections 2020-11-16T22:35:12.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:12.997+0000 7f00c5a6c700 1 -- 172.21.15.181:0/1263724060 wait complete. 2020-11-16T22:35:13.002 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:35:13.003 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:35:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.137+0000 7fade6f21700 1 Processor -- start 2020-11-16T22:35:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.137+0000 7fade6f21700 1 -- start start 2020-11-16T22:35:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fade013e870 0x7fade013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade015d180 con 0x7fade0061810 2020-11-16T22:35:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade013f140 con 0x7fade015fdc0 2020-11-16T22:35:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade015d810 con 0x7fade013e870 2020-11-16T22:35:13.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fade013e870 0x7fade013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:13.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fade013e870 0x7fade013ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36346/0 (socket says 172.21.15.181:36346) 2020-11-16T22:35:13.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade571e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fade015fdc0 0x7fade0061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50652/0 (socket says 172.21.15.181:50652) 2020-11-16T22:35:13.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 -- 172.21.15.181:0/911857480 learned_addr learned my addr 172.21.15.181:0/911857480 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:13.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 -- 172.21.15.181:0/911857480 >> v1:172.21.15.181:6789/0 conn(0x7fade015fdc0 legacy=0x7fade0061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 -- 172.21.15.181:0/911857480 >> v1:172.21.15.175:6789/0 conn(0x7fade0061810 legacy=0x7fade015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 -- 172.21.15.181:0/911857480 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade015dab0 con 0x7fade013e870 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade5f1f700 1 --2- 172.21.15.181:0/911857480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade013e870 0x7fade013ec50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7faddc009d50 tx=0x7faddc009290).ready entity=mon.1 client_cookie=75b0bab374a29a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade4f1d700 1 -- 172.21.15.181:0/911857480 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00bc80 con 0x7fade013e870 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade4f1d700 1 -- 172.21.15.181:0/911857480 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00bde0 con 0x7fade013e870 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade4f1d700 1 -- 172.21.15.181:0/911857480 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc011610 con 0x7fade013e870 2020-11-16T22:35:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- 172.21.15.181:0/911857480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade013e870 msgr2=0x7fade013ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 --2- 172.21.15.181:0/911857480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade013e870 0x7fade013ec50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7faddc009d50 tx=0x7faddc009290).stop 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- 172.21.15.181:0/911857480 shutdown_connections 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 --2- 172.21.15.181:0/911857480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade013e870 0x7fade013ec50 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- 172.21.15.181:0/911857480 >> 172.21.15.181:0/911857480 conn(0x7fade00cf630 msgr2=0x7fade00cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- 172.21.15.181:0/911857480 shutdown_connections 2020-11-16T22:35:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.141+0000 7fade6f21700 1 -- 172.21.15.181:0/911857480 wait complete. 2020-11-16T22:35:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 Processor -- start 2020-11-16T22:35:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- start start 2020-11-16T22:35:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 0x7fade0138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade015fdc0 0x7fade00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade016fa70 con 0x7fade013e870 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade016fd10 con 0x7fade015fdc0 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade016ffb0 con 0x7fade0061810 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44194/0 (socket says 172.21.15.181:44194) 2020-11-16T22:35:13.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 -- 172.21.15.181:0/2466346063 learned_addr learned my addr 172.21.15.181:0/2466346063 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6720700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade015fdc0 0x7fade00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade571e700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 0x7fade0138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 -- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade015fdc0 msgr2=0x7fade00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade015fdc0 0x7fade00cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 -- 172.21.15.181:0/2466346063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 msgr2=0x7fade0138380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 0x7fade0138380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 -- 172.21.15.181:0/2466346063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0143870 con 0x7fade0061810 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade571e700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 0x7fade0138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade5f1f700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7faddc0092d0 tx=0x7faddc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc01c680 con 0x7fade0061810 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc01c7e0 con 0x7fade0061810 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0143b10 con 0x7fade0061810 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc01cb60 con 0x7fade0061810 2020-11-16T22:35:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.145+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade016f6c0 con 0x7fade0061810 2020-11-16T22:35:13.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faddc017070 con 0x7fade0061810 2020-11-16T22:35:13.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade0170170 con 0x7fade0061810 2020-11-16T22:35:13.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fadcdffb700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 0x7fadb8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:13.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faddc06fa50 con 0x7fade0061810 2020-11-16T22:35:13.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fade571e700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 0x7fadb8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:13.139 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.149+0000 7fade571e700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 0x7fadb8057520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fade00cfd00 tx=0x7fadd0007120).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:13.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.153+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faddc045d30 con 0x7fade0061810 2020-11-16T22:35:13.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.353+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fade0044f60 con 0x7fade0061810 2020-11-16T22:35:13.905 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:35:13.914+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:35:13.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.933+0000 7fadcdffb700 1 -- 172.21.15.181:0/2466346063 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) v1 ==== 161+0+0 (secure 0 0 0) 0x7faddc015070 con 0x7fade0061810 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 msgr2=0x7fadb8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 0x7fadb8057520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fade00cfd00 tx=0x7fadd0007120).stop 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 msgr2=0x7fade016f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7faddc0092d0 tx=0x7faddc016000).stop 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 shutdown_connections 2020-11-16T22:35:13.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fadb8055060 0x7fadb8057520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade015fdc0 0x7fade00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fade0061810 0x7fade016f1d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 --2- 172.21.15.181:0/2466346063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade013e870 0x7fade0138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 >> 172.21.15.181:0/2466346063 conn(0x7fade00cf630 msgr2=0x7fade00b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 shutdown_connections 2020-11-16T22:35:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:13.937+0000 7fade6f21700 1 -- 172.21.15.181:0/2466346063 wait complete. 2020-11-16T22:35:13.928 INFO:tasks.ceph.mds.i.smithi175.stderr: -898> 2020-11-16T22:34:51.065+0000 7fd16e0d0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1) n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@4},l=6914 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55923443f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.928 INFO:tasks.ceph.mds.i.smithi175.stderr: -885> 2020-11-16T22:34:51.065+0000 7fd16e0d0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/AsLsXsFsx@2},l=6914 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5592345c8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:35:13.934 INFO:tasks.ceph.mds.b.smithi181.stderr: -1034> 2020-11-16T22:34:51.062+0000 7f4984ce9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1) n(v2 rc2020-11-16T22:34:21.372304+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@4},l=6914 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x560b0b97ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:34:21.372304+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.934 INFO:tasks.ceph.mds.b.smithi181.stderr: -915> 2020-11-16T22:34:51.066+0000 7f498acf5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1) n(v1 rc2020-11-16T22:34:21.372304+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6914=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x560b0b97f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:17.896375+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.896375+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:34:21.372304+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.946 INFO:tasks.ceph.mds.d.smithi181.stderr: -1834> 2020-11-16T22:34:51.062+0000 7f57517fb700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1) n(v4 rc2020-11-16T22:34:21.372304+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6914=pAsLsXsFsx/pAsLsXsFsx@5},l=6914 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561470c0bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.888375+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:34:21.372304+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.946 INFO:tasks.ceph.mds.d.smithi181.stderr: -1736> 2020-11-16T22:34:51.066+0000 7f5757807700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1) n(v3 rc2020-11-16T22:34:21.372304+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6914=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561470c0c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:34:21.372304+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:34:17.892375+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:34:21.372304+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:35:13.947 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:35:13.948 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:35:13.955 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:35:13.967 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:35:14.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 Processor -- start 2020-11-16T22:35:14.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 -- start start 2020-11-16T22:35:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09b8164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09b8164e10 0x7f09b8062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80c8db0 con 0x7f09b8161860 2020-11-16T22:35:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80c8330 con 0x7f09b8164e10 2020-11-16T22:35:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.081+0000 7f09be6ae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c85d0 con 0x7f09b8161480 2020-11-16T22:35:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f09b8161860 0x7f09b8062740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50688/0 (socket says 172.21.15.181:50688) 2020-11-16T22:35:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bd6ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09b8164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 -- 172.21.15.181:0/3548403212 learned_addr learned my addr 172.21.15.181:0/3548403212 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bceab700 1 --2- 172.21.15.181:0/3548403212 >> v2:172.21.15.175:3300/0 conn(0x7f09b8164e10 0x7f09b8062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199733664 0 0) 0x7f09b80c8db0 con 0x7f09b8161860 2020-11-16T22:35:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0998003240 con 0x7f09b8161860 2020-11-16T22:35:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3239198073 0 0) 0x7f0998003240 con 0x7f09b8161860 2020-11-16T22:35:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 msgr2=0x7f09b8164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09b8164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 >> v2:172.21.15.175:3300/0 conn(0x7f09b8164e10 msgr2=0x7f09b8062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 --2- 172.21.15.181:0/3548403212 >> v2:172.21.15.175:3300/0 conn(0x7f09b8164e10 0x7f09b8062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80c8870 con 0x7f09b8161860 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f09a00014e0 con 0x7f09b8161860 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 >> v1:172.21.15.181:6789/0 conn(0x7f09b8161860 legacy=0x7f09b8062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0998003730 0x7f0998003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09980044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09980049d0 0x7f0998004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c8330 con 0x7f0998003730 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c85d0 con 0x7f09980049d0 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09980041d0 con 0x7f09b8161480 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09980049d0 0x7f0998004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bceab700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09980044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bd6ac700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0998003730 0x7f0998003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 -- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 msgr2=0x7f09980044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b8161480 0x7f09980044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 -- 172.21.15.181:0/3548403212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0998003730 msgr2=0x7f0998003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0998003730 0x7f0998003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 -- 172.21.15.181:0/3548403212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0998003240 con 0x7f09980049d0 2020-11-16T22:35:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bd6ac700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0998003730 0x7f0998003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09980049d0 0x7f0998004e50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f09b8161c90 tx=0x7f09b8062130).ready entity=mon.1 client_cookie=9751e0a9a3a455e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00070d0 con 0x7f09980049d0 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0007230 con 0x7f09980049d0 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09b7fff700 1 -- 172.21.15.181:0/3548403212 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00055f0 con 0x7f09980049d0 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 -- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09980049d0 msgr2=0x7f0998004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 --2- 172.21.15.181:0/3548403212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09980049d0 0x7f0998004e50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f09b8161c90 tx=0x7f09b8062130).stop 2020-11-16T22:35:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09bdead700 1 -- 172.21.15.181:0/3548403212 reap_dead start 2020-11-16T22:35:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 -- 172.21.15.181:0/3548403212 shutdown_connections 2020-11-16T22:35:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 -- 172.21.15.181:0/3548403212 >> 172.21.15.181:0/3548403212 conn(0x7f09b80d2dc0 msgr2=0x7f09b80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 -- 172.21.15.181:0/3548403212 shutdown_connections 2020-11-16T22:35:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.085+0000 7f09be6ae700 1 -- 172.21.15.181:0/3548403212 wait complete. 2020-11-16T22:35:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 Processor -- start 2020-11-16T22:35:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- start start 2020-11-16T22:35:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b8173c80 0x7f09b8174100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b80b4320 0x7f09b80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8160770 con 0x7f09b8173c80 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b3010 con 0x7f09b8062380 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8062c50 con 0x7f09b80b4320 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36396/0 (socket says 172.21.15.181:36396) 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 -- 172.21.15.181:0/507468035 learned_addr learned my addr 172.21.15.181:0/507468035 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bdead700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b80b4320 0x7f09b80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bceab700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b8173c80 0x7f09b8174100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 -- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b80b4320 msgr2=0x7f09b80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b80b4320 0x7f09b80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 -- 172.21.15.181:0/507468035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b8173c80 msgr2=0x7f09b8174100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b8173c80 0x7f09b8174100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 -- 172.21.15.181:0/507468035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80b2d70 con 0x7f09b8062380 2020-11-16T22:35:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09bd6ac700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f09b0010a80 tx=0x7f09b0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000e8a0 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b000ea00 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b8174b80 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000ed80 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.089+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b8175360 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b81751a0 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f09b0024070 con 0x7f09b8062380 2020-11-16T22:35:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09b57fa700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 0x7f0994057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:14.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09bceab700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 0x7f0994057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:14.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f09b006e5b0 con 0x7f09b8062380 2020-11-16T22:35:14.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.093+0000 7f09bceab700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 0x7f0994057520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f09a80094d0 tx=0x7f09a80091d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:14.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.097+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f09b0044710 con 0x7f09b8062380 2020-11-16T22:35:14.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.297+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09b8044f60 con 0x7f09b8062380 2020-11-16T22:35:14.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.933+0000 7f09b57fa700 1 -- 172.21.15.181:0/507468035 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09b00180a0 con 0x7f09b8062380 2020-11-16T22:35:14.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 msgr2=0x7f0994057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 0x7f0994057520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f09a80094d0 tx=0x7f09a80091d0).stop 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 msgr2=0x7f09b8062760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f09b0010a80 tx=0x7f09b0008040).stop 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 shutdown_connections 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0994055060 0x7f0994057520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09b80b4320 0x7f09b80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09b8062380 0x7f09b8062760 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 --2- 172.21.15.181:0/507468035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09b8173c80 0x7f09b8174100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:14.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 >> 172.21.15.181:0/507468035 conn(0x7f09b80d2dc0 msgr2=0x7f09b8149580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:14.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 shutdown_connections 2020-11-16T22:35:14.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:14.937+0000 7f09be6ae700 1 -- 172.21.15.181:0/507468035 wait complete. 2020-11-16T22:35:14.939 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:35:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.133+0000 7f545a95c700 1 Processor -- start 2020-11-16T22:35:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.133+0000 7f545a95c700 1 -- start start 2020-11-16T22:35:15.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a95c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5454081170 0x7f5454081550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a95c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 0x7f5454161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a95c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5454081a40 con 0x7f54540d1060 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a95c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5454142f40 con 0x7f5454081170 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a95c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540d1800 con 0x7f5454142ad0 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5459159700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f54540d1060 0x7f54541423c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50708/0 (socket says 172.21.15.181:50708) 2020-11-16T22:35:15.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5459159700 1 -- 172.21.15.181:0/185844700 learned_addr learned my addr 172.21.15.181:0/185844700 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:15.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 --2- 172.21.15.181:0/185844700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 0x7f5454161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545995a700 1 --2- 172.21.15.181:0/185844700 >> v2:172.21.15.181:3300/0 conn(0x7f5454081170 0x7f5454081550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5458958700 1 -- 172.21.15.181:0/185844700 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234900910 0 0) 0x7f5454081a40 con 0x7f54540d1060 2020-11-16T22:35:15.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5458958700 1 -- 172.21.15.181:0/185844700 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f543c003240 con 0x7f54540d1060 2020-11-16T22:35:15.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 -- 172.21.15.181:0/185844700 >> v2:172.21.15.181:3300/0 conn(0x7f5454081170 msgr2=0x7f5454081550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 --2- 172.21.15.181:0/185844700 >> v2:172.21.15.181:3300/0 conn(0x7f5454081170 0x7f5454081550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 -- 172.21.15.181:0/185844700 >> v1:172.21.15.181:6789/0 conn(0x7f54540d1060 legacy=0x7f54541423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 -- 172.21.15.181:0/185844700 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540d1aa0 con 0x7f5454142ad0 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f545a15b700 1 --2- 172.21.15.181:0/185844700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 0x7f5454161630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f544c009c90 tx=0x7f544c0091d0).ready entity=mon.2 client_cookie=b2ea7f6536fc888e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5458958700 1 -- 172.21.15.181:0/185844700 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00b9c0 con 0x7f5454142ad0 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5458958700 1 -- 172.21.15.181:0/185844700 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c00bb20 con 0x7f5454142ad0 2020-11-16T22:35:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.137+0000 7f5458958700 1 -- 172.21.15.181:0/185844700 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00bea0 con 0x7f5454142ad0 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- 172.21.15.181:0/185844700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 msgr2=0x7f5454161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 --2- 172.21.15.181:0/185844700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 0x7f5454161630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f544c009c90 tx=0x7f544c0091d0).stop 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- 172.21.15.181:0/185844700 shutdown_connections 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 --2- 172.21.15.181:0/185844700 >> v2:172.21.15.181:3300/0 conn(0x7f5454081170 0x7f5454081550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 --2- 172.21.15.181:0/185844700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5454142ad0 0x7f5454161630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- 172.21.15.181:0/185844700 >> 172.21.15.181:0/185844700 conn(0x7f54540682f0 msgr2=0x7f54540d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:15.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- 172.21.15.181:0/185844700 shutdown_connections 2020-11-16T22:35:15.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- 172.21.15.181:0/185844700 wait complete. 2020-11-16T22:35:15.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 Processor -- start 2020-11-16T22:35:15.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.141+0000 7f545a95c700 1 -- start start 2020-11-16T22:35:15.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5454081170 0x7f54540d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 0x7f5454160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540cf630 con 0x7f5454081170 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5454141520 con 0x7f5454142ad0 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54541417c0 con 0x7f54540d1060 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a15b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 0x7f5454160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44254/0 (socket says 172.21.15.181:44254) 2020-11-16T22:35:15.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 -- 172.21.15.181:0/1382362469 learned_addr learned my addr 172.21.15.181:0/1382362469 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:15.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545995a700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5454081170 0x7f54540d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 -- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 msgr2=0x7f5454160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 0x7f5454160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 -- 172.21.15.181:0/1382362469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5454081170 msgr2=0x7f54540d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5454081170 0x7f54540d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 -- 172.21.15.181:0/1382362469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540cf860 con 0x7f54540d1060 2020-11-16T22:35:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a15b700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 0x7f5454160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5459159700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f544400c6c0 tx=0x7f5444007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54440072e0 con 0x7f54540d1060 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444007440 con 0x7f54540d1060 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540cfb00 con 0x7f54540d1060 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444005820 con 0x7f54540d1060 2020-11-16T22:35:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5454170730 con 0x7f54540d1060 2020-11-16T22:35:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5444010070 con 0x7f54540d1060 2020-11-16T22:35:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 0x7f542c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f545995a700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 0x7f542c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.145+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f544406d490 con 0x7f54540d1060 2020-11-16T22:35:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.149+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54540d1c80 con 0x7f54540d1060 2020-11-16T22:35:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.149+0000 7f545995a700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 0x7f542c057520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f54540cd6f0 tx=0x7f5450006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.153+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5444043520 con 0x7f54540d1060 2020-11-16T22:35:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.305+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5454170980 con 0x7f54540d1060 2020-11-16T22:35:15.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.937+0000 7f5441ffb700 1 -- 172.21.15.181:0/1382362469 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5444046b40 con 0x7f54540d1060 2020-11-16T22:35:15.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 msgr2=0x7f542c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 0x7f542c057520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f54540cd6f0 tx=0x7f5450006df0).stop 2020-11-16T22:35:15.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 msgr2=0x7f54540d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:15.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f544400c6c0 tx=0x7f5444007ac0).stop 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 shutdown_connections 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f542c055060 0x7f542c057520 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5454142ad0 0x7f5454160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5454081170 0x7f54540d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 --2- 172.21.15.181:0/1382362469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54540d1060 0x7f54540d0730 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.941+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 >> 172.21.15.181:0/1382362469 conn(0x7f54540682f0 msgr2=0x7f5454142190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:15.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.945+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 shutdown_connections 2020-11-16T22:35:15.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:15.945+0000 7f545a95c700 1 -- 172.21.15.181:0/1382362469 wait complete. 2020-11-16T22:35:15.934 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:35:15.948 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:35:16.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.089+0000 7f32211d2700 1 Processor -- start 2020-11-16T22:35:16.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.089+0000 7f32211d2700 1 -- start start 2020-11-16T22:35:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 0x7f321c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c160290 con 0x7f321c13e760 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c13f300 con 0x7f321c15f9c0 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c15e600 con 0x7f321c0ce5e0 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44262/0 (socket says 172.21.15.181:44262) 2020-11-16T22:35:16.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 -- 172.21.15.181:0/3495961622 learned_addr learned my addr 172.21.15.181:0/3495961622 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:16.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321bfff700 1 --2- 172.21.15.181:0/3495961622 >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 0x7f321c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 -- 172.21.15.181:0/3495961622 >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 msgr2=0x7f321c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 --2- 172.21.15.181:0/3495961622 >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 0x7f321c15fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 -- 172.21.15.181:0/3495961622 >> v1:172.21.15.175:6789/0 conn(0x7f321c13e760 legacy=0x7f321c13ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 -- 172.21.15.181:0/3495961622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c15e8a0 con 0x7f321c0ce5e0 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321bfff700 1 --2- 172.21.15.181:0/3495961622 >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 0x7f321c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321b7fe700 1 --2- 172.21.15.181:0/3495961622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3210006fd0 tx=0x7f3210006eb0).ready entity=mon.2 client_cookie=c3b782047a973c0c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321affd700 1 -- 172.21.15.181:0/3495961622 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210009810 con 0x7f321c0ce5e0 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321affd700 1 -- 172.21.15.181:0/3495961622 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3210009970 con 0x7f321c0ce5e0 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f321affd700 1 -- 172.21.15.181:0/3495961622 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210009cf0 con 0x7f321c0ce5e0 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- 172.21.15.181:0/3495961622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 msgr2=0x7f321c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 --2- 172.21.15.181:0/3495961622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f3210006fd0 tx=0x7f3210006eb0).stop 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- 172.21.15.181:0/3495961622 shutdown_connections 2020-11-16T22:35:16.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 --2- 172.21.15.181:0/3495961622 >> v2:172.21.15.181:3300/0 conn(0x7f321c15f9c0 0x7f321c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 --2- 172.21.15.181:0/3495961622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c0ce5e0 0x7f321c0cea60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- 172.21.15.181:0/3495961622 >> 172.21.15.181:0/3495961622 conn(0x7f321c0d35b0 msgr2=0x7f321c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:16.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- 172.21.15.181:0/3495961622 shutdown_connections 2020-11-16T22:35:16.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- 172.21.15.181:0/3495961622 wait complete. 2020-11-16T22:35:16.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 Processor -- start 2020-11-16T22:35:16.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.093+0000 7f32211d2700 1 -- start start 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 0x7f321c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c15f9c0 0x7f321c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 0x7f321c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c13b810 con 0x7f321c15f9c0 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c160d80 con 0x7f321c15b940 2020-11-16T22:35:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c161020 con 0x7f321c13e760 2020-11-16T22:35:16.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f321bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 0x7f321c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 0x7f321c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f321bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 0x7f321c15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44270/0 (socket says 172.21.15.181:44270) 2020-11-16T22:35:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f321bfff700 1 -- 172.21.15.181:0/1551633242 learned_addr learned my addr 172.21.15.181:0/1551633242 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f321b7fe700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c15f9c0 0x7f321c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 -- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 msgr2=0x7f321c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 0x7f321c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 -- 172.21.15.181:0/1551633242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c15f9c0 msgr2=0x7f321c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c15f9c0 0x7f321c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 -- 172.21.15.181:0/1551633242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c1612c0 con 0x7f321c15b940 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32209d1700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 0x7f321c0cdf60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f321400ca60 tx=0x7f3214007de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400ce30 con 0x7f321c15b940 2020-11-16T22:35:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321400a7c0 con 0x7f321c15b940 2020-11-16T22:35:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400ab80 con 0x7f321c15b940 2020-11-16T22:35:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c161560 con 0x7f321c15b940 2020-11-16T22:35:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.097+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c16c570 con 0x7f321c15b940 2020-11-16T22:35:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c044f60 con 0x7f321c15b940 2020-11-16T22:35:16.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3214019030 con 0x7f321c15b940 2020-11-16T22:35:16.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f31fbfff700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 0x7f31f4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f3214003030 con 0x7f321c15b940 2020-11-16T22:35:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f321bfff700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 0x7f31f4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.101+0000 7f321bfff700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 0x7f31f4057550 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f320c00ba80 tx=0x7f320c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.105+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3214043750 con 0x7f321c15b940 2020-11-16T22:35:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.257+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f321c0659e0 con 0x7f321c15b940 2020-11-16T22:35:16.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.945+0000 7f31fbfff700 1 -- 172.21.15.181:0/1551633242 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3214017020 con 0x7f321c15b940 2020-11-16T22:35:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 msgr2=0x7f31f4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 0x7f31f4057550 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f320c00ba80 tx=0x7f320c006df0).stop 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 msgr2=0x7f321c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 0x7f321c0cdf60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f321400ca60 tx=0x7f3214007de0).stop 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 shutdown_connections 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f31f4055090 0x7f31f4057550 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c15b940 0x7f321c0cdf60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c13e760 0x7f321c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 --2- 172.21.15.181:0/1551633242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f321c15f9c0 0x7f321c0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 >> 172.21.15.181:0/1551633242 conn(0x7f321c0d35b0 msgr2=0x7f321c0d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 shutdown_connections 2020-11-16T22:35:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:16.953+0000 7f32211d2700 1 -- 172.21.15.181:0/1551633242 wait complete. 2020-11-16T22:35:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2020-11-16T22:35:16.956 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:35:16.956 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:35:16.956 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:35:16.956 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.957 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:35:16.957 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:35:16.957 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:35:16.957 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.957 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:35:16.958 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:35:16.958 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:35:16.958 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.958 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-16T22:35:16.958 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-16T22:35:16.959 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:35:16.959 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.959 INFO:tasks.ceph.mds.e:Restarting daemon 2020-11-16T22:35:16.959 INFO:tasks.ceph.mds.e:Stopping old one... 2020-11-16T22:35:16.960 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:35:16.960 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.960 INFO:tasks.ceph.mds.f:Restarting daemon 2020-11-16T22:35:16.960 INFO:tasks.ceph.mds.f:Stopping old one... 2020-11-16T22:35:16.960 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:35:16.961 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.961 INFO:tasks.ceph.mds.g:Restarting daemon 2020-11-16T22:35:16.961 INFO:tasks.ceph.mds.g:Stopping old one... 2020-11-16T22:35:16.961 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:35:16.961 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.962 INFO:tasks.ceph.mds.h:Restarting daemon 2020-11-16T22:35:16.962 INFO:tasks.ceph.mds.h:Stopping old one... 2020-11-16T22:35:16.962 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:35:16.962 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.963 INFO:tasks.ceph.mds.i:Restarting daemon 2020-11-16T22:35:16.963 INFO:tasks.ceph.mds.i:Stopping old one... 2020-11-16T22:35:16.963 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:35:16.963 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:35:16.967 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:35:16.967 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:35:16.971 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:35:16.971 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:35:16.973 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:35:16.974 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:35:16.975 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:35:16.975 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:35:16.976 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-16T22:35:16.976 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:16.977 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:35:16.977 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2020-11-16T22:35:16.979 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:35:16.979 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2020-11-16T22:35:16.980 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:35:16.980 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i h 2020-11-16T22:35:16.981 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:35:16.981 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i g 2020-11-16T22:35:16.981 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:35:16.982 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i i 2020-11-16T22:35:17.020 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:35:17.021 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:35:17.021 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.021 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.022 INFO:tasks.ceph.mds.e:Started 2020-11-16T22:35:17.023 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.023 INFO:tasks.ceph.mds.d:Started 2020-11-16T22:35:17.024 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.024 INFO:tasks.ceph.mds.f:Started 2020-11-16T22:35:17.025 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.026 INFO:tasks.ceph.mds.h:Started 2020-11-16T22:35:17.027 INFO:tasks.ceph.mds.g:Started 2020-11-16T22:35:17.027 INFO:tasks.ceph.mds.i:Started 2020-11-16T22:35:17.027 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.027 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.028 DEBUG:teuthology.parallel:result is None 2020-11-16T22:35:17.028 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:35:17.044 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2020-11-16T22:35:17.092 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2020-11-16T22:35:17.099 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2020-11-16T22:35:17.100 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2020-11-16T22:35:17.110 INFO:tasks.ceph.mds.h.smithi175.stdout:starting mds.h at 2020-11-16T22:35:17.111 INFO:tasks.ceph.mds.e.smithi175.stdout:starting mds.e at 2020-11-16T22:35:17.113 INFO:tasks.ceph.mds.i.smithi175.stdout:starting mds.i at 2020-11-16T22:35:17.114 INFO:tasks.ceph.mds.g.smithi175.stdout:starting mds.g at 2020-11-16T22:35:17.115 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2020-11-16T22:35:17.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 Processor -- start 2020-11-16T22:35:17.152 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 -- start start 2020-11-16T22:35:17.153 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5d415f5b0 0x7fe5d40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d4160e40 con 0x7fe5d40615f0 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d4160ba0 con 0x7fe5d40d1830 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d9d0f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d41609a0 con 0x7fe5d415f5b0 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe5d40d1830 0x7fe5d4080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50788/0 (socket says 172.21.15.181:50788) 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d3fff700 1 -- 172.21.15.181:0/4096819707 learned_addr learned my addr 172.21.15.181:0/4096819707 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:17.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d37fe700 1 --2- 172.21.15.181:0/4096819707 >> v2:172.21.15.175:3300/0 conn(0x7fe5d415f5b0 0x7fe5d40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577741436 0 0) 0x7fe5d4160ba0 con 0x7fe5d40d1830 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c0003240 con 0x7fe5d40d1830 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d37fe700 1 -- 172.21.15.181:0/4096819707 >> v1:172.21.15.181:6789/0 conn(0x7fe5d40d1830 legacy=0x7fe5d4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d37fe700 1 -- 172.21.15.181:0/4096819707 >> v1:172.21.15.175:6789/0 conn(0x7fe5d40615f0 legacy=0x7fe5d40622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d37fe700 1 -- 172.21.15.181:0/4096819707 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4060970 con 0x7fe5d415f5b0 2020-11-16T22:35:17.156 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824478375 0 0) 0x7fe5d4160e40 con 0x7fe5d40615f0 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d37fe700 1 --2- 172.21.15.181:0/4096819707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d415f5b0 0x7fe5d40d1340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fe5c400b990 tx=0x7fe5c4006d90).ready entity=mon.0 client_cookie=b6915c79285af86 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4006030 con 0x7fe5d415f5b0 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4006190 con 0x7fe5d415f5b0 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.165+0000 7fe5d27fc700 1 -- 172.21.15.181:0/4096819707 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4006510 con 0x7fe5d415f5b0 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/4096819707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d415f5b0 msgr2=0x7fe5d40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/4096819707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d415f5b0 0x7fe5d40d1340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fe5c400b990 tx=0x7fe5c4006d90).stop 2020-11-16T22:35:17.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/4096819707 shutdown_connections 2020-11-16T22:35:17.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/4096819707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d415f5b0 0x7fe5d40d1340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:17.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/4096819707 >> 172.21.15.181:0/4096819707 conn(0x7fe5d40d69a0 msgr2=0x7fe5d40d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:17.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/4096819707 shutdown_connections 2020-11-16T22:35:17.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/4096819707 wait complete. 2020-11-16T22:35:17.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 Processor -- start 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- start start 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d40615f0 0x7fe5d40cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5d415f5b0 0x7fe5d41723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d41728c0 con 0x7fe5d40615f0 2020-11-16T22:35:17.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4170a10 con 0x7fe5d40d1830 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d9d0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40ba730 con 0x7fe5d415f5b0 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5d415f5b0 0x7fe5d41723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36488/0 (socket says 172.21.15.181:36488) 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 -- 172.21.15.181:0/796798213 learned_addr learned my addr 172.21.15.181:0/796798213 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d37fe700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d40615f0 0x7fe5d40cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:17.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 -- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5d415f5b0 msgr2=0x7fe5d41723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5d415f5b0 0x7fe5d41723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 -- 172.21.15.181:0/796798213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d40615f0 msgr2=0x7fe5d40cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d40615f0 0x7fe5d40cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.169+0000 7fe5d2ffd700 1 -- 172.21.15.181:0/796798213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40ba9d0 con 0x7fe5d40d1830 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d2ffd700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fe5c800c730 tx=0x7fe5c8007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800e490 con 0x7fe5d40d1830 2020-11-16T22:35:17.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c800e610 con 0x7fe5d40d1830 2020-11-16T22:35:17.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40bac70 con 0x7fe5d40d1830 2020-11-16T22:35:17.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800e9d0 con 0x7fe5d40d1830 2020-11-16T22:35:17.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d40bb1b0 con 0x7fe5d40d1830 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d40bafc0 con 0x7fe5d40d1830 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe5c8019030 con 0x7fe5d40d1830 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 0x7fe5ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d37fe700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 0x7fe5ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5511+0+0 (secure 0 0 0) 0x7fe5c806d0e0 con 0x7fe5d40d1830 2020-11-16T22:35:17.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.173+0000 7fe5d37fe700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 0x7fe5ac057520 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe5c400c010 tx=0x7fe5c4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:17.166 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.177+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe5c8043570 con 0x7fe5d40d1830 2020-11-16T22:35:17.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2020-11-16T22:35:17.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2020-11-16T22:35:17.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2020-11-16T22:35:17.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2020-11-16T22:35:17.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g is failed for ~0s 2020-11-16T22:35:17.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i is failed for ~0s 2020-11-16T22:35:17.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:17.345+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe5d4069eb0 con 0x7fe5d40d1830 2020-11-16T22:35:18.154 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.165+0000 7fe5b37fe700 1 -- 172.21.15.181:0/796798213 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe5c8046b90 con 0x7fe5d40d1830 2020-11-16T22:35:18.157 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 msgr2=0x7fe5ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 0x7fe5ac057520 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe5c400c010 tx=0x7fe5c4015000).stop 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 msgr2=0x7fe5d4171ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fe5c800c730 tx=0x7fe5c8007a90).stop 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 shutdown_connections 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe5ac055060 0x7fe5ac057520 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5d415f5b0 0x7fe5d41723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d40615f0 0x7fe5d40cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 --2- 172.21.15.181:0/796798213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5d40d1830 0x7fe5d4171ee0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 >> 172.21.15.181:0/796798213 conn(0x7fe5d40d69a0 msgr2=0x7fe5d415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:18.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 shutdown_connections 2020-11-16T22:35:18.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.169+0000 7fe5d9d0f700 1 -- 172.21.15.181:0/796798213 wait complete. 2020-11-16T22:35:18.159 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2020-11-16T22:35:18.175 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:35:18.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.365+0000 7ff99755f700 1 Processor -- start 2020-11-16T22:35:18.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.365+0000 7ff99755f700 1 -- start start 2020-11-16T22:35:18.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff990157a80 0x7ff9900cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff99015dfd0 0x7ff99015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9901611b0 con 0x7ff990061100 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99015f0b0 con 0x7ff99015dfd0 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99015dbe0 con 0x7ff990157a80 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff990157a80 0x7ff9900cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff990157a80 0x7ff9900cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36496/0 (socket says 172.21.15.181:36496) 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 -- 172.21.15.181:0/3072592725 learned_addr learned my addr 172.21.15.181:0/3072592725 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff995d5c700 1 --1- 172.21.15.181:0/3072592725 >> v1:172.21.15.181:6789/0 conn(0x7ff990061100 0x7ff9900615b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50802/0 (socket says 172.21.15.181:50802) 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff996d5e700 1 --2- 172.21.15.181:0/3072592725 >> v2:172.21.15.175:3300/0 conn(0x7ff99015dfd0 0x7ff99015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 -- 172.21.15.181:0/3072592725 >> v2:172.21.15.175:3300/0 conn(0x7ff99015dfd0 msgr2=0x7ff99015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 --2- 172.21.15.181:0/3072592725 >> v2:172.21.15.175:3300/0 conn(0x7ff99015dfd0 0x7ff99015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 -- 172.21.15.181:0/3072592725 >> v1:172.21.15.181:6789/0 conn(0x7ff990061100 legacy=0x7ff9900615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 -- 172.21.15.181:0/3072592725 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff990161450 con 0x7ff990157a80 2020-11-16T22:35:18.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99655d700 1 --2- 172.21.15.181:0/3072592725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990157a80 0x7ff9900cc0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7ff9900675c0 tx=0x7ff98c006eb0).ready entity=mon.1 client_cookie=c8af41c7b1557882 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99555b700 1 -- 172.21.15.181:0/3072592725 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0098d0 con 0x7ff990157a80 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99555b700 1 -- 172.21.15.181:0/3072592725 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c009a30 con 0x7ff990157a80 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99555b700 1 -- 172.21.15.181:0/3072592725 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c009db0 con 0x7ff990157a80 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- 172.21.15.181:0/3072592725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990157a80 msgr2=0x7ff9900cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 --2- 172.21.15.181:0/3072592725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990157a80 0x7ff9900cc0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7ff9900675c0 tx=0x7ff98c006eb0).stop 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- 172.21.15.181:0/3072592725 shutdown_connections 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 --2- 172.21.15.181:0/3072592725 >> v2:172.21.15.175:3300/0 conn(0x7ff99015dfd0 0x7ff99015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 --2- 172.21.15.181:0/3072592725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990157a80 0x7ff9900cc0c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.369+0000 7ff99755f700 1 -- 172.21.15.181:0/3072592725 >> 172.21.15.181:0/3072592725 conn(0x7ff9900cf630 msgr2=0x7ff9900cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:18.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- 172.21.15.181:0/3072592725 shutdown_connections 2020-11-16T22:35:18.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- 172.21.15.181:0/3072592725 wait complete. 2020-11-16T22:35:18.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 Processor -- start 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- start start 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990157a80 0x7ff990171100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff99015dfd0 0x7ff9900cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99015b8c0 con 0x7ff990157a80 2020-11-16T22:35:18.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99015bb60 con 0x7ff990061100 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99755f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99015be00 con 0x7ff99015dfd0 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99655d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.373+0000 7ff99655d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36502/0 (socket says 172.21.15.181:36502) 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 -- 172.21.15.181:0/3513458446 learned_addr learned my addr 172.21.15.181:0/3513458446 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff996d5e700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff99015dfd0 0x7ff9900cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff995d5c700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990157a80 0x7ff990171100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 -- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff99015dfd0 msgr2=0x7ff9900cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff99015dfd0 0x7ff9900cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 -- 172.21.15.181:0/3513458446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990157a80 msgr2=0x7ff990171100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990157a80 0x7ff990171100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 -- 172.21.15.181:0/3513458446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99015c0a0 con 0x7ff990061100 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99655d700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7ff9900673c0 tx=0x7ff98c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c011c80 con 0x7ff990061100 2020-11-16T22:35:18.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c011de0 con 0x7ff990061100 2020-11-16T22:35:18.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99015c340 con 0x7ff990061100 2020-11-16T22:35:18.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c01e670 con 0x7ff990061100 2020-11-16T22:35:18.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901715f0 con 0x7ff990061100 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9900cd270 con 0x7ff990061100 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff98c025030 con 0x7ff990061100 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 0x7ff968057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff98c06f070 con 0x7ff990061100 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.377+0000 7ff995d5c700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 0x7ff968057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.381+0000 7ff995d5c700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 0x7ff968057520 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ff99016aee0 tx=0x7ff980007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.389+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff9900cd270 con 0x7ff990061100 2020-11-16T22:35:18.578 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.589+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9900cd270 con 0x7ff990061100 2020-11-16T22:35:18.579 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.589+0000 7ff97e7fc700 1 -- 172.21.15.181:0/3513458446 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5978 (secure 0 0 0) 0x7ff9900cd270 con 0x7ff990061100 2020-11-16T22:35:18.579 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:18.579 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:35:18.579 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfQYPzMhAAb0ES2Fvnsteuy3SzUoCCUA==", 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.580 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfXl4yOxAAzSkcjXVPkYU99fn/Vwe8Vg==", 2020-11-16T22:35:18.581 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfeq0PCBAA9RM57y8RHX2hWVxlrySR2Q==", 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.583 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf+ctaEBAAwzA8uU7zKIh++PciVgfhzA==", 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.584 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.e", 2020-11-16T22:35:18.585 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfsl3uFRAAccssZGkJyLQnz4BDRo7JsA==", 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.f", 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfp0O/HhAA0ManuOf8GStpv8zP5zwNmQ==", 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.587 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.g", 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJf8D0SJxAAKEWuT/tgc4FD0EMUgnanRg==", 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.588 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.589 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.589 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.589 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.589 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.h", 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfgF6kLxAAffde2wBBq5I8tDBcaQwNpw==", 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.591 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.i", 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB4/LJfrN5OOBAAptlErsVOiEoAMOeK4tzUFg==", 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB7/LJfIAATOxAAQyuZ8HcwmwhY99JF4H0x/Q==", 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.593 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB//LJfZEV0ARAAybzJB93itQ6aZmMJ16ivng==", 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.594 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2020-11-16T22:35:18.595 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCC/LJfl2dfBhAAEW8q5IDF8MarpHjM9tgMfg==", 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCF/LJfEcMtCxAAdLkWwgLk0nUPGzQwSTvE0w==", 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCK/LJfvyWhNRAAshjRAtRdTF99dSzcCzS0+g==", 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.598 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCN/LJfrCOzOBAAuA57sAxbR1+vmCUExENmZw==", 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.599 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2020-11-16T22:35:18.600 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCR/LJfBUzkAxAA9ce4HNWvICX1qXGVmiFN3Q==", 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCU/LJfdAlQBxAAWA1LJFEhK69dj/deLhYepw==", 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJfOqMxBBAA7rcmEzQ0ajIgzdXZuorcmA==", 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2020-11-16T22:35:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB5/LJf/kwtCRAAdrr/JqNNxaeTpp4//UBByw==", 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.604 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2020-11-16T22:35:18.605 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfxsgKDxAAHL3EB03lrVRDNb/N0Tg5rg==", 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJfLr8eMBAAO5TiJRNW7GkSI/7WZqcA0Q==", 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:35:18.607 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQB3/LJf1sabLRAAm4cjaC3qE06M/wXdDX1q0g==", 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2020-11-16T22:35:18.608 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2020-11-16T22:35:18.609 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2020-11-16T22:35:18.609 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.609 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:18.609 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2020-11-16T22:35:18.609 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 msgr2=0x7ff968057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 0x7ff968057520 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ff99016aee0 tx=0x7ff980007b80).stop 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 msgr2=0x7ff99015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7ff9900673c0 tx=0x7ff98c015000).stop 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 shutdown_connections 2020-11-16T22:35:18.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff968055060 0x7ff968057520 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff99015dfd0 0x7ff9900cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff990061100 0x7ff99015ca80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 --2- 172.21.15.181:0/3513458446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990157a80 0x7ff990171100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 >> 172.21.15.181:0/3513458446 conn(0x7ff9900cf630 msgr2=0x7ff990156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 shutdown_connections 2020-11-16T22:35:18.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.593+0000 7ff99755f700 1 -- 172.21.15.181:0/3513458446 wait complete. 2020-11-16T22:35:18.613 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:18.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 Processor -- start 2020-11-16T22:35:18.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 -- start start 2020-11-16T22:35:18.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98300b2900 0x7f98300b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 0x7f983015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98300b31d0 con 0x7f983015d2e0 2020-11-16T22:35:18.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300c8910 con 0x7f98300b2900 2020-11-16T22:35:18.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98362f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300b3460 con 0x7f98300cf6b0 2020-11-16T22:35:18.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9835af0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f983015d2e0 0x7f98300c7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50814/0 (socket says 172.21.15.181:50814) 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f98352ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98300b2900 0x7f98300b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9835af0700 1 -- 172.21.15.181:0/16844168 learned_addr learned my addr 172.21.15.181:0/16844168 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 --2- 172.21.15.181:0/16844168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 0x7f983015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9823fff700 1 -- 172.21.15.181:0/16844168 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048323053 0 0) 0x7f98300b31d0 con 0x7f983015d2e0 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 -- 172.21.15.181:0/16844168 >> v2:172.21.15.181:3300/0 conn(0x7f98300b2900 msgr2=0x7f98300b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:18.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 --2- 172.21.15.181:0/16844168 >> v2:172.21.15.181:3300/0 conn(0x7f98300b2900 0x7f98300b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 -- 172.21.15.181:0/16844168 >> v1:172.21.15.181:6789/0 conn(0x7f983015d2e0 legacy=0x7f98300c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 -- 172.21.15.181:0/16844168 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300c8130 con 0x7f98300cf6b0 2020-11-16T22:35:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9834aee700 1 --2- 172.21.15.181:0/16844168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 0x7f983015cdf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f9824009c90 tx=0x7f98240091d0).ready entity=mon.2 client_cookie=9324c46e592ac9c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9823fff700 1 -- 172.21.15.181:0/16844168 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982400ba70 con 0x7f98300cf6b0 2020-11-16T22:35:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9823fff700 1 -- 172.21.15.181:0/16844168 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982400bbd0 con 0x7f98300cf6b0 2020-11-16T22:35:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.753+0000 7f9823fff700 1 -- 172.21.15.181:0/16844168 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824011400 con 0x7f98300cf6b0 2020-11-16T22:35:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 -- 172.21.15.181:0/16844168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 msgr2=0x7f983015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 --2- 172.21.15.181:0/16844168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 0x7f983015cdf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f9824009c90 tx=0x7f98240091d0).stop 2020-11-16T22:35:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 -- 172.21.15.181:0/16844168 shutdown_connections 2020-11-16T22:35:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 --2- 172.21.15.181:0/16844168 >> v2:172.21.15.181:3300/0 conn(0x7f98300b2900 0x7f98300b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 --2- 172.21.15.181:0/16844168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300cf6b0 0x7f983015cdf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.757+0000 7f98362f1700 1 -- 172.21.15.181:0/16844168 >> 172.21.15.181:0/16844168 conn(0x7f98300d2dc0 msgr2=0x7f98300ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- 172.21.15.181:0/16844168 shutdown_connections 2020-11-16T22:35:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- 172.21.15.181:0/16844168 wait complete. 2020-11-16T22:35:18.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 Processor -- start 2020-11-16T22:35:18.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- start start 2020-11-16T22:35:18.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2900 0x7f98300b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98300cf6b0 0x7f98300b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830171330 con 0x7f98300cf6b0 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300b7970 con 0x7f983015d2e0 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98362f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9830170410 con 0x7f98300b2900 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36518/0 (socket says 172.21.15.181:36518) 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f98352ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2900 0x7f98300b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 -- 172.21.15.181:0/2551275594 learned_addr learned my addr 172.21.15.181:0/2551275594 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9834aee700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98300cf6b0 0x7f98300b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 -- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2900 msgr2=0x7f98300b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2900 0x7f98300b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 -- 172.21.15.181:0/2551275594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98300cf6b0 msgr2=0x7f98300b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98300cf6b0 0x7f98300b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 -- 172.21.15.181:0/2551275594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98301706b0 con 0x7f983015d2e0 2020-11-16T22:35:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.761+0000 7f9835af0700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f982800c710 tx=0x7f9828007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280072e0 con 0x7f983015d2e0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828007440 con 0x7f983015d2e0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828005820 con 0x7f983015d2e0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9830170950 con 0x7f983015d2e0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983015c7b0 con 0x7f983015d2e0 2020-11-16T22:35:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9830044f60 con 0x7f983015d2e0 2020-11-16T22:35:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9828010070 con 0x7f983015d2e0 2020-11-16T22:35:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 0x7f9808057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98352ef700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 0x7f9808057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f982806d080 con 0x7f983015d2e0 2020-11-16T22:35:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.765+0000 7f98352ef700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 0x7f9808057520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f982c006fd0 tx=0x7f982c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.769+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f98280435b0 con 0x7f983015d2e0 2020-11-16T22:35:18.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.917+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f983005d550 con 0x7f983015d2e0 2020-11-16T22:35:18.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98217fa700 1 -- 172.21.15.181:0/2551275594 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13725 (secure 0 0 0) 0x7f9828046bd0 con 0x7f983015d2e0 2020-11-16T22:35:18.908 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:18.908 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":104,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:18.155854+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:18.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 msgr2=0x7f9808057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 0x7f9808057520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f982c006fd0 tx=0x7f982c006df0).stop 2020-11-16T22:35:18.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 msgr2=0x7f9830156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:18.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f982800c710 tx=0x7f9828007960).stop 2020-11-16T22:35:18.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 shutdown_connections 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9808055060 0x7f9808057520 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f983015d2e0 0x7f9830156f10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2900 0x7f98300b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 --2- 172.21.15.181:0/2551275594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98300cf6b0 0x7f98300b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 >> 172.21.15.181:0/2551275594 conn(0x7f98300d2dc0 msgr2=0x7f98301589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:18.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 shutdown_connections 2020-11-16T22:35:18.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:18.921+0000 7f98362f1700 1 -- 172.21.15.181:0/2551275594 wait complete. 2020-11-16T22:35:18.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:35:18.923 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:35:19.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 Processor -- start 2020-11-16T22:35:19.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 -- start start 2020-11-16T22:35:19.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 0x7fd588166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd58816d400 0x7fd5880c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd588166e40 con 0x7fd58816fee0 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd588170840 con 0x7fd58816d400 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd587fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd588170ae0 con 0x7fd588166570 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd586ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 0x7fd588166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd5867fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd58816d400 0x7fd5880c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd5867fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd58816d400 0x7fd5880c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36524/0 (socket says 172.21.15.181:36524) 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd586ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 0x7fd588166950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44370/0 (socket says 172.21.15.181:44370) 2020-11-16T22:35:19.216 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd5867fc700 1 -- 172.21.15.181:0/3478826134 learned_addr learned my addr 172.21.15.181:0/3478826134 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.069+0000 7fd585ffb700 1 -- 172.21.15.181:0/3478826134 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126342652 0 0) 0x7fd588166e40 con 0x7fd58816fee0 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd585ffb700 1 -- 172.21.15.181:0/3478826134 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd570003220 con 0x7fd58816fee0 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd5867fc700 1 -- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 msgr2=0x7fd588166950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd5867fc700 1 --2- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 0x7fd588166950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd5867fc700 1 -- 172.21.15.181:0/3478826134 >> v1:172.21.15.175:6789/0 conn(0x7fd58816fee0 legacy=0x7fd5880621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd5867fc700 1 -- 172.21.15.181:0/3478826134 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588170d80 con 0x7fd58816d400 2020-11-16T22:35:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd5867fc700 1 --2- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd58816d400 0x7fd5880c7250 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fd57c006fd0 tx=0x7fd57c006eb0).ready entity=mon.1 client_cookie=43182f74c547a098 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd585ffb700 1 -- 172.21.15.181:0/3478826134 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c013070 con 0x7fd58816d400 2020-11-16T22:35:19.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd585ffb700 1 -- 172.21.15.181:0/3478826134 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00c070 con 0x7fd58816d400 2020-11-16T22:35:19.218 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.225+0000 7fd585ffb700 1 -- 172.21.15.181:0/3478826134 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c009ae0 con 0x7fd58816d400 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 -- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd58816d400 msgr2=0x7fd5880c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 --2- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd58816d400 0x7fd5880c7250 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fd57c006fd0 tx=0x7fd57c006eb0).stop 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 -- 172.21.15.181:0/3478826134 shutdown_connections 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 --2- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd588166570 0x7fd588166950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 --2- 172.21.15.181:0/3478826134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd58816d400 0x7fd5880c7250 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.219 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 -- 172.21.15.181:0/3478826134 >> 172.21.15.181:0/3478826134 conn(0x7fd5880d22f0 msgr2=0x7fd5880d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:19.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 -- 172.21.15.181:0/3478826134 shutdown_connections 2020-11-16T22:35:19.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.229+0000 7fd587fff700 1 -- 172.21.15.181:0/3478826134 wait complete. 2020-11-16T22:35:19.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 Processor -- start 2020-11-16T22:35:19.220 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 -- start start 2020-11-16T22:35:19.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58816d400 0x7fd5880c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd58816fee0 0x7fd5880c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.223 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880cd520 con 0x7fd58816d400 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880cd7c0 con 0x7fd588166570 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd587fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880cda60 con 0x7fd58816fee0 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd586ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd586ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36530/0 (socket says 172.21.15.181:36530) 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd586ffd700 1 -- 172.21.15.181:0/3381762483 learned_addr learned my addr 172.21.15.181:0/3381762483 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.233+0000 7fd5877fe700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd58816fee0 0x7fd5880c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.224 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd5867fc700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58816d400 0x7fd5880c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 -- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd58816fee0 msgr2=0x7fd5880c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.225 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd58816fee0 0x7fd5880c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 -- 172.21.15.181:0/3381762483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58816d400 msgr2=0x7fd5880c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58816d400 0x7fd5880c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 -- 172.21.15.181:0/3381762483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880cdd00 con 0x7fd588166570 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd586ffd700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fd57800eca0 tx=0x7fd578006b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800c920 con 0x7fd588166570 2020-11-16T22:35:19.226 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800ca80 con 0x7fd588166570 2020-11-16T22:35:19.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880cdfa0 con 0x7fd588166570 2020-11-16T22:35:19.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800ce20 con 0x7fd588166570 2020-11-16T22:35:19.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.237+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5880ce6e0 con 0x7fd588166570 2020-11-16T22:35:19.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588044f60 con 0x7fd588166570 2020-11-16T22:35:19.227 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd578021070 con 0x7fd588166570 2020-11-16T22:35:19.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd56effd700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 0x7fd564057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.228 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd57806dea0 con 0x7fd588166570 2020-11-16T22:35:19.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd5867fc700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 0x7fd564057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.229 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.241+0000 7fd5867fc700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 0x7fd564057520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd57c006ef0 tx=0x7fd57c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.232 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.245+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd5780442c0 con 0x7fd588166570 2020-11-16T22:35:19.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.397+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd58805d550 con 0x7fd588166570 2020-11-16T22:35:19.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.397+0000 7fd56effd700 1 -- 172.21.15.181:0/3381762483 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v104) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd57801f030 con 0x7fd588166570 2020-11-16T22:35:19.386 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 msgr2=0x7fd564057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 0x7fd564057520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd57c006ef0 tx=0x7fd57c015000).stop 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 msgr2=0x7fd5880ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fd57800eca0 tx=0x7fd578006b60).stop 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 shutdown_connections 2020-11-16T22:35:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fd564055060 0x7fd564057520 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd58816fee0 0x7fd5880c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd588166570 0x7fd5880ccbc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 --2- 172.21.15.181:0/3381762483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58816d400 0x7fd5880c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 >> 172.21.15.181:0/3381762483 conn(0x7fd5880d22f0 msgr2=0x7fd58816f410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 shutdown_connections 2020-11-16T22:35:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.401+0000 7fd587fff700 1 -- 172.21.15.181:0/3381762483 wait complete. 2020-11-16T22:35:19.400 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:35:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.541+0000 7f2170b91700 1 Processor -- start 2020-11-16T22:35:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.541+0000 7f2170b91700 1 -- start start 2020-11-16T22:35:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f216c062790 0x7f216c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c160e40 con 0x7f216c15f5b0 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c1609a0 con 0x7f216c062790 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c160ba0 con 0x7f216c0615f0 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f216ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f216c062790 0x7f216c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44382/0 (socket says 172.21.15.181:44382) 2020-11-16T22:35:19.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 -- 172.21.15.181:0/3148467358 learned_addr learned my addr 172.21.15.181:0/3148467358 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f216a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f216c15f5b0 0x7f216c0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50842/0 (socket says 172.21.15.181:50842) 2020-11-16T22:35:19.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 -- 172.21.15.181:0/3148467358 >> v2:172.21.15.181:3300/0 conn(0x7f216c062790 msgr2=0x7f216c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:19.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 --2- 172.21.15.181:0/3148467358 >> v2:172.21.15.181:3300/0 conn(0x7f216c062790 0x7f216c080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 -- 172.21.15.181:0/3148467358 >> v1:172.21.15.181:6789/0 conn(0x7f216c15f5b0 legacy=0x7f216c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 -- 172.21.15.181:0/3148467358 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c060970 con 0x7f216c0615f0 2020-11-16T22:35:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2169d9b700 1 --2- 172.21.15.181:0/3148467358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2160006fd0 tx=0x7f2160006eb0).ready entity=mon.2 client_cookie=5bf3b8e24c10bde4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f216959a700 1 -- 172.21.15.181:0/3148467358 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160009810 con 0x7f216c0615f0 2020-11-16T22:35:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f216959a700 1 -- 172.21.15.181:0/3148467358 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160009970 con 0x7f216c0615f0 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f216959a700 1 -- 172.21.15.181:0/3148467358 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160009cf0 con 0x7f216c0615f0 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- 172.21.15.181:0/3148467358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 msgr2=0x7f216c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 --2- 172.21.15.181:0/3148467358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2160006fd0 tx=0x7f2160006eb0).stop 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- 172.21.15.181:0/3148467358 shutdown_connections 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 --2- 172.21.15.181:0/3148467358 >> v2:172.21.15.181:3300/0 conn(0x7f216c062790 0x7f216c080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 --2- 172.21.15.181:0/3148467358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c0622a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- 172.21.15.181:0/3148467358 >> 172.21.15.181:0/3148467358 conn(0x7f216c0d69a0 msgr2=0x7f216c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- 172.21.15.181:0/3148467358 shutdown_connections 2020-11-16T22:35:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.545+0000 7f2170b91700 1 -- 172.21.15.181:0/3148467358 wait complete. 2020-11-16T22:35:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.549+0000 7f2170b91700 1 Processor -- start 2020-11-16T22:35:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.549+0000 7f2170b91700 1 -- start start 2020-11-16T22:35:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c16fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c15f5b0 0x7f216c0d6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0d4d40 con 0x7f216c15f5b0 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0d4fe0 con 0x7f216c062790 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0d5280 con 0x7f216c0615f0 2020-11-16T22:35:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f216a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c16fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36546/0 (socket says 172.21.15.181:36546) 2020-11-16T22:35:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 -- 172.21.15.181:0/371295139 learned_addr learned my addr 172.21.15.181:0/371295139 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f216ad9d700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c15f5b0 0x7f216c0d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 -- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 msgr2=0x7f216c16fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c16fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 -- 172.21.15.181:0/371295139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c15f5b0 msgr2=0x7f216c0d6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c15f5b0 0x7f216c0d6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 -- 172.21.15.181:0/371295139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0d5520 con 0x7f216c062790 2020-11-16T22:35:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f216a59c700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c16fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2169d9b700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f2160010000 tx=0x7f2160015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011d50 con 0x7f216c062790 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216001e460 con 0x7f216c062790 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c16f710 con 0x7f216c062790 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216001e7e0 con 0x7f216c062790 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c0b6e40 con 0x7f216c062790 2020-11-16T22:35:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c0b7090 con 0x7f216c062790 2020-11-16T22:35:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.553+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2160025030 con 0x7f216c062790 2020-11-16T22:35:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.557+0000 7f215a7fc700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 0x7f2140057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.557+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f216006f0e0 con 0x7f216c062790 2020-11-16T22:35:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.557+0000 7f216a59c700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 0x7f2140057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.557+0000 7f216a59c700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 0x7f2140057520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f215c000bc0 tx=0x7f215c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.550 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.561+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f216c0b7090 con 0x7f216c062790 2020-11-16T22:35:19.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.717+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f216c0b7090 con 0x7f216c062790 2020-11-16T22:35:19.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.717+0000 7f215a7fc700 1 -- 172.21.15.181:0/371295139 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7f216c0b7090 con 0x7f216c062790 2020-11-16T22:35:19.704 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:19.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 msgr2=0x7f2140057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 0x7f2140057520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f215c000bc0 tx=0x7f215c009040).stop 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 msgr2=0x7f216c0d5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f2160010000 tx=0x7f2160015000).stop 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 shutdown_connections 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2140055060 0x7f2140057520 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f216c15f5b0 0x7f216c0d6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0615f0 0x7f216c16fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 --2- 172.21.15.181:0/371295139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c062790 0x7f216c0d5e30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 >> 172.21.15.181:0/371295139 conn(0x7f216c0d69a0 msgr2=0x7f216c15d640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:19.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 shutdown_connections 2020-11-16T22:35:19.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.721+0000 7f2170b91700 1 -- 172.21.15.181:0/371295139 wait complete. 2020-11-16T22:35:19.726 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:35:19.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.857+0000 7fc052e4c700 1 Processor -- start 2020-11-16T22:35:19.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.857+0000 7fc052e4c700 1 -- start start 2020-11-16T22:35:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 0x7fc04c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 0x7fc04c160430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0ce460 con 0x7fc04c0b2570 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0ce700 con 0x7fc04c15ff80 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c0ce9a0 con 0x7fc04c0b2190 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc05264b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 0x7fc04c160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44394/0 (socket says 172.21.15.181:44394) 2020-11-16T22:35:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 -- 172.21.15.181:0/4275785833 learned_addr learned my addr 172.21.15.181:0/4275785833 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051649700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 0x7fc04c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 -- 172.21.15.181:0/4275785833 >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 msgr2=0x7fc04c160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 0x7fc04c160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 -- 172.21.15.181:0/4275785833 >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 msgr2=0x7fc04c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 0x7fc04c0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 -- 172.21.15.181:0/4275785833 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0cec40 con 0x7fc04c0b2190 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc05264b700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 0x7fc04c160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051649700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 0x7fc04c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc051e4a700 1 --2- 172.21.15.181:0/4275785833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc048006fd0 tx=0x7fc048006f40).ready entity=mon.2 client_cookie=66a9cf971251f833 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc050e48700 1 -- 172.21.15.181:0/4275785833 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800dc20 con 0x7fc04c0b2190 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc050e48700 1 -- 172.21.15.181:0/4275785833 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800dd80 con 0x7fc04c0b2190 2020-11-16T22:35:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc050e48700 1 -- 172.21.15.181:0/4275785833 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048010680 con 0x7fc04c0b2190 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- 172.21.15.181:0/4275785833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 msgr2=0x7fc04c15de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- 172.21.15.181:0/4275785833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc048006fd0 tx=0x7fc048006f40).stop 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- 172.21.15.181:0/4275785833 shutdown_connections 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.181:3300/0 conn(0x7fc04c15ff80 0x7fc04c160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- 172.21.15.181:0/4275785833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c0b2190 0x7fc04c15de80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 --2- 172.21.15.181:0/4275785833 >> v2:172.21.15.175:3300/0 conn(0x7fc04c0b2570 0x7fc04c0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- 172.21.15.181:0/4275785833 >> 172.21.15.181:0/4275785833 conn(0x7fc04c0cfe90 msgr2=0x7fc04c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- 172.21.15.181:0/4275785833 shutdown_connections 2020-11-16T22:35:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.861+0000 7fc052e4c700 1 -- 172.21.15.181:0/4275785833 wait complete. 2020-11-16T22:35:19.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 Processor -- start 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- start start 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 0x7fc04c0d2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04c0b2570 0x7fc04c1334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c0d2d00 con 0x7fc04c0b2570 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c16b350 con 0x7fc04c0b2190 2020-11-16T22:35:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c16b5f0 con 0x7fc04c15ff80 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc051e4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 0x7fc04c0d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44404/0 (socket says 172.21.15.181:44404) 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc051e4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 0x7fc04c0d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36558/0 (socket says 172.21.15.181:36558) 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 -- 172.21.15.181:0/2757648343 learned_addr learned my addr 172.21.15.181:0/2757648343 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc051649700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04c0b2570 0x7fc04c1334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 -- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 msgr2=0x7fc04c0d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 0x7fc04c0d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 -- 172.21.15.181:0/2757648343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04c0b2570 msgr2=0x7fc04c1334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04c0b2570 0x7fc04c1334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 -- 172.21.15.181:0/2757648343 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c16b890 con 0x7fc04c15ff80 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc05264b700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fc04400ef50 tx=0x7fc0440091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400cc60 con 0x7fc04c15ff80 2020-11-16T22:35:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400cdc0 con 0x7fc04c15ff80 2020-11-16T22:35:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c16bb30 con 0x7fc04c15ff80 2020-11-16T22:35:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044012640 con 0x7fc04c15ff80 2020-11-16T22:35:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.865+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c1339e0 con 0x7fc04c15ff80 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c068fb0 con 0x7fc04c15ff80 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc044022070 con 0x7fc04c15ff80 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc039ffb700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 0x7fc024057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc04406e2e0 con 0x7fc04c15ff80 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc051e4a700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 0x7fc024057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.869+0000 7fc051e4a700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 0x7fc024057520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fc04800d920 tx=0x7fc048006f40).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:19.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:19.877+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc04c068fb0 con 0x7fc04c15ff80 2020-11-16T22:35:20.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.033+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc04c068fb0 con 0x7fc04c15ff80 2020-11-16T22:35:20.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.033+0000 7fc039ffb700 1 -- 172.21.15.181:0/2757648343 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc0440160a0 con 0x7fc04c15ff80 2020-11-16T22:35:20.022 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:20.022 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "device_health_metrics" 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 2, 2020-11-16T22:35:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "rbd" 2020-11-16T22:35:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:35:20.024 INFO:teuthology.orchestra.run.smithi181.stdout:] 2020-11-16T22:35:20.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 msgr2=0x7fc024057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 0x7fc024057520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fc04800d920 tx=0x7fc048006f40).stop 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 msgr2=0x7fc04c0b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fc04400ef50 tx=0x7fc0440091d0).stop 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 shutdown_connections 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc024055060 0x7fc024057520 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc04c15ff80 0x7fc04c0b8410 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc04c0b2190 0x7fc04c0d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 --2- 172.21.15.181:0/2757648343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04c0b2570 0x7fc04c1334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 >> 172.21.15.181:0/2757648343 conn(0x7fc04c0cfe90 msgr2=0x7fc04c1455d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:20.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 shutdown_connections 2020-11-16T22:35:20.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.037+0000 7fc052e4c700 1 -- 172.21.15.181:0/2757648343 wait complete. 2020-11-16T22:35:20.042 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:35:20.042 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:35:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 Processor -- start 2020-11-16T22:35:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- start start 2020-11-16T22:35:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c4015f5b0 0x7f9c400d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9c400615f0 0x7f9c400d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c40160e40 con 0x7f9c400622a0 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c401609a0 con 0x7f9c400615f0 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c40160ba0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c4015f5b0 0x7f9c400d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c4015f5b0 0x7f9c400d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36566/0 (socket says 172.21.15.181:36566) 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 -- 172.21.15.181:0/3130186792 learned_addr learned my addr 172.21.15.181:0/3130186792 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3e59c700 1 --2- 172.21.15.181:0/3130186792 >> v2:172.21.15.175:3300/0 conn(0x7f9c400615f0 0x7f9c400d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 -- 172.21.15.181:0/3130186792 >> v2:172.21.15.175:3300/0 conn(0x7f9c400615f0 msgr2=0x7f9c400d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 --2- 172.21.15.181:0/3130186792 >> v2:172.21.15.175:3300/0 conn(0x7f9c400615f0 0x7f9c400d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 -- 172.21.15.181:0/3130186792 >> v1:172.21.15.175:6789/0 conn(0x7f9c400622a0 legacy=0x7f9c40062730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 -- 172.21.15.181:0/3130186792 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40060970 con 0x7f9c4015f5b0 2020-11-16T22:35:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3ed9d700 1 --2- 172.21.15.181:0/3130186792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4015f5b0 0x7f9c400d1340 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f9c40083ad0 tx=0x7f9c30006e50).ready entity=mon.1 client_cookie=1d2a21bdfd725a7d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3dd9b700 1 -- 172.21.15.181:0/3130186792 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30006030 con 0x7f9c4015f5b0 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3dd9b700 1 -- 172.21.15.181:0/3130186792 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c30006190 con 0x7f9c4015f5b0 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c3dd9b700 1 -- 172.21.15.181:0/3130186792 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30006510 con 0x7f9c4015f5b0 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- 172.21.15.181:0/3130186792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4015f5b0 msgr2=0x7f9c400d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 --2- 172.21.15.181:0/3130186792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4015f5b0 0x7f9c400d1340 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f9c40083ad0 tx=0x7f9c30006e50).stop 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- 172.21.15.181:0/3130186792 shutdown_connections 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 --2- 172.21.15.181:0/3130186792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4015f5b0 0x7f9c400d1340 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 --2- 172.21.15.181:0/3130186792 >> v2:172.21.15.175:3300/0 conn(0x7f9c400615f0 0x7f9c400d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- 172.21.15.181:0/3130186792 >> 172.21.15.181:0/3130186792 conn(0x7f9c400d69a0 msgr2=0x7f9c400d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- 172.21.15.181:0/3130186792 shutdown_connections 2020-11-16T22:35:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.189+0000 7f9c45051700 1 -- 172.21.15.181:0/3130186792 wait complete. 2020-11-16T22:35:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 Processor -- start 2020-11-16T22:35:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 -- start start 2020-11-16T22:35:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c400615f0 0x7f9c400d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c400622a0 0x7f9c400d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400d3f20 con 0x7f9c400622a0 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4015b9c0 con 0x7f9c400615f0 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c45051700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4015bc60 con 0x7f9c4015f5b0 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44414/0 (socket says 172.21.15.181:44414) 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 -- 172.21.15.181:0/642439131 learned_addr learned my addr 172.21.15.181:0/642439131 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3ed9d700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c400615f0 0x7f9c400d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3e59c700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c400622a0 0x7f9c400d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 -- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c400615f0 msgr2=0x7f9c400d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c400615f0 0x7f9c400d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 -- 172.21.15.181:0/642439131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c400622a0 msgr2=0x7f9c400d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c400622a0 0x7f9c400d47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.193+0000 7f9c3f59e700 1 -- 172.21.15.181:0/642439131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400d3050 con 0x7f9c4015f5b0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c3f59e700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9c3800ca60 tx=0x7f9c38007e70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800a770 con 0x7f9c4015f5b0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800a8d0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c400d32f0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800ac70 con 0x7f9c4015f5b0 2020-11-16T22:35:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c400d4cc0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c40069eb0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.187 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9c38019030 con 0x7f9c4015f5b0 2020-11-16T22:35:20.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 0x7f9c1c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:20.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c3ed9d700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 0x7f9c1c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:20.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9c3806cf50 con 0x7f9c4015f5b0 2020-11-16T22:35:20.188 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.197+0000 7f9c3ed9d700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 0x7f9c1c057520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9c3000b1b0 tx=0x7f9c30015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:20.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.201+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9c38043590 con 0x7f9c4015f5b0 2020-11-16T22:35:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.353+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9c400600f0 con 0x7f9c4015f5b0 2020-11-16T22:35:20.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c26ffd700 1 -- 172.21.15.181:0/642439131 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v105) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9c38017070 con 0x7f9c4015f5b0 2020-11-16T22:35:20.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 msgr2=0x7f9c1c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 0x7f9c1c057520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9c3000b1b0 tx=0x7f9c30015000).stop 2020-11-16T22:35:20.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 msgr2=0x7f9c4015d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:20.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9c3800ca60 tx=0x7f9c38007e70).stop 2020-11-16T22:35:20.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 shutdown_connections 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9c1c055060 0x7f9c1c057520 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4015f5b0 0x7f9c4015d3f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c400615f0 0x7f9c400d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 --2- 172.21.15.181:0/642439131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c400622a0 0x7f9c400d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 >> 172.21.15.181:0/642439131 conn(0x7f9c400d69a0 msgr2=0x7f9c40062730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 shutdown_connections 2020-11-16T22:35:20.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:20.981+0000 7f9c45051700 1 -- 172.21.15.181:0/642439131 wait complete. 2020-11-16T22:35:20.972 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2020-11-16T22:35:20.985 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:35:21.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.125+0000 7fdd9d2b6700 1 Processor -- start 2020-11-16T22:35:21.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.125+0000 7fdd9d2b6700 1 -- start start 2020-11-16T22:35:21.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd9d2b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 0x7fdd98145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:21.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd9d2b6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd98061b50 con 0x7fdd980816e0 2020-11-16T22:35:21.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd9d2b6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd980618b0 con 0x7fdd981492c0 2020-11-16T22:35:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd9d2b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980cb930 con 0x7fdd98145a30 2020-11-16T22:35:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd8f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdd981492c0 0x7fdd98080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50884/0 (socket says 172.21.15.181:50884) 2020-11-16T22:35:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd8f7fe700 1 -- 172.21.15.181:0/499123250 learned_addr learned my addr 172.21.15.181:0/499123250 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd97fff700 1 --2- 172.21.15.181:0/499123250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 0x7fdd98145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd97fff700 1 -- 172.21.15.181:0/499123250 >> v1:172.21.15.181:6789/0 conn(0x7fdd981492c0 legacy=0x7fdd98080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:21.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd97fff700 1 -- 172.21.15.181:0/499123250 >> v1:172.21.15.175:6789/0 conn(0x7fdd980816e0 legacy=0x7fdd980cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:21.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd97fff700 1 -- 172.21.15.181:0/499123250 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd980cbbd0 con 0x7fdd98145a30 2020-11-16T22:35:21.118 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.129+0000 7fdd97fff700 1 --2- 172.21.15.181:0/499123250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 0x7fdd98145e10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fdd8000b9d0 tx=0x7fdd80006df0).ready entity=mon.2 client_cookie=fa26137f4affb415 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd977fe700 1 -- 172.21.15.181:0/499123250 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80009870 con 0x7fdd98145a30 2020-11-16T22:35:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd977fe700 1 -- 172.21.15.181:0/499123250 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800099d0 con 0x7fdd98145a30 2020-11-16T22:35:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd977fe700 1 -- 172.21.15.181:0/499123250 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80009d50 con 0x7fdd98145a30 2020-11-16T22:35:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/499123250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 msgr2=0x7fdd98145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/499123250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 0x7fdd98145e10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fdd8000b9d0 tx=0x7fdd80006df0).stop 2020-11-16T22:35:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/499123250 shutdown_connections 2020-11-16T22:35:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/499123250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd98145a30 0x7fdd98145e10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/499123250 >> 172.21.15.181:0/499123250 conn(0x7fdd980d68c0 msgr2=0x7fdd980d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/499123250 shutdown_connections 2020-11-16T22:35:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/499123250 wait complete. 2020-11-16T22:35:21.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 Processor -- start 2020-11-16T22:35:21.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.133+0000 7fdd9d2b6700 1 -- start start 2020-11-16T22:35:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 0x7fdd9816ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd98145a30 0x7fdd9813c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 0x7fdd9813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9813d0d0 con 0x7fdd98145a30 2020-11-16T22:35:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980d1300 con 0x7fdd980816e0 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981415d0 con 0x7fdd981492c0 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 0x7fdd9816ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd97fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 0x7fdd9816ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36586/0 (socket says 172.21.15.181:36586) 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd97fff700 1 -- 172.21.15.181:0/710461083 learned_addr learned my addr 172.21.15.181:0/710461083 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 0x7fdd9813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd8f7fe700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd98145a30 0x7fdd9813c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 -- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 msgr2=0x7fdd9816ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 0x7fdd9816ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 -- 172.21.15.181:0/710461083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd98145a30 msgr2=0x7fdd9813c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd98145a30 0x7fdd9813c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 -- 172.21.15.181:0/710461083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9813d2c0 con 0x7fdd981492c0 2020-11-16T22:35:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9cab5700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 0x7fdd9813cbe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fdd9000ca60 tx=0x7fdd90007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000ce30 con 0x7fdd981492c0 2020-11-16T22:35:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9000a730 con 0x7fdd981492c0 2020-11-16T22:35:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd980d1560 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000aab0 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9813d560 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.137+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd98044f60 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.141+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdd90019030 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.141+0000 7fdd94ff9700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 0x7fdd74057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.141+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fdd9006d1d0 con 0x7fdd981492c0 2020-11-16T22:35:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.141+0000 7fdd97fff700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 0x7fdd74057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.141+0000 7fdd97fff700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 0x7fdd74057550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fdd80006e30 tx=0x7fdd80015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:21.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.145+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdd900436b0 con 0x7fdd981492c0 2020-11-16T22:35:21.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:21.301+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fdd98068e30 con 0x7fdd981492c0 2020-11-16T22:35:22.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.185+0000 7fdd94ff9700 1 -- 172.21.15.181:0/710461083 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v109) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdd90046cd0 con 0x7fdd981492c0 2020-11-16T22:35:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 msgr2=0x7fdd74057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 0x7fdd74057550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fdd80006e30 tx=0x7fdd80015000).stop 2020-11-16T22:35:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 msgr2=0x7fdd9813cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 0x7fdd9813cbe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fdd9000ca60 tx=0x7fdd90007e20).stop 2020-11-16T22:35:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 shutdown_connections 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd74055090 0x7fdd74057550 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd981492c0 0x7fdd9813cbe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd980816e0 0x7fdd9816ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 --2- 172.21.15.181:0/710461083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd98145a30 0x7fdd9813c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 >> 172.21.15.181:0/710461083 conn(0x7fdd980d68c0 msgr2=0x7fdd98080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 shutdown_connections 2020-11-16T22:35:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.189+0000 7fdd9d2b6700 1 -- 172.21.15.181:0/710461083 wait complete. 2020-11-16T22:35:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2020-11-16T22:35:22.195 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:35:22.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.341+0000 7fddcdced700 1 Processor -- start 2020-11-16T22:35:22.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.341+0000 7fddcdced700 1 -- start start 2020-11-16T22:35:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.341+0000 7fddcdced700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fddc8065bf0 0x7fddc8065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc80664c0 con 0x7fddc8163ce0 2020-11-16T22:35:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc8060270 con 0x7fddc8065bf0 2020-11-16T22:35:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8060510 con 0x7fddc8164190 2020-11-16T22:35:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fddc8065bf0 0x7fddc8065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44446/0 (socket says 172.21.15.181:44446) 2020-11-16T22:35:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 -- 172.21.15.181:0/3877360904 learned_addr learned my addr 172.21.15.181:0/3877360904 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 -- 172.21.15.181:0/3877360904 >> v2:172.21.15.181:3300/0 conn(0x7fddc8065bf0 msgr2=0x7fddc8065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 --2- 172.21.15.181:0/3877360904 >> v2:172.21.15.181:3300/0 conn(0x7fddc8065bf0 0x7fddc8065fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 -- 172.21.15.181:0/3877360904 >> v1:172.21.15.175:6789/0 conn(0x7fddc8163ce0 legacy=0x7fddc80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 -- 172.21.15.181:0/3877360904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80607b0 con 0x7fddc8164190 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc7fff700 1 --2- 172.21.15.181:0/3877360904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fddc0009d50 tx=0x7fddc0009290).ready entity=mon.2 client_cookie=37cf134718b9a42 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc67fc700 1 -- 172.21.15.181:0/3877360904 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000bc50 con 0x7fddc8164190 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc67fc700 1 -- 172.21.15.181:0/3877360904 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000bdb0 con 0x7fddc8164190 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddc67fc700 1 -- 172.21.15.181:0/3877360904 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0011610 con 0x7fddc8164190 2020-11-16T22:35:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- 172.21.15.181:0/3877360904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 msgr2=0x7fddc80cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 --2- 172.21.15.181:0/3877360904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fddc0009d50 tx=0x7fddc0009290).stop 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- 172.21.15.181:0/3877360904 shutdown_connections 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 --2- 172.21.15.181:0/3877360904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8164190 0x7fddc80cf170 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 --2- 172.21.15.181:0/3877360904 >> v2:172.21.15.181:3300/0 conn(0x7fddc8065bf0 0x7fddc8065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- 172.21.15.181:0/3877360904 >> 172.21.15.181:0/3877360904 conn(0x7fddc80cf770 msgr2=0x7fddc80cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- 172.21.15.181:0/3877360904 shutdown_connections 2020-11-16T22:35:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.345+0000 7fddcdced700 1 -- 172.21.15.181:0/3877360904 wait complete. 2020-11-16T22:35:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.349+0000 7fddcdced700 1 Processor -- start 2020-11-16T22:35:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.349+0000 7fddcdced700 1 -- start start 2020-11-16T22:35:22.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.349+0000 7fddcdced700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.349+0000 7fddcdced700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 0x7fddc815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc8164190 0x7fddc80613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc815d790 con 0x7fddc8163ce0 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc816fc70 con 0x7fddc8164190 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc816ff10 con 0x7fddc8065bf0 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44448/0 (socket says 172.21.15.181:44448) 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 -- 172.21.15.181:0/1716409531 learned_addr learned my addr 172.21.15.181:0/1716409531 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc7fff700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc8164190 0x7fddc80613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc6ffd700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 0x7fddc815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 -- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc8164190 msgr2=0x7fddc80613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc8164190 0x7fddc80613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 -- 172.21.15.181:0/1716409531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 msgr2=0x7fddc815d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 0x7fddc815d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 -- 172.21.15.181:0/1716409531 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc81701b0 con 0x7fddc8065bf0 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc6ffd700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 0x7fddc815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddc77fe700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fddb800eba0 tx=0x7fddb8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800c9b0 con 0x7fddc8065bf0 2020-11-16T22:35:22.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800cb10 con 0x7fddc8065bf0 2020-11-16T22:35:22.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc8170450 con 0x7fddc8065bf0 2020-11-16T22:35:22.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800ce90 con 0x7fddc8065bf0 2020-11-16T22:35:22.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc813f090 con 0x7fddc8065bf0 2020-11-16T22:35:22.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc8069eb0 con 0x7fddc8065bf0 2020-11-16T22:35:22.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.353+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fddb8012600 con 0x7fddc8065bf0 2020-11-16T22:35:22.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.357+0000 7fddbf7fe700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 0x7fdd9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.357+0000 7fddc6ffd700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 0x7fdd9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.357+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fddb806e1f0 con 0x7fddc8065bf0 2020-11-16T22:35:22.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.357+0000 7fddc6ffd700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 0x7fdd9c057520 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fddc8140800 tx=0x7fddb4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.361+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fddb8044350 con 0x7fddc8065bf0 2020-11-16T22:35:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.565+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fddc8162700 con 0x7fddc8065bf0 2020-11-16T22:35:22.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.577+0000 7fddbf7fe700 1 -- 172.21.15.181:0/1716409531 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v112) v1 ==== 189+0+0 (secure 0 0 0) 0x7fddb8047970 con 0x7fddc8065bf0 2020-11-16T22:35:22.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 msgr2=0x7fdd9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 0x7fdd9c057520 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fddc8140800 tx=0x7fddb4009040).stop 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 msgr2=0x7fddc816f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fddb800eba0 tx=0x7fddb8006df0).stop 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 shutdown_connections 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fdd9c055060 0x7fdd9c057520 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.571 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc8164190 0x7fddc80613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc8065bf0 0x7fddc816f680 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 --2- 172.21.15.181:0/1716409531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddc8163ce0 0x7fddc815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.581+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 >> 172.21.15.181:0/1716409531 conn(0x7fddc80cf770 msgr2=0x7fddc8162d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:22.572 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.585+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 shutdown_connections 2020-11-16T22:35:22.573 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.585+0000 7fddcdced700 1 -- 172.21.15.181:0/1716409531 wait complete. 2020-11-16T22:35:22.579 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 15 and data pool 16 2020-11-16T22:35:22.590 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:22.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 Processor -- start 2020-11-16T22:35:22.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 -- start start 2020-11-16T22:35:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f73e41461a0 0x7f73e4061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e40c8420 con 0x7f73e40615f0 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e40c86c0 con 0x7f73e41461a0 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40c8960 con 0x7f73e4145dc0 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44462/0 (socket says 172.21.15.181:44462) 2020-11-16T22:35:22.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e259c700 1 -- 172.21.15.181:0/4193537568 learned_addr learned my addr 172.21.15.181:0/4193537568 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.713+0000 7f73e1d9b700 1 --2- 172.21.15.181:0/4193537568 >> v2:172.21.15.175:3300/0 conn(0x7f73e41461a0 0x7f73e4061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e259c700 1 -- 172.21.15.181:0/4193537568 >> v2:172.21.15.175:3300/0 conn(0x7f73e41461a0 msgr2=0x7f73e4061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e259c700 1 --2- 172.21.15.181:0/4193537568 >> v2:172.21.15.175:3300/0 conn(0x7f73e41461a0 0x7f73e4061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e259c700 1 -- 172.21.15.181:0/4193537568 >> v1:172.21.15.175:6789/0 conn(0x7f73e40615f0 legacy=0x7f73e40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e259c700 1 -- 172.21.15.181:0/4193537568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40c8c00 con 0x7f73e4145dc0 2020-11-16T22:35:22.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e259c700 1 --2- 172.21.15.181:0/4193537568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f73d400ba20 tx=0x7f73d4006eb0).ready entity=mon.2 client_cookie=f4fa63eaf185f05a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e159a700 1 -- 172.21.15.181:0/4193537568 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d40097d0 con 0x7f73e4145dc0 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e159a700 1 -- 172.21.15.181:0/4193537568 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d4009930 con 0x7f73e4145dc0 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e159a700 1 -- 172.21.15.181:0/4193537568 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4009cb0 con 0x7f73e4145dc0 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 -- 172.21.15.181:0/4193537568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 msgr2=0x7f73e40c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 --2- 172.21.15.181:0/4193537568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f73d400ba20 tx=0x7f73d4006eb0).stop 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 -- 172.21.15.181:0/4193537568 shutdown_connections 2020-11-16T22:35:22.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 --2- 172.21.15.181:0/4193537568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e4145dc0 0x7f73e40c7af0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 --2- 172.21.15.181:0/4193537568 >> v2:172.21.15.175:3300/0 conn(0x7f73e41461a0 0x7f73e4061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 -- 172.21.15.181:0/4193537568 >> 172.21.15.181:0/4193537568 conn(0x7f73e40cf630 msgr2=0x7f73e40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:22.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 -- 172.21.15.181:0/4193537568 shutdown_connections 2020-11-16T22:35:22.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.717+0000 7f73e359e700 1 -- 172.21.15.181:0/4193537568 wait complete. 2020-11-16T22:35:22.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 Processor -- start 2020-11-16T22:35:22.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- start start 2020-11-16T22:35:22.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 0x7f73e40cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40b8c40 con 0x7f73e40615f0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40b5b20 con 0x7f73e4145dc0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40b5dc0 con 0x7f73e41461a0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 0x7f73e40cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:50628/0 (socket says 172.21.15.181:50628) 2020-11-16T22:35:22.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36628/0 (socket says 172.21.15.181:36628) 2020-11-16T22:35:22.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 0x7f73e40cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44472/0 (socket says 172.21.15.181:44472) 2020-11-16T22:35:22.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 -- 172.21.15.181:0/4115392158 learned_addr learned my addr 172.21.15.181:0/4115392158 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 -- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 msgr2=0x7f73e40cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 0x7f73e40cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 -- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 msgr2=0x7f73e40cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 -- 172.21.15.181:0/4115392158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40b6060 con 0x7f73e40615f0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e1d9b700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e259c700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f73d400bee0 tx=0x7f73d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4011d50 con 0x7f73e40615f0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d401e460 con 0x7f73e40615f0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e40b6300 con 0x7f73e40615f0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d401e7e0 con 0x7f73e40615f0 2020-11-16T22:35:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.721+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e40b6ae0 con 0x7f73e40615f0 2020-11-16T22:35:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e4044f60 con 0x7f73e40615f0 2020-11-16T22:35:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f73d4025030 con 0x7f73e40615f0 2020-11-16T22:35:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73d27fc700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 0x7f73b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f73d406f340 con 0x7f73e40615f0 2020-11-16T22:35:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73e1d9b700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 0x7f73b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.725+0000 7f73e1d9b700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 0x7f73b8057520 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f73e40cc370 tx=0x7f73d8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.737+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f73d4045650 con 0x7f73e40615f0 2020-11-16T22:35:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.889+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73e405c310 con 0x7f73e40615f0 2020-11-16T22:35:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.893+0000 7f73d27fc700 1 -- 172.21.15.181:0/4115392158 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16899 (secure 0 0 0) 0x7f73d4031220 con 0x7f73e40615f0 2020-11-16T22:35:22.881 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:22.881 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":110,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:22.567556+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 msgr2=0x7f73b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 0x7f73b8057520 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f73e40cc370 tx=0x7f73d8006df0).stop 2020-11-16T22:35:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 msgr2=0x7f73e40b8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f73d400bee0 tx=0x7f73d4015000).stop 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 shutdown_connections 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f73b8055060 0x7f73b8057520 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73e41461a0 0x7f73e40cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73e40615f0 0x7f73e40b8750 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 --2- 172.21.15.181:0/4115392158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e4145dc0 0x7f73e40cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 >> 172.21.15.181:0/4115392158 conn(0x7f73e40cf630 msgr2=0x7f73e41424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 shutdown_connections 2020-11-16T22:35:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:22.897+0000 7f73e359e700 1 -- 172.21.15.181:0/4115392158 wait complete. 2020-11-16T22:35:22.902 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:23.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 Processor -- start 2020-11-16T22:35:23.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 -- start start 2020-11-16T22:35:23.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa01c0b7680 0x7fa01c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0c9800 con 0x7fa01c1606c0 2020-11-16T22:35:23.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01c0c9aa0 con 0x7fa01c0b7680 2020-11-16T22:35:23.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa0234a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0c9d40 con 0x7fa01c160aa0 2020-11-16T22:35:23.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa022ca8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa01c0b7680 0x7fa01c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa022ca8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa01c0b7680 0x7fa01c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36640/0 (socket says 172.21.15.181:36640) 2020-11-16T22:35:23.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa022ca8700 1 -- 172.21.15.181:0/1947617427 learned_addr learned my addr 172.21.15.181:0/1947617427 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:23.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.037+0000 7fa021ca6700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa022ca8700 1 -- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 msgr2=0x7fa01c0b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:23.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa022ca8700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c0b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa022ca8700 1 -- 172.21.15.181:0/1947617427 >> v1:172.21.15.175:6789/0 conn(0x7fa01c1606c0 legacy=0x7fa01c0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa022ca8700 1 -- 172.21.15.181:0/1947617427 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0c9fe0 con 0x7fa01c0b7680 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa022ca8700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c0b7680 0x7fa01c0cdff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fa014009d50 tx=0x7fa014009290).ready entity=mon.1 client_cookie=cf50c8a6c40ada6b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0214a5700 1 -- 172.21.15.181:0/1947617427 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400bbc0 con 0x7fa01c0b7680 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0214a5700 1 -- 172.21.15.181:0/1947617427 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01400bd20 con 0x7fa01c0b7680 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0214a5700 1 -- 172.21.15.181:0/1947617427 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014011610 con 0x7fa01c0b7680 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 -- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c0b7680 msgr2=0x7fa01c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c0b7680 0x7fa01c0cdff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fa014009d50 tx=0x7fa014009290).stop 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 -- 172.21.15.181:0/1947617427 shutdown_connections 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c0b7680 0x7fa01c0cdff0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 --2- 172.21.15.181:0/1947617427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c0b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 -- 172.21.15.181:0/1947617427 >> 172.21.15.181:0/1947617427 conn(0x7fa01c0d33f0 msgr2=0x7fa01c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 -- 172.21.15.181:0/1947617427 shutdown_connections 2020-11-16T22:35:23.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.041+0000 7fa0234a9700 1 -- 172.21.15.181:0/1947617427 wait complete. 2020-11-16T22:35:23.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 Processor -- start 2020-11-16T22:35:23.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- start start 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 0x7fa01c0628f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c16d7d0 con 0x7fa01c0b7680 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c16da70 con 0x7fa01c1606c0 2020-11-16T22:35:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c170230 con 0x7fa01c160aa0 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa022ca8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa022ca8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44488/0 (socket says 172.21.15.181:44488) 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa022ca8700 1 -- 172.21.15.181:0/4012292247 learned_addr learned my addr 172.21.15.181:0/4012292247 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36644/0 (socket says 172.21.15.181:36644) 2020-11-16T22:35:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0224a7700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 0x7fa01c0628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 -- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 msgr2=0x7fa01c15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 -- 172.21.15.181:0/4012292247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 msgr2=0x7fa01c0628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 0x7fa01c0628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 -- 172.21.15.181:0/4012292247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c1704d0 con 0x7fa01c1606c0 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0224a7700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 0x7fa01c0628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa021ca6700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fa00c00eba0 tx=0x7fa00c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00c970 con 0x7fa01c1606c0 2020-11-16T22:35:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00cad0 con 0x7fa01c1606c0 2020-11-16T22:35:23.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c170770 con 0x7fa01c1606c0 2020-11-16T22:35:23.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00ce50 con 0x7fa01c1606c0 2020-11-16T22:35:23.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.045+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c170eb0 con 0x7fa01c1606c0 2020-11-16T22:35:23.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c044f60 con 0x7fa01c1606c0 2020-11-16T22:35:23.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa00c012600 con 0x7fa01c1606c0 2020-11-16T22:35:23.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa00a7fc700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 0x7f9ff8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa00c06e100 con 0x7fa01c1606c0 2020-11-16T22:35:23.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa0224a7700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 0x7f9ff8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.049+0000 7fa0224a7700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 0x7f9ff8057520 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa018007f90 tx=0x7fa018009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.057+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa00c044300 con 0x7fa01c1606c0 2020-11-16T22:35:23.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa01c068fb0 con 0x7fa01c1606c0 2020-11-16T22:35:23.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa00a7fc700 1 -- 172.21.15.181:0/4012292247 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16899 (secure 0 0 0) 0x7fa00c047920 con 0x7fa01c1606c0 2020-11-16T22:35:23.201 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:23.201 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":110,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:22.567556+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:23.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 msgr2=0x7f9ff8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 0x7f9ff8057520 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fa018007f90 tx=0x7fa018009040).stop 2020-11-16T22:35:23.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 msgr2=0x7fa01c15e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fa00c00eba0 tx=0x7fa00c006df0).stop 2020-11-16T22:35:23.203 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 shutdown_connections 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9ff8055060 0x7f9ff8057520 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01c160aa0 0x7fa01c15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa01c0b7680 0x7fa01c0628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 --2- 172.21.15.181:0/4012292247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa01c1606c0 0x7fa01c15e5b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.213+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 >> 172.21.15.181:0/4012292247 conn(0x7fa01c0d33f0 msgr2=0x7fa01c0c8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.217+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 shutdown_connections 2020-11-16T22:35:23.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.217+0000 7fa0234a9700 1 -- 172.21.15.181:0/4012292247 wait complete. 2020-11-16T22:35:23.217 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:35:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 Processor -- start 2020-11-16T22:35:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- start start 2020-11-16T22:35:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faef015f5b0 0x7faef00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef0160e40 con 0x7faef00615f0 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef01609a0 con 0x7faef015f5b0 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef0160ba0 con 0x7faef00d1830 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeef7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faef015f5b0 0x7faef00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44494/0 (socket says 172.21.15.181:44494) 2020-11-16T22:35:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 -- 172.21.15.181:0/2088269048 learned_addr learned my addr 172.21.15.181:0/2088269048 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 -- 172.21.15.181:0/2088269048 >> v2:172.21.15.181:3300/0 conn(0x7faef015f5b0 msgr2=0x7faef00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 --2- 172.21.15.181:0/2088269048 >> v2:172.21.15.181:3300/0 conn(0x7faef015f5b0 0x7faef00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 -- 172.21.15.181:0/2088269048 >> v1:172.21.15.175:6789/0 conn(0x7faef00615f0 legacy=0x7faef00622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 -- 172.21.15.181:0/2088269048 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0060970 con 0x7faef00d1830 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeeffff700 1 --2- 172.21.15.181:0/2088269048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7faee8009d50 tx=0x7faee8009290).ready entity=mon.2 client_cookie=905ddc32c2d21f5f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeee7fc700 1 -- 172.21.15.181:0/2088269048 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800bbc0 con 0x7faef00d1830 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeee7fc700 1 -- 172.21.15.181:0/2088269048 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee800bd20 con 0x7faef00d1830 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faeee7fc700 1 -- 172.21.15.181:0/2088269048 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8011610 con 0x7faef00d1830 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- 172.21.15.181:0/2088269048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 msgr2=0x7faef00629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 --2- 172.21.15.181:0/2088269048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7faee8009d50 tx=0x7faee8009290).stop 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- 172.21.15.181:0/2088269048 shutdown_connections 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 --2- 172.21.15.181:0/2088269048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00d1830 0x7faef00629b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 --2- 172.21.15.181:0/2088269048 >> v2:172.21.15.181:3300/0 conn(0x7faef015f5b0 0x7faef00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.361+0000 7faef5b79700 1 -- 172.21.15.181:0/2088269048 >> 172.21.15.181:0/2088269048 conn(0x7faef00d69a0 msgr2=0x7faef00d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:23.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- 172.21.15.181:0/2088269048 shutdown_connections 2020-11-16T22:35:23.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- 172.21.15.181:0/2088269048 wait complete. 2020-11-16T22:35:23.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 Processor -- start 2020-11-16T22:35:23.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- start start 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef00d1830 0x7faef015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faef015f5b0 0x7faef015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef016f930 con 0x7faef00d1830 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef016fbd0 con 0x7faef015f5b0 2020-11-16T22:35:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef016fe70 con 0x7faef00615f0 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44498/0 (socket says 172.21.15.181:44498) 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 -- 172.21.15.181:0/1752048142 learned_addr learned my addr 172.21.15.181:0/1752048142 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeeffff700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faef015f5b0 0x7faef015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeeeffd700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef00d1830 0x7faef015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 -- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faef015f5b0 msgr2=0x7faef015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faef015f5b0 0x7faef015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 -- 172.21.15.181:0/1752048142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef00d1830 msgr2=0x7faef015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef00d1830 0x7faef015b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 -- 172.21.15.181:0/1752048142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef00ba7a0 con 0x7faef00615f0 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faeef7fe700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7faee000eb20 tx=0x7faee0006d90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0016070 con 0x7faef00615f0 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef00baa40 con 0x7faef00615f0 2020-11-16T22:35:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000c920 con 0x7faef00615f0 2020-11-16T22:35:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000b350 con 0x7faef00615f0 2020-11-16T22:35:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.365+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef00b9af0 con 0x7faef00615f0 2020-11-16T22:35:23.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef00b9da0 con 0x7faef00615f0 2020-11-16T22:35:23.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faee001b410 con 0x7faef00615f0 2020-11-16T22:35:23.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faecf7fe700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 0x7faec8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:23.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faeeeffd700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 0x7faec8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:23.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faee006df10 con 0x7faef00615f0 2020-11-16T22:35:23.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.369+0000 7faeeeffd700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 0x7faec8057520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7faef0061a50 tx=0x7faee4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:23.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.377+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faee0044110 con 0x7faef00615f0 2020-11-16T22:35:23.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:23.573+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faef0170000 con 0x7faef00615f0 2020-11-16T22:35:24.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2020-11-16T22:35:24.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2020-11-16T22:35:24.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2020-11-16T22:35:24.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2020-11-16T22:35:24.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g has been restored 2020-11-16T22:35:24.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i has been restored 2020-11-16T22:35:24.577 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.589+0000 7faecf7fe700 1 -- 172.21.15.181:0/1752048142 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v114) v1 ==== 118+0+0 (secure 0 0 0) 0x7faee0047730 con 0x7faef00615f0 2020-11-16T22:35:24.581 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 msgr2=0x7faec8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 0x7faec8057520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7faef0061a50 tx=0x7faee4009040).stop 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 msgr2=0x7faef016f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7faee000eb20 tx=0x7faee0006d90).stop 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 shutdown_connections 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faec8055060 0x7faec8057520 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faef015f5b0 0x7faef015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faef00615f0 0x7faef016f0a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 --2- 172.21.15.181:0/1752048142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef00d1830 0x7faef015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 >> 172.21.15.181:0/1752048142 conn(0x7faef00d69a0 msgr2=0x7faef0061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 shutdown_connections 2020-11-16T22:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.593+0000 7faef5b79700 1 -- 172.21.15.181:0/1752048142 wait complete. 2020-11-16T22:35:24.599 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:24.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 Processor -- start 2020-11-16T22:35:24.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 -- start start 2020-11-16T22:35:24.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:24.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c063b10 con 0x7fc77c160530 2020-11-16T22:35:24.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c062700 con 0x7fc77c142660 2020-11-16T22:35:24.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc780840700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0629a0 con 0x7fc77c142a40 2020-11-16T22:35:24.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:24.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44506/0 (socket says 172.21.15.181:44506) 2020-11-16T22:35:24.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 -- 172.21.15.181:0/2031350380 learned_addr learned my addr 172.21.15.181:0/2031350380 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:24.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc77b7fe700 1 --1- 172.21.15.181:0/2031350380 >> v1:172.21.15.181:6789/0 conn(0x7fc77c142660 0x7fc77c063ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50970/0 (socket says 172.21.15.181:50970) 2020-11-16T22:35:24.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 -- 172.21.15.181:0/2031350380 >> v1:172.21.15.181:6789/0 conn(0x7fc77c142660 legacy=0x7fc77c063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 -- 172.21.15.181:0/2031350380 >> v1:172.21.15.175:6789/0 conn(0x7fc77c160530 legacy=0x7fc77c061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 -- 172.21.15.181:0/2031350380 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c062c40 con 0x7fc77c142a40 2020-11-16T22:35:24.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc773fff700 1 --2- 172.21.15.181:0/2031350380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc76400ba20 tx=0x7fc764006eb0).ready entity=mon.2 client_cookie=465bdb1dd7aebdcb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:24.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc77affd700 1 -- 172.21.15.181:0/2031350380 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640098d0 con 0x7fc77c142a40 2020-11-16T22:35:24.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc77affd700 1 -- 172.21.15.181:0/2031350380 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764009a30 con 0x7fc77c142a40 2020-11-16T22:35:24.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.741+0000 7fc77affd700 1 -- 172.21.15.181:0/2031350380 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764009db0 con 0x7fc77c142a40 2020-11-16T22:35:24.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 -- 172.21.15.181:0/2031350380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 msgr2=0x7fc77c063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 --2- 172.21.15.181:0/2031350380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc76400ba20 tx=0x7fc764006eb0).stop 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 -- 172.21.15.181:0/2031350380 shutdown_connections 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 --2- 172.21.15.181:0/2031350380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142a40 0x7fc77c063620 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 -- 172.21.15.181:0/2031350380 >> 172.21.15.181:0/2031350380 conn(0x7fc77c0d2dc0 msgr2=0x7fc77c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 -- 172.21.15.181:0/2031350380 shutdown_connections 2020-11-16T22:35:24.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.745+0000 7fc780840700 1 -- 172.21.15.181:0/2031350380 wait complete. 2020-11-16T22:35:24.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 Processor -- start 2020-11-16T22:35:24.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- start start 2020-11-16T22:35:24.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142660 0x7fc77c0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 0x7fc77c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c170350 con 0x7fc77c142a40 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1700b0 con 0x7fc77c160530 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1430e0 con 0x7fc77c142660 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:24.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142660 0x7fc77c0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:24.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36670/0 (socket says 172.21.15.181:36670) 2020-11-16T22:35:24.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 -- 172.21.15.181:0/1701727611 learned_addr learned my addr 172.21.15.181:0/1701727611 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:24.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc773fff700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 0x7fc77c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:24.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 -- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142660 msgr2=0x7fc77c0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142660 0x7fc77c0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.739 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 -- 172.21.15.181:0/1701727611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 msgr2=0x7fc77c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 0x7fc77c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 -- 172.21.15.181:0/1701727611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c143380 con 0x7fc77c160530 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc773fff700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 0x7fc77c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc77bfff700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fc77400ca60 tx=0x7fc774007e70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400a770 con 0x7fc77c160530 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400a8d0 con 0x7fc77c160530 2020-11-16T22:35:24.740 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c143620 con 0x7fc77c160530 2020-11-16T22:35:24.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400ac70 con 0x7fc77c160530 2020-11-16T22:35:24.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.749+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0b4af0 con 0x7fc77c160530 2020-11-16T22:35:24.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c044f60 con 0x7fc77c160530 2020-11-16T22:35:24.741 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc774018020 con 0x7fc77c160530 2020-11-16T22:35:24.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc7737fe700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 0x7fc758057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:24.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc77406d970 con 0x7fc77c160530 2020-11-16T22:35:24.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc77b7fe700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 0x7fc758057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:24.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.753+0000 7fc77b7fe700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 0x7fc758057520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc76c007270 tx=0x7fc76c007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:24.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.761+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc774043bd0 con 0x7fc77c160530 2020-11-16T22:35:24.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.965+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc77c063b10 con 0x7fc77c160530 2020-11-16T22:35:24.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.965+0000 7fc7737fe700 1 -- 172.21.15.181:0/1701727611 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4636 (secure 0 0 0) 0x7fc77c063b10 con 0x7fc77c160530 2020-11-16T22:35:24.954 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:24.954 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":114,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8040,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8043,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:24.582056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:24.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 msgr2=0x7fc758057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 0x7fc758057520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc76c007270 tx=0x7fc76c007b80).stop 2020-11-16T22:35:24.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 msgr2=0x7fc77c172000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:24.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fc77400ca60 tx=0x7fc774007e70).stop 2020-11-16T22:35:24.957 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 shutdown_connections 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc758055060 0x7fc758057520 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c160530 0x7fc77c172000 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c142660 0x7fc77c0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 --2- 172.21.15.181:0/1701727611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc77c142a40 0x7fc77c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 >> 172.21.15.181:0/1701727611 conn(0x7fc77c0d2dc0 msgr2=0x7fc77c0667d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 shutdown_connections 2020-11-16T22:35:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:24.969+0000 7fc780840700 1 -- 172.21.15.181:0/1701727611 wait complete. 2020-11-16T22:35:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 114 2020-11-16T22:35:24.974 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:25.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 Processor -- start 2020-11-16T22:35:25.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 -- start start 2020-11-16T22:35:25.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 0x7f6d1c164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0c8db0 con 0x7f6d1c161860 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c8330 con 0x7f6d1c161480 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c85d0 con 0x7f6d1c164e10 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44520/0 (socket says 172.21.15.181:44520) 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/152409617 learned_addr learned my addr 172.21.15.181:0/152409617 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1affd700 1 --2- 172.21.15.181:0/152409617 >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 0x7f6d1c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/152409617 >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 msgr2=0x7f6d1c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 --2- 172.21.15.181:0/152409617 >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 0x7f6d1c164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/152409617 >> v1:172.21.15.175:6789/0 conn(0x7f6d1c161860 legacy=0x7f6d1c062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/152409617 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0c8870 con 0x7f6d1c164e10 2020-11-16T22:35:25.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1affd700 1 --2- 172.21.15.181:0/152409617 >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 0x7f6d1c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:35:25.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.105+0000 7f6d1a7fc700 1 --2- 172.21.15.181:0/152409617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f6d1c0c8260 tx=0x7f6d04006eb0).ready entity=mon.2 client_cookie=30d0d155777bf896 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d19ffb700 1 -- 172.21.15.181:0/152409617 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04009890 con 0x7f6d1c164e10 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d19ffb700 1 -- 172.21.15.181:0/152409617 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d040099f0 con 0x7f6d1c164e10 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d19ffb700 1 -- 172.21.15.181:0/152409617 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04009d70 con 0x7f6d1c164e10 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- 172.21.15.181:0/152409617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 msgr2=0x7f6d1c062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/152409617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f6d1c0c8260 tx=0x7f6d04006eb0).stop 2020-11-16T22:35:25.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- 172.21.15.181:0/152409617 shutdown_connections 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/152409617 >> v2:172.21.15.175:3300/0 conn(0x7f6d1c161480 0x7f6d1c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/152409617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c164e10 0x7f6d1c062250 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- 172.21.15.181:0/152409617 >> 172.21.15.181:0/152409617 conn(0x7f6d1c0d2dc0 msgr2=0x7f6d1c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- 172.21.15.181:0/152409617 shutdown_connections 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- 172.21.15.181:0/152409617 wait complete. 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 Processor -- start 2020-11-16T22:35:25.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- start start 2020-11-16T22:35:25.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d1c161860 0x7f6d1c166960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c166e50 0x7f6d1c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b58d0 con 0x7f6d1c161860 2020-11-16T22:35:25.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b5b70 con 0x7f6d1c166e50 2020-11-16T22:35:25.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.109+0000 7f6d1bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b5e10 con 0x7f6d1c161480 2020-11-16T22:35:25.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44526/0 (socket says 172.21.15.181:44526) 2020-11-16T22:35:25.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 -- 172.21.15.181:0/2106423775 learned_addr learned my addr 172.21.15.181:0/2106423775 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1b7fe700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c166e50 0x7f6d1c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1a7fc700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d1c161860 0x7f6d1c166960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 -- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c166e50 msgr2=0x7f6d1c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c166e50 0x7f6d1c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 -- 172.21.15.181:0/2106423775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d1c161860 msgr2=0x7f6d1c166960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d1c161860 0x7f6d1c166960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 -- 172.21.15.181:0/2106423775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0b60b0 con 0x7f6d1c161480 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1affd700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6d0c00c8f0 tx=0x7f6d0c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00e5d0 con 0x7f6d1c161480 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00e730 con 0x7f6d1c161480 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0b6350 con 0x7f6d1c161480 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00eab0 con 0x7f6d1c161480 2020-11-16T22:35:25.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c0caf90 con 0x7f6d1c161480 2020-11-16T22:35:25.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c044f60 con 0x7f6d1c161480 2020-11-16T22:35:25.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6d0c01b460 con 0x7f6d1c161480 2020-11-16T22:35:25.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 0x7f6cec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d1a7fc700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 0x7f6cec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.113+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d0c06df80 con 0x7f6d1c161480 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.117+0000 7f6d1a7fc700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 0x7f6cec057520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f6d04006ef0 tx=0x7f6d04015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.121+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6d0c044070 con 0x7f6d1c161480 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.269+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c0b1070 con 0x7f6d1c161480 2020-11-16T22:35:25.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d12ffd700 1 -- 172.21.15.181:0/2106423775 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f6d0c00ee60 con 0x7f6d1c161480 2020-11-16T22:35:25.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 msgr2=0x7f6cec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:25.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 0x7f6cec057520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f6d04006ef0 tx=0x7f6d04015000).stop 2020-11-16T22:35:25.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 msgr2=0x7f6d1c0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:25.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6d0c00c8f0 tx=0x7f6d0c008040).stop 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 shutdown_connections 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6cec055060 0x7f6cec057520 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c166e50 0x7f6d1c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1c161480 0x7f6d1c0b5280 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 --2- 172.21.15.181:0/2106423775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d1c161860 0x7f6d1c166960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:25.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 >> 172.21.15.181:0/2106423775 conn(0x7f6d1c0d2dc0 msgr2=0x7f6d1c164410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:25.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 shutdown_connections 2020-11-16T22:35:25.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:25.273+0000 7f6d1bfff700 1 -- 172.21.15.181:0/2106423775 wait complete. 2020-11-16T22:35:25.512 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:25.512 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:23.571640+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:25.919 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:35:26.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.277+0000 7f35d979f700 1 Processor -- start 2020-11-16T22:35:26.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.277+0000 7f35d979f700 1 -- start start 2020-11-16T22:35:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35d4146230 0x7f35d40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d415cc40 con 0x7f35d4145e50 2020-11-16T22:35:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cf4d0 con 0x7f35d4146230 2020-11-16T22:35:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40cf770 con 0x7f35d40b4df0 2020-11-16T22:35:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35d4146230 0x7f35d40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44538/0 (socket says 172.21.15.181:44538) 2020-11-16T22:35:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d37fe700 1 -- 172.21.15.181:0/2044601682 learned_addr learned my addr 172.21.15.181:0/2044601682 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 -- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 msgr2=0x7f35d40ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 -- 172.21.15.181:0/2044601682 >> v1:172.21.15.175:6789/0 conn(0x7f35d4145e50 legacy=0x7f35d415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 -- 172.21.15.181:0/2044601682 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40cfa10 con 0x7f35d4146230 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d37fe700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35ca7fc700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4146230 0x7f35d40b4900 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f35bc00b9a0 tx=0x7f35bc006df0).ready entity=mon.1 client_cookie=9316885d0c4d644d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d27fc700 1 -- 172.21.15.181:0/2044601682 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc009840 con 0x7f35d4146230 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d27fc700 1 -- 172.21.15.181:0/2044601682 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc0099a0 con 0x7f35d4146230 2020-11-16T22:35:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d27fc700 1 -- 172.21.15.181:0/2044601682 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc009d20 con 0x7f35d4146230 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4146230 msgr2=0x7f35d40b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4146230 0x7f35d40b4900 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f35bc00b9a0 tx=0x7f35bc006df0).stop 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- 172.21.15.181:0/2044601682 shutdown_connections 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d40ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 --2- 172.21.15.181:0/2044601682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4146230 0x7f35d40b4900 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- 172.21.15.181:0/2044601682 >> 172.21.15.181:0/2044601682 conn(0x7f35d40d3730 msgr2=0x7f35d40d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- 172.21.15.181:0/2044601682 shutdown_connections 2020-11-16T22:35:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.281+0000 7f35d979f700 1 -- 172.21.15.181:0/2044601682 wait complete. 2020-11-16T22:35:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 Processor -- start 2020-11-16T22:35:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- start start 2020-11-16T22:35:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d413f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4146230 0x7f35d41431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41739b0 con 0x7f35d4146230 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4141f30 con 0x7f35d4145e50 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4173710 con 0x7f35d40b4df0 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d413f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36702/0 (socket says 172.21.15.181:36702) 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 -- 172.21.15.181:0/331753878 learned_addr learned my addr 172.21.15.181:0/331753878 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d37fe700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4146230 0x7f35d41431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 -- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 msgr2=0x7f35d413f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d413f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 -- 172.21.15.181:0/331753878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4146230 msgr2=0x7f35d41431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4146230 0x7f35d41431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 -- 172.21.15.181:0/331753878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4142120 con 0x7f35d4145e50 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d2ffd700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d413f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35ca7fc700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f35bc006e30 tx=0x7f35bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011d50 con 0x7f35d4145e50 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc01e460 con 0x7f35d4145e50 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc01e7e0 con 0x7f35d4145e50 2020-11-16T22:35:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d41423c0 con 0x7f35d4145e50 2020-11-16T22:35:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.285+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d4142da0 con 0x7f35d4145e50 2020-11-16T22:35:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d4069bd0 con 0x7f35d4145e50 2020-11-16T22:35:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f35bc025030 con 0x7f35d4145e50 2020-11-16T22:35:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35cb7fe700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35d2ffd700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35bc06f220 con 0x7f35d4145e50 2020-11-16T22:35:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.289+0000 7f35d2ffd700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 0x7f35b0057520 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f35c40016e0 tx=0x7f35c4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.297+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f35bc045530 con 0x7f35d4145e50 2020-11-16T22:35:26.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.501+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f35d4062210 con 0x7f35d4145e50 2020-11-16T22:35:26.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35cb7fe700 1 -- 172.21.15.181:0/331753878 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f35bc023070 con 0x7f35d4145e50 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 msgr2=0x7f35b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 0x7f35b0057520 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f35c40016e0 tx=0x7f35c4009040).stop 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 msgr2=0x7f35d413f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f35bc006e30 tx=0x7f35bc015000).stop 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 shutdown_connections 2020-11-16T22:35:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4146230 0x7f35d41431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35d40b4df0 0x7f35d413f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 --2- 172.21.15.181:0/331753878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d4145e50 0x7f35d413f760 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 >> 172.21.15.181:0/331753878 conn(0x7f35d40d3730 msgr2=0x7f35d40600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 shutdown_connections 2020-11-16T22:35:26.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.513+0000 7f35d979f700 1 -- 172.21.15.181:0/331753878 wait complete. 2020-11-16T22:35:26.504 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2020-11-16T22:35:26.516 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:26.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.649+0000 7f4993f60700 1 Processor -- start 2020-11-16T22:35:26.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.649+0000 7f4993f60700 1 -- start start 2020-11-16T22:35:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4993f60700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f498c061680 0x7f498c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4993f60700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f498c15fa60 0x7f498c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4993f60700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c062b70 con 0x7f498c0612a0 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4993f60700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c15db10 con 0x7f498c061680 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4993f60700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0cd790 con 0x7f498c15fa60 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f498c15fa60 0x7f498c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49914fb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f498c061680 0x7f498c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f498c15fa60 0x7f498c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36710/0 (socket says 172.21.15.181:36710) 2020-11-16T22:35:26.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 -- 172.21.15.181:0/3298262897 learned_addr learned my addr 172.21.15.181:0/3298262897 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:26.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 -- 172.21.15.181:0/3298262897 >> v2:172.21.15.175:3300/0 conn(0x7f498c061680 msgr2=0x7f498c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:26.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 --2- 172.21.15.181:0/3298262897 >> v2:172.21.15.175:3300/0 conn(0x7f498c061680 0x7f498c15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 -- 172.21.15.181:0/3298262897 >> v1:172.21.15.175:6789/0 conn(0x7f498c0612a0 legacy=0x7f498c065fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 -- 172.21.15.181:0/3298262897 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0cda30 con 0x7f498c15fa60 2020-11-16T22:35:26.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f49924fd700 1 --2- 172.21.15.181:0/3298262897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c0ce6b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f498c0b2e20 tx=0x7f4984006eb0).ready entity=mon.1 client_cookie=265051262cec38 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4990cfa700 1 -- 172.21.15.181:0/3298262897 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984009810 con 0x7f498c15fa60 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4990cfa700 1 -- 172.21.15.181:0/3298262897 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984009970 con 0x7f498c15fa60 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.653+0000 7f4990cfa700 1 -- 172.21.15.181:0/3298262897 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984009cf0 con 0x7f498c15fa60 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 -- 172.21.15.181:0/3298262897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 msgr2=0x7f498c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 --2- 172.21.15.181:0/3298262897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c0ce6b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f498c0b2e20 tx=0x7f4984006eb0).stop 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 -- 172.21.15.181:0/3298262897 shutdown_connections 2020-11-16T22:35:26.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 --2- 172.21.15.181:0/3298262897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c0ce6b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 --2- 172.21.15.181:0/3298262897 >> v2:172.21.15.175:3300/0 conn(0x7f498c061680 0x7f498c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 -- 172.21.15.181:0/3298262897 >> 172.21.15.181:0/3298262897 conn(0x7f498c0cf890 msgr2=0x7f498c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:26.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 -- 172.21.15.181:0/3298262897 shutdown_connections 2020-11-16T22:35:26.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.657+0000 7f4993f60700 1 -- 172.21.15.181:0/3298262897 wait complete. 2020-11-16T22:35:26.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 Processor -- start 2020-11-16T22:35:26.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- start start 2020-11-16T22:35:26.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 0x7f498c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f498c061680 0x7f498c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c064d20 con 0x7f498c061680 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c064fc0 con 0x7f498c15fa60 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c065260 con 0x7f498c0612a0 2020-11-16T22:35:26.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36716/0 (socket says 172.21.15.181:36716) 2020-11-16T22:35:26.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 -- 172.21.15.181:0/793435603 learned_addr learned my addr 172.21.15.181:0/793435603 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:26.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4991cfc700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 0x7f498c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49914fb700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f498c061680 0x7f498c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 -- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 msgr2=0x7f498c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 0x7f498c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 -- 172.21.15.181:0/793435603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f498c061680 msgr2=0x7f498c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f498c061680 0x7f498c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 -- 172.21.15.181:0/793435603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c065500 con 0x7f498c15fa60 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4991cfc700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 0x7f498c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f49924fd700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f498400beb0 tx=0x7f4984015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984006be0 con 0x7f498c15fa60 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49840093a0 con 0x7f498c15fa60 2020-11-16T22:35:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498401d800 con 0x7f498c15fa60 2020-11-16T22:35:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c0657a0 con 0x7f498c15fa60 2020-11-16T22:35:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.661+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0d2870 con 0x7f498c15fa60 2020-11-16T22:35:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c069eb0 con 0x7f498c15fa60 2020-11-16T22:35:26.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4984025030 con 0x7f498c15fa60 2020-11-16T22:35:26.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4979ffb700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 0x7f4964057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:26.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f498406f630 con 0x7f498c15fa60 2020-11-16T22:35:26.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4991cfc700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 0x7f4964057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:26.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.665+0000 7f4991cfc700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 0x7f4964057520 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f4988001d50 tx=0x7f4988009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:26.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.669+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4984045830 con 0x7f498c15fa60 2020-11-16T22:35:26.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f498c16d0a0 con 0x7f498c15fa60 2020-11-16T22:35:26.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4979ffb700 1 -- 172.21.15.181:0/793435603 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4636 (secure 0 0 0) 0x7f4984023070 con 0x7f498c15fa60 2020-11-16T22:35:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:26.861 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":114,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8040,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8043,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:24.582056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 msgr2=0x7f4964057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 0x7f4964057520 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f4988001d50 tx=0x7f4988009040).stop 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 msgr2=0x7f498c150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f498400beb0 tx=0x7f4984015000).stop 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 shutdown_connections 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4964055060 0x7f4964057520 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498c15fa60 0x7f498c150f20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0612a0 0x7f498c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 --2- 172.21.15.181:0/793435603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f498c061680 0x7f498c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:26.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 >> 172.21.15.181:0/793435603 conn(0x7f498c0cf890 msgr2=0x7f498c0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:26.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 shutdown_connections 2020-11-16T22:35:26.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:26.873+0000 7f4993f60700 1 -- 172.21.15.181:0/793435603 wait complete. 2020-11-16T22:35:26.864 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 114 2020-11-16T22:35:27.786 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:35:22.567943+0000', 'modified': '2020-11-16T22:35:24.582056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8101': {'gid': 8101, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2750102703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2750102703}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2750102703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:35:27.787 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:35:27.788 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:35:27.924 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":8101,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.3545503229999998,"mdsmap_epoch":114,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":10.820709565} 2020-11-16T22:35:27.924 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 8101, "mdsmap_epoch": 114, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 5.354550323, "state": "up:active", "uptime": 10.820709565, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:35:27.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:35:27.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2020-11-16T22:35:27.925 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:35:27.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:35:27.926 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:35:27.926 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:35:27.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:35:27.927 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:35:27.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:27.928 INFO:teuthology.orchestra.run.smithi106:> ip addr 2020-11-16T22:35:27.945 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:35:27.945 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:35:27.945 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:35:27.945 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:35:27.946 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2020-11-16T22:35:27.947 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::1069:89ff:fe05:8e38/64 scope link 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.948 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 9a:80:18:3d:1f:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9880:18ff:fe3d:1f5a/64 scope link 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether ba:ee:ae:80:c8:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::b8ee:aeff:fe80:c8b7/64 scope link 2020-11-16T22:35:27.949 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:35:27.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:27.951 INFO:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:35:27.972 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:35:27.985+0000 7ff288533700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:35:28.027 INFO:teuthology.orchestra.run.smithi106.stdout:1 2020-11-16T22:35:28.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:28.031 INFO:teuthology.orchestra.run.smithi106:> route 2020-11-16T22:35:28.048 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2020-11-16T22:35:28.049 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:35:28.049 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:35:28.049 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:35:28.049 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:35:28.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:28.050 INFO:teuthology.orchestra.run.smithi106:> (cd / && exec sudo bash -c ' 2020-11-16T22:35:28.051 INFO:teuthology.orchestra.run.smithi106:> set -e 2020-11-16T22:35:28.051 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:35:28.051 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:35:28.051 INFO:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:35:28.052 INFO:teuthology.orchestra.run.smithi106:> ') 2020-11-16T22:35:28.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:28.150 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:35:28.165 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:35:28.165 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:35:28.166 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:35:28.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:28.167 INFO:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2020-11-16T22:35:28.195 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:35:28.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.341+0000 7f715b59e700 1 Processor -- start 2020-11-16T22:35:28.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.341+0000 7f715b59e700 1 -- start start 2020-11-16T22:35:28.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715b59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c15e2e0 con 0x7f715c15de80 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c0ce460 con 0x7f715c0b2190 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c066a60 con 0x7f715c160690 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f715c0b2190 0x7f715c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51024/0 (socket says 172.21.15.181:51024) 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715a59c700 1 -- 172.21.15.181:0/3682903548 learned_addr learned my addr 172.21.15.181:0/3682903548 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:28.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715ad9d700 1 --2- 172.21.15.181:0/3682903548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892435795 0 0) 0x7f715c0ce460 con 0x7f715c0b2190 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7140003240 con 0x7f715c0b2190 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773178101 0 0) 0x7f715c15e2e0 con 0x7f715c15de80 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715ad9d700 1 -- 172.21.15.181:0/3682903548 >> v1:172.21.15.181:6789/0 conn(0x7f715c0b2190 legacy=0x7f715c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715ad9d700 1 -- 172.21.15.181:0/3682903548 >> v1:172.21.15.175:6789/0 conn(0x7f715c15de80 legacy=0x7f715c15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715ad9d700 1 -- 172.21.15.181:0/3682903548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0ce700 con 0x7f715c160690 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715ad9d700 1 --2- 172.21.15.181:0/3682903548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c066570 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f715400b9d0 tx=0x7f7154006df0).ready entity=mon.2 client_cookie=bcda5d3254d29701 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71540097b0 con 0x7f715c160690 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154009910 con 0x7f715c160690 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.345+0000 7f715959a700 1 -- 172.21.15.181:0/3682903548 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154009c90 con 0x7f715c160690 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- 172.21.15.181:0/3682903548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 msgr2=0x7f715c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 --2- 172.21.15.181:0/3682903548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c066570 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f715400b9d0 tx=0x7f7154006df0).stop 2020-11-16T22:35:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- 172.21.15.181:0/3682903548 shutdown_connections 2020-11-16T22:35:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 --2- 172.21.15.181:0/3682903548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c066570 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- 172.21.15.181:0/3682903548 >> 172.21.15.181:0/3682903548 conn(0x7f715c0cfe90 msgr2=0x7f715c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- 172.21.15.181:0/3682903548 shutdown_connections 2020-11-16T22:35:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- 172.21.15.181:0/3682903548 wait complete. 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 Processor -- start 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.349+0000 7f715b59e700 1 -- start start 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f715c15de80 0x7f715c13cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c151dd0 con 0x7f715c15de80 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c157180 con 0x7f715c0b2190 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c157420 con 0x7f715c160690 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36726/0 (socket says 172.21.15.181:36726) 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 -- 172.21.15.181:0/2002543871 learned_addr learned my addr 172.21.15.181:0/2002543871 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f7159d9b700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f715c15de80 0x7f715c13cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 -- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 msgr2=0x7f715c163500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c163500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 -- 172.21.15.181:0/2002543871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f715c15de80 msgr2=0x7f715c13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f715c15de80 0x7f715c13cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 -- 172.21.15.181:0/2002543871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c1576c0 con 0x7f715c0b2190 2020-11-16T22:35:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715ad9d700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:28.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715a59c700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f714c00ca60 tx=0x7f714c007dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c005b80 con 0x7f715c0b2190 2020-11-16T22:35:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c005ce0 con 0x7f715c0b2190 2020-11-16T22:35:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c157960 con 0x7f715c0b2190 2020-11-16T22:35:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c007340 con 0x7f715c0b2190 2020-11-16T22:35:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c157ea0 con 0x7f715c0b2190 2020-11-16T22:35:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f714c010030 con 0x7f715c0b2190 2020-11-16T22:35:28.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c158260 con 0x7f715c0b2190 2020-11-16T22:35:28.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.353+0000 7f714a7fc700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 0x7f7130057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.357+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f714c06ddb0 con 0x7f715c0b2190 2020-11-16T22:35:28.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.357+0000 7f7159d9b700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 0x7f7130057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.357+0000 7f7159d9b700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 0x7f7130057520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f715c0ce230 tx=0x7f7150007910).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.361+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f714c044010 con 0x7f715c0b2190 2020-11-16T22:35:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.513+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f715c1580a0 con 0x7f715c0b2190 2020-11-16T22:35:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.513+0000 7f714a7fc700 1 -- 172.21.15.181:0/2002543871 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f714c047630 con 0x7f715c0b2190 2020-11-16T22:35:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:28.503 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:23.571640+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:35:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 msgr2=0x7f7130057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 0x7f7130057520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f715c0ce230 tx=0x7f7150007910).stop 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 msgr2=0x7f715c159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f714c00ca60 tx=0x7f714c007dc0).stop 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 shutdown_connections 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7130055060 0x7f7130057520 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c160690 0x7f715c163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c0b2190 0x7f715c159b60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 --2- 172.21.15.181:0/2002543871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f715c15de80 0x7f715c13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 >> 172.21.15.181:0/2002543871 conn(0x7f715c0cfe90 msgr2=0x7f715c15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 shutdown_connections 2020-11-16T22:35:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.517+0000 7f715b59e700 1 -- 172.21.15.181:0/2002543871 wait complete. 2020-11-16T22:35:28.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:26:38.724005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:26:46.352024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:35:20.412645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:35:21.362082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:35:28.518 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:35:28.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.661+0000 7f7dc3fff700 1 Processor -- start 2020-11-16T22:35:28.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.661+0000 7f7dc3fff700 1 -- start start 2020-11-16T22:35:28.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 0x7f7dc40cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc4163970 0x7f7dc4163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc4065190 con 0x7f7dc40600f0 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc40cfc60 con 0x7f7dc40cf390 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc4065430 con 0x7f7dc4163970 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc4163970 0x7f7dc4163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7dc40600f0 0x7f7dc4060580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51042/0 (socket says 172.21.15.181:51042) 2020-11-16T22:35:28.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc4163970 0x7f7dc4163df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36734/0 (socket says 172.21.15.181:36734) 2020-11-16T22:35:28.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc37fe700 1 -- 172.21.15.181:0/3908846308 learned_addr learned my addr 172.21.15.181:0/3908846308 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:28.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc2ffd700 1 --2- 172.21.15.181:0/3908846308 >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 0x7f7dc40cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 -- 172.21.15.181:0/3908846308 >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 msgr2=0x7f7dc40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:28.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 --2- 172.21.15.181:0/3908846308 >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 0x7f7dc40cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 -- 172.21.15.181:0/3908846308 >> v1:172.21.15.181:6789/0 conn(0x7f7dc40600f0 legacy=0x7f7dc4060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 -- 172.21.15.181:0/3908846308 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40656d0 con 0x7f7dc4163970 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc1ffb700 1 -- 172.21.15.181:0/3908846308 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065843708 0 0) 0x7f7dc4065190 con 0x7f7dc40600f0 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc2ffd700 1 --2- 172.21.15.181:0/3908846308 >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 0x7f7dc40cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc27fc700 1 --2- 172.21.15.181:0/3908846308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc4163970 0x7f7dc4163df0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f7dc40ca440 tx=0x7f7dac006df0).ready entity=mon.1 client_cookie=5dbbd1dd34b06946 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc1ffb700 1 -- 172.21.15.181:0/3908846308 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0097b0 con 0x7f7dc4163970 2020-11-16T22:35:28.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc1ffb700 1 -- 172.21.15.181:0/3908846308 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac009910 con 0x7f7dc4163970 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc1ffb700 1 -- 172.21.15.181:0/3908846308 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac009c90 con 0x7f7dc4163970 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3908846308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc4163970 msgr2=0x7f7dc4163df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3908846308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc4163970 0x7f7dc4163df0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f7dc40ca440 tx=0x7f7dac006df0).stop 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3908846308 shutdown_connections 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3908846308 >> v2:172.21.15.175:3300/0 conn(0x7f7dc40cf390 0x7f7dc40cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3908846308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc4163970 0x7f7dc4163df0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3908846308 >> 172.21.15.181:0/3908846308 conn(0x7f7dc40cfe90 msgr2=0x7f7dc40d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:28.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3908846308 shutdown_connections 2020-11-16T22:35:28.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.665+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3908846308 wait complete. 2020-11-16T22:35:28.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.669+0000 7f7dc3fff700 1 Processor -- start 2020-11-16T22:35:28.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.669+0000 7f7dc3fff700 1 -- start start 2020-11-16T22:35:28.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dc40cf390 0x7f7dc413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc4163970 0x7f7dc40d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40d27e0 con 0x7f7dc4163970 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41727a0 con 0x7f7dc40600f0 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4172a40 con 0x7f7dc40cf390 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc27fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dc40cf390 0x7f7dc413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36740/0 (socket says 172.21.15.181:36740) 2020-11-16T22:35:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 -- 172.21.15.181:0/3552463581 learned_addr learned my addr 172.21.15.181:0/3552463581 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc37fe700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc4163970 0x7f7dc40d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 -- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dc40cf390 msgr2=0x7f7dc413ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dc40cf390 0x7f7dc413ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 -- 172.21.15.181:0/3552463581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc4163970 msgr2=0x7f7dc40d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc4163970 0x7f7dc40d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 -- 172.21.15.181:0/3552463581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4172ce0 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc2ffd700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f7db400cb50 tx=0x7f7db4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4007710 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4007870 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc4172f80 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4007bf0 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc40d2030 con 0x7f7dc40600f0 2020-11-16T22:35:28.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc4044f60 con 0x7f7dc40600f0 2020-11-16T22:35:28.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7db401b460 con 0x7f7dc40600f0 2020-11-16T22:35:28.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.673+0000 7f7dbaffd700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 0x7f7da0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.677+0000 7f7dc27fc700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 0x7f7da0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.677+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7db406de60 con 0x7f7dc40600f0 2020-11-16T22:35:28.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.677+0000 7f7dc27fc700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 0x7f7da0057520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f7dac00b150 tx=0x7f7dac015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.681+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7db4044060 con 0x7f7dc40600f0 2020-11-16T22:35:28.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7dc405c4a0 con 0x7f7dc40600f0 2020-11-16T22:35:28.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dbaffd700 1 -- 172.21.15.181:0/3552463581 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7db4047680 con 0x7f7dc40600f0 2020-11-16T22:35:28.825 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2020-11-16T22:35:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 msgr2=0x7f7da0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 0x7f7da0057520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f7dac00b150 tx=0x7f7dac015000).stop 2020-11-16T22:35:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 msgr2=0x7f7dc413c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.837+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f7db400cb50 tx=0x7f7db4008040).stop 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 shutdown_connections 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7da0055060 0x7f7da0057520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dc4163970 0x7f7dc40d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc40600f0 0x7f7dc413c510 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 --2- 172.21.15.181:0/3552463581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dc40cf390 0x7f7dc413ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 >> 172.21.15.181:0/3552463581 conn(0x7f7dc40cfe90 msgr2=0x7f7dc4060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 shutdown_connections 2020-11-16T22:35:28.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.841+0000 7f7dc3fff700 1 -- 172.21.15.181:0/3552463581 wait complete. 2020-11-16T22:35:28.840 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:35:28.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.973+0000 7fdf0ae17700 1 Processor -- start 2020-11-16T22:35:28.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.973+0000 7fdf0ae17700 1 -- start start 2020-11-16T22:35:28.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.973+0000 7fdf0ae17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 0x7fdf04167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf041672c0 0x7fdf040b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf04168390 con 0x7fdf040b3220 2020-11-16T22:35:28.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf04167770 con 0x7fdf041672c0 2020-11-16T22:35:28.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040b5f00 con 0x7fdf04167ac0 2020-11-16T22:35:28.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf03fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdf040b3220 0x7fdf04166bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51052/0 (socket says 172.21.15.181:51052) 2020-11-16T22:35:28.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf03fff700 1 -- 172.21.15.181:0/778873128 learned_addr learned my addr 172.21.15.181:0/778873128 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:28.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf093b4700 1 --2- 172.21.15.181:0/778873128 >> v2:172.21.15.181:3300/0 conn(0x7fdf041672c0 0x7fdf040b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/778873128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 0x7fdf04167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 -- 172.21.15.181:0/778873128 >> v2:172.21.15.181:3300/0 conn(0x7fdf041672c0 msgr2=0x7fdf040b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf037fe700 1 -- 172.21.15.181:0/778873128 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189135915 0 0) 0x7fdf04168390 con 0x7fdf040b3220 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/778873128 >> v2:172.21.15.181:3300/0 conn(0x7fdf041672c0 0x7fdf040b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 -- 172.21.15.181:0/778873128 >> v1:172.21.15.181:6789/0 conn(0x7fdf040b3220 legacy=0x7fdf04166bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 -- 172.21.15.181:0/778873128 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040b2060 con 0x7fdf04167ac0 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/778873128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 0x7fdf04167ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fdefc00b9a0 tx=0x7fdefc006df0).ready entity=mon.2 client_cookie=f7098d66cc22644f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf037fe700 1 -- 172.21.15.181:0/778873128 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0097b0 con 0x7fdf04167ac0 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf037fe700 1 -- 172.21.15.181:0/778873128 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc009910 con 0x7fdf04167ac0 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf037fe700 1 -- 172.21.15.181:0/778873128 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc009c90 con 0x7fdf04167ac0 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- 172.21.15.181:0/778873128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 msgr2=0x7fdf04167ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/778873128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 0x7fdf04167ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fdefc00b9a0 tx=0x7fdefc006df0).stop 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- 172.21.15.181:0/778873128 shutdown_connections 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/778873128 >> v2:172.21.15.181:3300/0 conn(0x7fdf041672c0 0x7fdf040b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/778873128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf04167ac0 0x7fdf04167ea0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- 172.21.15.181:0/778873128 >> 172.21.15.181:0/778873128 conn(0x7fdf04060bc0 msgr2=0x7fdf040d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- 172.21.15.181:0/778873128 shutdown_connections 2020-11-16T22:35:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.977+0000 7fdf0ae17700 1 -- 172.21.15.181:0/778873128 wait complete. 2020-11-16T22:35:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.981+0000 7fdf0ae17700 1 Processor -- start 2020-11-16T22:35:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.981+0000 7fdf0ae17700 1 -- start start 2020-11-16T22:35:28.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf040b3220 0x7fdf04176330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf041672c0 0x7fdf04174990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04176820 con 0x7fdf040b3220 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04175170 con 0x7fdf04167ac0 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04175410 con 0x7fdf041672c0 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf03fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf041672c0 0x7fdf04174990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36758/0 (socket says 172.21.15.181:36758) 2020-11-16T22:35:28.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 -- 172.21.15.181:0/2166003908 learned_addr learned my addr 172.21.15.181:0/2166003908 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:28.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf040b3220 0x7fdf04176330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 -- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf041672c0 msgr2=0x7fdf04174990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf041672c0 0x7fdf04174990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 -- 172.21.15.181:0/2166003908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf040b3220 msgr2=0x7fdf04176330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf040b3220 0x7fdf04176330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 -- 172.21.15.181:0/2166003908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf041756b0 con 0x7fdf04167ac0 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf093b4700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fdeec00eba0 tx=0x7fdeec006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00c820 con 0x7fdf04167ac0 2020-11-16T22:35:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00c980 con 0x7fdf04167ac0 2020-11-16T22:35:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf04175950 con 0x7fdf04167ac0 2020-11-16T22:35:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00cd40 con 0x7fdf04167ac0 2020-11-16T22:35:28.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf04163950 con 0x7fdf04167ac0 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04175ae0 con 0x7fdf04167ac0 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.985+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdeec012600 con 0x7fdf04167ac0 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.989+0000 7fdf00ff9700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 0x7fded4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.989+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdeec06df10 con 0x7fdf04167ac0 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.989+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 0x7fded4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:28.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.989+0000 7fdf08bb3700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 0x7fded4057550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fdefc006e30 tx=0x7fdefc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:28.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:28.993+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdeec044170 con 0x7fdf04167ac0 2020-11-16T22:35:29.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf04069eb0 con 0x7fdf04167ac0 2020-11-16T22:35:29.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf00ff9700 1 -- 172.21.15.181:0/2166003908 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdeec047790 con 0x7fdf04167ac0 2020-11-16T22:35:29.132 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 msgr2=0x7fded4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 0x7fded4057550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fdefc006e30 tx=0x7fdefc015000).stop 2020-11-16T22:35:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 msgr2=0x7fdf0414e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fdeec00eba0 tx=0x7fdeec006df0).stop 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 shutdown_connections 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fded4055090 0x7fded4057550 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf04167ac0 0x7fdf0414e3a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf040b3220 0x7fdf04176330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 --2- 172.21.15.181:0/2166003908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf041672c0 0x7fdf04174990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.145+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 >> 172.21.15.181:0/2166003908 conn(0x7fdf04060bc0 msgr2=0x7fdf04166ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.149+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 shutdown_connections 2020-11-16T22:35:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.149+0000 7fdf0ae17700 1 -- 172.21.15.181:0/2166003908 wait complete. 2020-11-16T22:35:29.151 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:35:29.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 Processor -- start 2020-11-16T22:35:29.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- start start 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f674015f5b0 0x7f67400d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67400d1830 0x7f67400629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6740160e40 con 0x7f67400615f0 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67401609a0 con 0x7f674015f5b0 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6740160ba0 con 0x7f67400d1830 2020-11-16T22:35:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67400d1830 0x7f67400629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67400d1830 0x7f67400629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36762/0 (socket says 172.21.15.181:36762) 2020-11-16T22:35:29.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 -- 172.21.15.181:0/2782249375 learned_addr learned my addr 172.21.15.181:0/2782249375 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673f7fe700 1 --2- 172.21.15.181:0/2782249375 >> v2:172.21.15.175:3300/0 conn(0x7f674015f5b0 0x7f67400d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 -- 172.21.15.181:0/2782249375 >> v2:172.21.15.175:3300/0 conn(0x7f674015f5b0 msgr2=0x7f67400d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 --2- 172.21.15.181:0/2782249375 >> v2:172.21.15.175:3300/0 conn(0x7f674015f5b0 0x7f67400d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 -- 172.21.15.181:0/2782249375 >> v1:172.21.15.175:6789/0 conn(0x7f67400615f0 legacy=0x7f67400622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 -- 172.21.15.181:0/2782249375 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740060970 con 0x7f67400d1830 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673ffff700 1 --2- 172.21.15.181:0/2782249375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67400629b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f67400d2ea0 tx=0x7f6738009290).ready entity=mon.1 client_cookie=9358b034f9d07101 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673effd700 1 -- 172.21.15.181:0/2782249375 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800bbc0 con 0x7f67400d1830 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673effd700 1 -- 172.21.15.181:0/2782249375 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800bd20 con 0x7f67400d1830 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f673effd700 1 -- 172.21.15.181:0/2782249375 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738011610 con 0x7f67400d1830 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- 172.21.15.181:0/2782249375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 msgr2=0x7f67400629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 --2- 172.21.15.181:0/2782249375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67400629b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f67400d2ea0 tx=0x7f6738009290).stop 2020-11-16T22:35:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- 172.21.15.181:0/2782249375 shutdown_connections 2020-11-16T22:35:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 --2- 172.21.15.181:0/2782249375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67400629b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 --2- 172.21.15.181:0/2782249375 >> v2:172.21.15.175:3300/0 conn(0x7f674015f5b0 0x7f67400d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- 172.21.15.181:0/2782249375 >> 172.21.15.181:0/2782249375 conn(0x7f67400d69a0 msgr2=0x7f67400d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- 172.21.15.181:0/2782249375 shutdown_connections 2020-11-16T22:35:29.278 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.285+0000 7f6745d1f700 1 -- 172.21.15.181:0/2782249375 wait complete. 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 Processor -- start 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 -- start start 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674015f5b0 0x7f67401706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67401627f0 con 0x7f674015f5b0 2020-11-16T22:35:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740162a90 con 0x7f67400d1830 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6740162d30 con 0x7f67400615f0 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f673f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36768/0 (socket says 172.21.15.181:36768) 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 -- 172.21.15.181:0/337131007 learned_addr learned my addr 172.21.15.181:0/337131007 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f673f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44608/0 (socket says 172.21.15.181:44608) 2020-11-16T22:35:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f673ffff700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674015f5b0 0x7f67401706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 -- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 msgr2=0x7f67400628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 -- 172.21.15.181:0/337131007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674015f5b0 msgr2=0x7f67401706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674015f5b0 0x7f67401706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 -- 172.21.15.181:0/337131007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740162fd0 con 0x7f67400d1830 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f673f7fe700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6737fff700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f672800c710 tx=0x7f6728007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728007020 con 0x7f67400d1830 2020-11-16T22:35:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728007180 con 0x7f67400d1830 2020-11-16T22:35:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6740163270 con 0x7f67400d1830 2020-11-16T22:35:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.289+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280064f0 con 0x7f67400d1830 2020-11-16T22:35:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6740162440 con 0x7f67400d1830 2020-11-16T22:35:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6740069eb0 con 0x7f67400d1830 2020-11-16T22:35:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6728010070 con 0x7f67400d1830 2020-11-16T22:35:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f67377fe700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 0x7f671c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f673f7fe700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 0x7f671c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f672806d0f0 con 0x7f67400d1830 2020-11-16T22:35:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.293+0000 7f673f7fe700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 0x7f671c057550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f6730006fd0 tx=0x7f6730008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.305+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f672806aef0 con 0x7f67400d1830 2020-11-16T22:35:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67400600f0 con 0x7f67400d1830 2020-11-16T22:35:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f67377fe700 1 -- 172.21.15.181:0/337131007 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6728043240 con 0x7f67400d1830 2020-11-16T22:35:29.444 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 msgr2=0x7f671c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 0x7f671c057550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f6730006fd0 tx=0x7f6730008040).stop 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 msgr2=0x7f67401639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f672800c710 tx=0x7f6728007c20).stop 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 shutdown_connections 2020-11-16T22:35:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f671c055090 0x7f671c057550 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f674015f5b0 0x7f67401706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400615f0 0x7f67400628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 --2- 172.21.15.181:0/337131007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67400d1830 0x7f67401639b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 >> 172.21.15.181:0/337131007 conn(0x7f67400d69a0 msgr2=0x7f6740061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 shutdown_connections 2020-11-16T22:35:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.457+0000 7f6745d1f700 1 -- 172.21.15.181:0/337131007 wait complete. 2020-11-16T22:35:29.465 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:35:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.601+0000 7fab62cf3700 1 Processor -- start 2020-11-16T22:35:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.601+0000 7fab62cf3700 1 -- start start 2020-11-16T22:35:29.591 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fab5c161480 0x7fab5c164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fab5c164e10 0x7fab5c062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c0c8db0 con 0x7fab5c164e10 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c0c8330 con 0x7fab5c161480 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0c85d0 con 0x7fab5c161860 2020-11-16T22:35:29.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44620/0 (socket says 172.21.15.181:44620) 2020-11-16T22:35:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 -- 172.21.15.181:0/1833598598 learned_addr learned my addr 172.21.15.181:0/1833598598 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab61cf1700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.181:3300/0 conn(0x7fab5c161480 0x7fab5c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab614f0700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.175:3300/0 conn(0x7fab5c164e10 0x7fab5c062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 -- 172.21.15.181:0/1833598598 >> v2:172.21.15.181:3300/0 conn(0x7fab5c161480 msgr2=0x7fab5c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:29.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.181:3300/0 conn(0x7fab5c161480 0x7fab5c164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 -- 172.21.15.181:0/1833598598 >> v2:172.21.15.175:3300/0 conn(0x7fab5c164e10 msgr2=0x7fab5c062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.175:3300/0 conn(0x7fab5c164e10 0x7fab5c062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 -- 172.21.15.181:0/1833598598 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0c8870 con 0x7fab5c161860 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab624f2700 1 --2- 172.21.15.181:0/1833598598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fab5c066500 tx=0x7fab4c006eb0).ready entity=mon.2 client_cookie=71e697a56c43436b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab60cef700 1 -- 172.21.15.181:0/1833598598 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c009720 con 0x7fab5c161860 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab60cef700 1 -- 172.21.15.181:0/1833598598 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c009880 con 0x7fab5c161860 2020-11-16T22:35:29.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab60cef700 1 -- 172.21.15.181:0/1833598598 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c009c00 con 0x7fab5c161860 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- 172.21.15.181:0/1833598598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 msgr2=0x7fab5c062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- 172.21.15.181:0/1833598598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fab5c066500 tx=0x7fab4c006eb0).stop 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- 172.21.15.181:0/1833598598 shutdown_connections 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- 172.21.15.181:0/1833598598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c161860 0x7fab5c062740 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.181:3300/0 conn(0x7fab5c161480 0x7fab5c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 --2- 172.21.15.181:0/1833598598 >> v2:172.21.15.175:3300/0 conn(0x7fab5c164e10 0x7fab5c062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- 172.21.15.181:0/1833598598 >> 172.21.15.181:0/1833598598 conn(0x7fab5c0d2dc0 msgr2=0x7fab5c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- 172.21.15.181:0/1833598598 shutdown_connections 2020-11-16T22:35:29.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.605+0000 7fab62cf3700 1 -- 172.21.15.181:0/1833598598 wait complete. 2020-11-16T22:35:29.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 Processor -- start 2020-11-16T22:35:29.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 -- start start 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 0x7fab5c1748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 0x7fab5c173440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 0x7fab5c146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c173930 con 0x7fab5c161860 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c173b40 con 0x7fab5c161480 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab61cf1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 0x7fab5c1748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab62cf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c173de0 con 0x7fab5c0cc000 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 0x7fab5c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab61cf1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 0x7fab5c1748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36780/0 (socket says 172.21.15.181:36780) 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab61cf1700 1 -- 172.21.15.181:0/276840615 learned_addr learned my addr 172.21.15.181:0/276840615 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab614f0700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 0x7fab5c173440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 -- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 msgr2=0x7fab5c1748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 0x7fab5c1748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 -- 172.21.15.181:0/276840615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 msgr2=0x7fab5c173440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 0x7fab5c173440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 -- 172.21.15.181:0/276840615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c174080 con 0x7fab5c0cc000 2020-11-16T22:35:29.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab614f0700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 0x7fab5c173440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:29.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab624f2700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 0x7fab5c146530 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fab4c012000 tx=0x7fab4c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c006b20 con 0x7fab5c0cc000 2020-11-16T22:35:29.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.609+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c01d5d0 con 0x7fab5c0cc000 2020-11-16T22:35:29.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c174320 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c01d970 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c146090 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c044f60 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fab4c025030 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab51ffb700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 0x7fab34057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab4c06f600 con 0x7fab5c0cc000 2020-11-16T22:35:29.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab61cf1700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 0x7fab34057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.613+0000 7fab61cf1700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 0x7fab34057550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fab5400c7a0 tx=0x7fab54006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.621+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fab4c045840 con 0x7fab5c0cc000 2020-11-16T22:35:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.773+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab5c0671e0 con 0x7fab5c0cc000 2020-11-16T22:35:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.773+0000 7fab51ffb700 1 -- 172.21.15.181:0/276840615 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7fab4c023020 con 0x7fab5c0cc000 2020-11-16T22:35:29.762 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 8 2020-11-16T22:35:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 msgr2=0x7fab34057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 0x7fab34057550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fab5400c7a0 tx=0x7fab54006df0).stop 2020-11-16T22:35:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 msgr2=0x7fab5c146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 0x7fab5c146530 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fab4c012000 tx=0x7fab4c015000).stop 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 shutdown_connections 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fab34055090 0x7fab34057550 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab5c0cc000 0x7fab5c146530 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab5c161480 0x7fab5c1748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 --2- 172.21.15.181:0/276840615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab5c161860 0x7fab5c173440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 >> 172.21.15.181:0/276840615 conn(0x7fab5c0d2dc0 msgr2=0x7fab5c063100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 shutdown_connections 2020-11-16T22:35:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.777+0000 7fab62cf3700 1 -- 172.21.15.181:0/276840615 wait complete. 2020-11-16T22:35:29.777 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:35:29.778 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:29.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.913+0000 7f71835a0700 1 Processor -- start 2020-11-16T22:35:29.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.913+0000 7f71835a0700 1 -- start start 2020-11-16T22:35:29.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 0x7f717c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f717c0c9640 0x7f717c064970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c164580 con 0x7f717c0612a0 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c164360 con 0x7f717c0c9640 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c164820 con 0x7f717c061680 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f717c0c9640 0x7f717c064970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f717c0c9640 0x7f717c064970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36792/0 (socket says 172.21.15.181:36792) 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 -- 172.21.15.181:0/268830222 learned_addr learned my addr 172.21.15.181:0/268830222 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7180b3b700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 0x7f717c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 -- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 msgr2=0x7f717c0c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 0x7f717c0c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 -- 172.21.15.181:0/268830222 >> v1:172.21.15.175:6789/0 conn(0x7f717c0612a0 legacy=0x7f717c062ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 -- 172.21.15.181:0/268830222 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c164ac0 con 0x7f717c0c9640 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f716bfff700 1 -- 172.21.15.181:0/268830222 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111702875 0 0) 0x7f717c164580 con 0x7f717c0612a0 2020-11-16T22:35:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7180b3b700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 0x7f717c0c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f7181b3d700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c064970 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f7174006fd0 tx=0x7f7174006eb0).ready entity=mon.1 client_cookie=75d622ab23a5478b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f716bfff700 1 -- 172.21.15.181:0/268830222 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174009810 con 0x7f717c0c9640 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f716bfff700 1 -- 172.21.15.181:0/268830222 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7174009970 con 0x7f717c0c9640 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f716bfff700 1 -- 172.21.15.181:0/268830222 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174009cf0 con 0x7f717c0c9640 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 msgr2=0x7f717c064970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c064970 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f7174006fd0 tx=0x7f7174006eb0).stop 2020-11-16T22:35:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- 172.21.15.181:0/268830222 shutdown_connections 2020-11-16T22:35:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c064970 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 --2- 172.21.15.181:0/268830222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c061680 0x7f717c0c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- 172.21.15.181:0/268830222 >> 172.21.15.181:0/268830222 conn(0x7f717c0cf890 msgr2=0x7f717c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- 172.21.15.181:0/268830222 shutdown_connections 2020-11-16T22:35:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.917+0000 7f71835a0700 1 -- 172.21.15.181:0/268830222 wait complete. 2020-11-16T22:35:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.921+0000 7f71835a0700 1 Processor -- start 2020-11-16T22:35:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.921+0000 7f71835a0700 1 -- start start 2020-11-16T22:35:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 0x7f717c063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 0x7f717c1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c167300 con 0x7f717c061680 2020-11-16T22:35:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c1675a0 con 0x7f717c0c9640 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c167840 con 0x7f717c0612a0 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f718133c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 0x7f717c063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f718133c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 0x7f717c063cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44636/0 (socket says 172.21.15.181:44636) 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f718133c700 1 -- 172.21.15.181:0/2048740393 learned_addr learned my addr 172.21.15.181:0/2048740393 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7180b3b700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 0x7f717c1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 -- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 msgr2=0x7f717c063cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 0x7f717c063cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 -- 172.21.15.181:0/2048740393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 msgr2=0x7f717c1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 0x7f717c1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 -- 172.21.15.181:0/2048740393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c167ae0 con 0x7f717c0c9640 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7180b3b700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 0x7f717c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:35:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f7181b3d700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c160ec0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f7174006fd0 tx=0x7f7174015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174006b60 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717401d5d0 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c167d80 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717401d950 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.925+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0641e0 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c1749a0 con 0x7f717c0c9640 2020-11-16T22:35:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f717401a040 con 0x7f717c0c9640 2020-11-16T22:35:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f71697fa700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 0x7f7154057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f717406ef80 con 0x7f717c0c9640 2020-11-16T22:35:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f718133c700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 0x7f7154057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.929+0000 7f718133c700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 0x7f7154057520 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f717c064c80 tx=0x7f7178009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:29.937+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7174045310 con 0x7f717c0c9640 2020-11-16T22:35:30.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.137+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f717c164ca0 con 0x7f717c0c9640 2020-11-16T22:35:30.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.137+0000 7f71697fa700 1 -- 172.21.15.181:0/2048740393 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4636 (secure 0 0 0) 0x7f717402d230 con 0x7f717c0c9640 2020-11-16T22:35:30.126 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:30.126 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":114,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8040,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8043,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:24.582056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 msgr2=0x7f7154057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 0x7f7154057520 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f717c064c80 tx=0x7f7178009040).stop 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 msgr2=0x7f717c160ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c160ec0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f7174006fd0 tx=0x7f7174015000).stop 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 shutdown_connections 2020-11-16T22:35:30.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7154055060 0x7f7154057520 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f717c0c9640 0x7f717c160ec0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f717c0612a0 0x7f717c063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 --2- 172.21.15.181:0/2048740393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f717c061680 0x7f717c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 >> 172.21.15.181:0/2048740393 conn(0x7f717c0cf890 msgr2=0x7f717c0ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 shutdown_connections 2020-11-16T22:35:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:30.141+0000 7f71835a0700 1 -- 172.21.15.181:0/2048740393 wait complete. 2020-11-16T22:35:30.131 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 114 2020-11-16T22:35:31.050 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:35:22.567943+0000', 'modified': '2020-11-16T22:35:24.582056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8101': {'gid': 8101, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2750102703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2750102703}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2750102703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:35:31.051 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:35:31.051 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:35:31.192 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":8101,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.6231318300000002,"mdsmap_epoch":114,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":14.089285084} 2020-11-16T22:35:31.193 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 8101, "mdsmap_epoch": 114, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 8.62313183, "state": "up:active", "uptime": 14.089285084, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:35:31.193 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:35:31.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:31.194 INFO:teuthology.orchestra.run.smithi106:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:35:31.213 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:35:31.213 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:35:31.282 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:35:31.283 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:35:31.305 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:35:31.305 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:35:31.312 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:35:31.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:35:31.312 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:35:31.339 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:35:31.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 Processor -- start 2020-11-16T22:35:31.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 -- start start 2020-11-16T22:35:31.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 0x7f5d00159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d00161350 con 0x7f5d000b2900 2020-11-16T22:35:31.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d001615f0 con 0x7f5d00159b40 2020-11-16T22:35:31.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00161890 con 0x7f5d00062210 2020-11-16T22:35:31.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5d00159b40 0x7f5d000b21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51106/0 (socket says 172.21.15.181:51106) 2020-11-16T22:35:31.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfeffd700 1 -- 172.21.15.181:0/9127270 learned_addr learned my addr 172.21.15.181:0/9127270 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:31.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cff7fe700 1 --2- 172.21.15.181:0/9127270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 0x7f5d00159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496278985 0 0) 0x7f5d001615f0 con 0x7f5d00159b40 2020-11-16T22:35:31.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cec003240 con 0x7f5d00159b40 2020-11-16T22:35:31.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cff7fe700 1 -- 172.21.15.181:0/9127270 >> v1:172.21.15.181:6789/0 conn(0x7f5d00159b40 legacy=0x7f5d000b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405072339 0 0) 0x7f5d00161350 con 0x7f5d000b2900 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cff7fe700 1 -- 172.21.15.181:0/9127270 >> v1:172.21.15.175:6789/0 conn(0x7f5d000b2900 legacy=0x7f5d00162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cff7fe700 1 -- 172.21.15.181:0/9127270 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00161b30 con 0x7f5d00062210 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cff7fe700 1 --2- 172.21.15.181:0/9127270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 0x7f5d00159f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f5cf8009cc0 tx=0x7f5cf80091d0).ready entity=mon.2 client_cookie=20b0dc0f73bbee2a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.493+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800bad0 con 0x7f5d00062210 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf800bc30 con 0x7f5d00062210 2020-11-16T22:35:31.485 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfdffb700 1 -- 172.21.15.181:0/9127270 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011400 con 0x7f5d00062210 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- 172.21.15.181:0/9127270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 msgr2=0x7f5d00159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 --2- 172.21.15.181:0/9127270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 0x7f5d00159f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f5cf8009cc0 tx=0x7f5cf80091d0).stop 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- 172.21.15.181:0/9127270 shutdown_connections 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 --2- 172.21.15.181:0/9127270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00062210 0x7f5d00159f20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- 172.21.15.181:0/9127270 >> 172.21.15.181:0/9127270 conn(0x7f5d000d4570 msgr2=0x7f5d000d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- 172.21.15.181:0/9127270 shutdown_connections 2020-11-16T22:35:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- 172.21.15.181:0/9127270 wait complete. 2020-11-16T22:35:31.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 Processor -- start 2020-11-16T22:35:31.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.497+0000 7f5cfffff700 1 -- start start 2020-11-16T22:35:31.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d00062210 0x7f5d00162550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d000b2900 0x7f5d001634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cf180 con 0x7f5d000b2900 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cf420 con 0x7f5d00062210 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cf6c0 con 0x7f5d00159b40 2020-11-16T22:35:31.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d00062210 0x7f5d00162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44654/0 (socket says 172.21.15.181:44654) 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 -- 172.21.15.181:0/2744270258 learned_addr learned my addr 172.21.15.181:0/2744270258 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfe7fc700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d000b2900 0x7f5d001634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 -- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d00062210 msgr2=0x7f5d00162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d00062210 0x7f5d00162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 -- 172.21.15.181:0/2744270258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d000b2900 msgr2=0x7f5d001634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d000b2900 0x7f5d001634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 -- 172.21.15.181:0/2744270258 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000cf960 con 0x7f5d00159b40 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cff7fe700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5d000b2f30 tx=0x7f5cf8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8010700 con 0x7f5d00159b40 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8010860 con 0x7f5d00159b40 2020-11-16T22:35:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d000cfc00 con 0x7f5d00159b40 2020-11-16T22:35:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8010be0 con 0x7f5d00159b40 2020-11-16T22:35:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d000cfea0 con 0x7f5d00159b40 2020-11-16T22:35:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d000cefe0 con 0x7f5d00159b40 2020-11-16T22:35:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.501+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5cf8024030 con 0x7f5d00159b40 2020-11-16T22:35:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.505+0000 7f5ce6ffd700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 0x7f5cdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.505+0000 7f5cfeffd700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 0x7f5cdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.505+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5cf806fa20 con 0x7f5d00159b40 2020-11-16T22:35:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.505+0000 7f5cfeffd700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 0x7f5cdc057520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f5cf00055a0 tx=0x7f5cf0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:31.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.509+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5d000cefe0 con 0x7f5d00159b40 2020-11-16T22:35:31.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.709+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f5d000cefe0 con 0x7f5d00159b40 2020-11-16T22:35:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.849+0000 7f5ce6ffd700 1 -- 172.21.15.181:0/2744270258 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v115) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5d000cefe0 con 0x7f5d00159b40 2020-11-16T22:35:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 msgr2=0x7f5cdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 0x7f5cdc057520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f5cf00055a0 tx=0x7f5cf0009040).stop 2020-11-16T22:35:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 msgr2=0x7f5d001639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5d000b2f30 tx=0x7f5cf8016000).stop 2020-11-16T22:35:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 shutdown_connections 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5cdc055060 0x7f5cdc057520 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d00159b40 0x7f5d001639d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d00062210 0x7f5d00162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 --2- 172.21.15.181:0/2744270258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d000b2900 0x7f5d001634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 >> 172.21.15.181:0/2744270258 conn(0x7f5d000d4570 msgr2=0x7f5d000d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 shutdown_connections 2020-11-16T22:35:31.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.853+0000 7f5cfffff700 1 -- 172.21.15.181:0/2744270258 wait complete. 2020-11-16T22:35:31.861 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.985+0000 7f5d0ca4f700 1 Processor -- start 2020-11-16T22:35:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.985+0000 7f5d0ca4f700 1 -- start start 2020-11-16T22:35:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d0813e870 0x7f5d0813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08061810 0x7f5d08160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0815d180 con 0x7f5d0815fdc0 2020-11-16T22:35:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d08160730 con 0x7f5d0813e870 2020-11-16T22:35:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0813f140 con 0x7f5d08061810 2020-11-16T22:35:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08061810 0x7f5d08160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08061810 0x7f5d08160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36820/0 (socket says 172.21.15.181:36820) 2020-11-16T22:35:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 -- 172.21.15.181:0/2270970672 learned_addr learned my addr 172.21.15.181:0/2270970672 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d077fe700 1 --2- 172.21.15.181:0/2270970672 >> v2:172.21.15.175:3300/0 conn(0x7f5d0813e870 0x7f5d0813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 -- 172.21.15.181:0/2270970672 >> v2:172.21.15.175:3300/0 conn(0x7f5d0813e870 msgr2=0x7f5d0813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 --2- 172.21.15.181:0/2270970672 >> v2:172.21.15.175:3300/0 conn(0x7f5d0813e870 0x7f5d0813ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 -- 172.21.15.181:0/2270970672 >> v1:172.21.15.175:6789/0 conn(0x7f5d0815fdc0 legacy=0x7f5d08061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 -- 172.21.15.181:0/2270970672 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0815d810 con 0x7f5d08061810 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d07fff700 1 --2- 172.21.15.181:0/2270970672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08061810 0x7f5d08160240 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f5d08160870 tx=0x7f5d0000a430).ready entity=mon.1 client_cookie=e27d5bf49e4e6ca2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d067fc700 1 -- 172.21.15.181:0/2270970672 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ce60 con 0x7f5d08061810 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d067fc700 1 -- 172.21.15.181:0/2270970672 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00010410 con 0x7f5d08061810 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d067fc700 1 -- 172.21.15.181:0/2270970672 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00010790 con 0x7f5d08061810 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/2270970672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08061810 msgr2=0x7f5d08160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/2270970672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08061810 0x7f5d08160240 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f5d08160870 tx=0x7f5d0000a430).stop 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/2270970672 shutdown_connections 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/2270970672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08061810 0x7f5d08160240 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/2270970672 >> v2:172.21.15.175:3300/0 conn(0x7f5d0813e870 0x7f5d0813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/2270970672 >> 172.21.15.181:0/2270970672 conn(0x7f5d080cf630 msgr2=0x7f5d080cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/2270970672 shutdown_connections 2020-11-16T22:35:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.989+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/2270970672 wait complete. 2020-11-16T22:35:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.993+0000 7f5d0ca4f700 1 Processor -- start 2020-11-16T22:35:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.993+0000 7f5d0ca4f700 1 -- start start 2020-11-16T22:35:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d08061810 0x7f5d0815f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0813e870 0x7f5d0813d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0813d8e0 con 0x7f5d08061810 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0816e320 con 0x7f5d0813e870 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0816dde0 con 0x7f5d0815fdc0 2020-11-16T22:35:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d06ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0813e870 0x7f5d0813d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44668/0 (socket says 172.21.15.181:44668) 2020-11-16T22:35:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 -- 172.21.15.181:0/1719748893 learned_addr learned my addr 172.21.15.181:0/1719748893 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d077fe700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d08061810 0x7f5d0815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 -- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0813e870 msgr2=0x7f5d0813d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0813e870 0x7f5d0813d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 -- 172.21.15.181:0/1719748893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d08061810 msgr2=0x7f5d0815f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d08061810 0x7f5d0815f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 -- 172.21.15.181:0/1719748893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0816e080 con 0x7f5d0815fdc0 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d07fff700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f5d0001a720 tx=0x7f5d080ca230).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ce60 con 0x7f5d0815fdc0 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0001aae0 con 0x7f5d0815fdc0 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0001ae80 con 0x7f5d0815fdc0 2020-11-16T22:35:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0813a5b0 con 0x7f5d0815fdc0 2020-11-16T22:35:32.190 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d08061e60 con 0x7f5d0815fdc0 2020-11-16T22:35:32.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:31.997+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d08062080 con 0x7f5d0815fdc0 2020-11-16T22:35:32.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.001+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5d0002a070 con 0x7f5d0815fdc0 2020-11-16T22:35:32.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.001+0000 7f5ce77fe700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 0x7f5ce0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:32.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.001+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5d0006f610 con 0x7f5d0815fdc0 2020-11-16T22:35:32.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.001+0000 7f5d077fe700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 0x7f5ce0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:32.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.001+0000 7f5d077fe700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 0x7f5ce0057520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f5cf8006fd0 tx=0x7f5cf8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:32.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.009+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5d00045870 con 0x7f5d0815fdc0 2020-11-16T22:35:32.198 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.209+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d08044f60 con 0x7f5d0815fdc0 2020-11-16T22:35:32.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5ce77fe700 1 -- 172.21.15.181:0/1719748893 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4652 (secure 0 0 0) 0x7f5d00048e90 con 0x7f5d0815fdc0 2020-11-16T22:35:32.200 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:32.200 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":116,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8040,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:31.848989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8101,"mds_1":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:creating","state_seq":1,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 msgr2=0x7f5ce0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 0x7f5ce0057520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f5cf8006fd0 tx=0x7f5cf8006df0).stop 2020-11-16T22:35:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 msgr2=0x7f5d0816d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f5d0001a720 tx=0x7f5d080ca230).stop 2020-11-16T22:35:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 shutdown_connections 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f5ce0055060 0x7f5ce0057520 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0815fdc0 0x7f5d0816d3c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d08061810 0x7f5d0815f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 --2- 172.21.15.181:0/1719748893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0813e870 0x7f5d0813d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.213+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 >> 172.21.15.181:0/1719748893 conn(0x7f5d080cf630 msgr2=0x7f5d080cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.217+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 shutdown_connections 2020-11-16T22:35:32.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:32.217+0000 7f5d0ca4f700 1 -- 172.21.15.181:0/1719748893 wait complete. 2020-11-16T22:35:32.608 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 116 2020-11-16T22:35:32.973 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:35:32.985+0000 7ff288533700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:35:33.515 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:35:22.567943+0000', 'modified': '2020-11-16T22:35:31.848989+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8101, 'mds_1': 8043}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8043': {'gid': 8043, 'name': 'i', 'rank': 1, 'incarnation': 116, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.175:6840/3163754567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3163754567}, {'type': 'v1', 'addr': '172.21.15.175:6840', 'nonce': 3163754567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_8101': {'gid': 8101, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2750102703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2750102703}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2750102703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:35:33.515 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8043:up:creating 2020-11-16T22:35:33.997 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:34.004 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:34.006 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:35:34.517 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:34.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 Processor -- start 2020-11-16T22:35:34.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 -- start start 2020-11-16T22:35:34.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec161480 0x7f79ec164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:34.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec0c8db0 con 0x7f79ec164e10 2020-11-16T22:35:34.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec0c8330 con 0x7f79ec161860 2020-11-16T22:35:34.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79ebfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0c85d0 con 0x7f79ec161480 2020-11-16T22:35:34.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79eb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f79ec161860 0x7f79ec0644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51136/0 (socket says 172.21.15.181:51136) 2020-11-16T22:35:34.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79eb7fe700 1 -- 172.21.15.181:0/3612235804 learned_addr learned my addr 172.21.15.181:0/3612235804 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:34.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79eaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec161480 0x7f79ec164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:34.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760169380 0 0) 0x7f79ec0c8330 con 0x7f79ec161860 2020-11-16T22:35:34.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.661+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d0003240 con 0x7f79ec161860 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350915861 0 0) 0x7f79ec0c8db0 con 0x7f79ec164e10 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eaffd700 1 -- 172.21.15.181:0/3612235804 >> v1:172.21.15.181:6789/0 conn(0x7f79ec161860 legacy=0x7f79ec0644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eaffd700 1 -- 172.21.15.181:0/3612235804 >> v1:172.21.15.175:6789/0 conn(0x7f79ec164e10 legacy=0x7f79ec062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eaffd700 1 -- 172.21.15.181:0/3612235804 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0c8870 con 0x7f79ec161480 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eaffd700 1 --2- 172.21.15.181:0/3612235804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161480 0x7f79ec164920 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f79d400b9a0 tx=0x7f79d4006df0).ready entity=mon.1 client_cookie=4321ae90a411b2ba server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:34.653 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40097b0 con 0x7f79ec161480 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4009910 con 0x7f79ec161480 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79e9ffb700 1 -- 172.21.15.181:0/3612235804 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4009c90 con 0x7f79ec161480 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- 172.21.15.181:0/3612235804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161480 msgr2=0x7f79ec164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 --2- 172.21.15.181:0/3612235804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161480 0x7f79ec164920 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f79d400b9a0 tx=0x7f79d4006df0).stop 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- 172.21.15.181:0/3612235804 shutdown_connections 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 --2- 172.21.15.181:0/3612235804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161480 0x7f79ec164920 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:34.654 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- 172.21.15.181:0/3612235804 >> 172.21.15.181:0/3612235804 conn(0x7f79ec0d2dc0 msgr2=0x7f79ec0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:34.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- 172.21.15.181:0/3612235804 shutdown_connections 2020-11-16T22:35:34.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- 172.21.15.181:0/3612235804 wait complete. 2020-11-16T22:35:34.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 Processor -- start 2020-11-16T22:35:34.656 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- start start 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 0x7f79ec161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec1621b0 0x7f79ec065160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec145de0 con 0x7f79ec161480 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec146080 con 0x7f79ec161860 2020-11-16T22:35:34.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ebfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec146320 con 0x7f79ec1621b0 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36836/0 (socket says 172.21.15.181:36836) 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79ea7fc700 1 -- 172.21.15.181:0/556281866 learned_addr learned my addr 172.21.15.181:0/556281866 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eb7fe700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec1621b0 0x7f79ec065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.665+0000 7f79eaffd700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 0x7f79ec161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:34.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 -- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec1621b0 msgr2=0x7f79ec065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec1621b0 0x7f79ec065160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 -- 172.21.15.181:0/556281866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 msgr2=0x7f79ec161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 0x7f79ec161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 -- 172.21.15.181:0/556281866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec1465c0 con 0x7f79ec161860 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79eaffd700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 0x7f79ec161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ea7fc700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f79e400cae0 tx=0x7f79e4007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:34.659 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400a730 con 0x7f79ec161860 2020-11-16T22:35:34.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400a890 con 0x7f79ec161860 2020-11-16T22:35:34.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec146860 con 0x7f79ec161860 2020-11-16T22:35:34.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400ac50 con 0x7f79ec161860 2020-11-16T22:35:34.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec144cf0 con 0x7f79ec161860 2020-11-16T22:35:34.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f79e4019030 con 0x7f79ec161860 2020-11-16T22:35:34.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec044f60 con 0x7f79ec161860 2020-11-16T22:35:34.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 0x7f79bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:34.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79e4003030 con 0x7f79ec161860 2020-11-16T22:35:34.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79eaffd700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 0x7f79bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:34.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.669+0000 7f79eaffd700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 0x7f79bc057520 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f79d400be60 tx=0x7f79d4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:34.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.677+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f79e40438f0 con 0x7f79ec161860 2020-11-16T22:35:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.873+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79ec0671e0 con 0x7f79ec161860 2020-11-16T22:35:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.873+0000 7f79e2ffd700 1 -- 172.21.15.181:0/556281866 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f79e4046f10 con 0x7f79ec161860 2020-11-16T22:35:34.863 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:34.863 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:33.872772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 msgr2=0x7f79bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 0x7f79bc057520 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f79d400be60 tx=0x7f79d4015000).stop 2020-11-16T22:35:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 msgr2=0x7f79ec064c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f79e400cae0 tx=0x7f79e4007d40).stop 2020-11-16T22:35:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 shutdown_connections 2020-11-16T22:35:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f79bc055060 0x7f79bc057520 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec1621b0 0x7f79ec065160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:35.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79ec161480 0x7f79ec161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:35.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 --2- 172.21.15.181:0/556281866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec161860 0x7f79ec064c70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:35.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 >> 172.21.15.181:0/556281866 conn(0x7f79ec0d2dc0 msgr2=0x7f79ec0cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:35.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 shutdown_connections 2020-11-16T22:35:35.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:34.877+0000 7f79ebfff700 1 -- 172.21.15.181:0/556281866 wait complete. 2020-11-16T22:35:35.068 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 119 2020-11-16T22:35:35.973 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:35:22.567943+0000', 'modified': '2020-11-16T22:35:33.872772+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8101, 'mds_1': 8043, 'mds_2': 8040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8040': {'gid': 8040, 'name': 'e', 'rank': 2, 'incarnation': 118, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6837/722500686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 722500686}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 722500686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_8043': {'gid': 8043, 'name': 'i', 'rank': 1, 'incarnation': 116, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.175:6840/3163754567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3163754567}, {'type': 'v1', 'addr': '172.21.15.175:6840', 'nonce': 3163754567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_8101': {'gid': 8101, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2750102703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2750102703}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2750102703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:35:35.974 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:35:35.975 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json status 2020-11-16T22:35:36.113 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":2,"id":8040,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.2441990829999998,"mdsmap_epoch":119,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":18.989907785} 2020-11-16T22:35:36.113 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 8040, "mdsmap_epoch": 119, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 3.244199083, "state": "up:active", "uptime": 18.989907785, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:35:36.114 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json status 2020-11-16T22:35:36.241 INFO:teuthology.orchestra.run.smithi175.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":1,"id":8043,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3913696959999999,"mdsmap_epoch":119,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":19.115728652000001} 2020-11-16T22:35:36.242 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 8043, "mdsmap_epoch": 119, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 4.391369696, "state": "up:active", "uptime": 19.115728652, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:35:36.242 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2020-11-16T22:35:36.380 INFO:teuthology.orchestra.run.smithi181.stdout:{"cluster_fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","whoami":0,"id":8101,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.810769201999999,"mdsmap_epoch":119,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":19.276924232999999} 2020-11-16T22:35:36.380 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0da1f45c-24e6-440b-b459-36ddfb20a11a", "fs_name": "cephfs", "id": 8101, "mdsmap_epoch": 119, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 13.810769202, "state": "up:active", "uptime": 19.276924233, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:35:36.381 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:35:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 Processor -- start 2020-11-16T22:35:36.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 -- start start 2020-11-16T22:35:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0194156a90 0x7f0194061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f019415f3c0 0x7f01940cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01940cccd0 con 0x7f01940615f0 2020-11-16T22:35:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940ccf70 con 0x7f0194156a90 2020-11-16T22:35:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019415e030 con 0x7f019415f3c0 2020-11-16T22:35:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0194156a90 0x7f0194061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f0191d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f01940615f0 0x7f019415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51150/0 (socket says 172.21.15.181:51150) 2020-11-16T22:35:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f0191d9b700 1 -- 172.21.15.181:0/2248041986 learned_addr learned my addr 172.21.15.181:0/2248041986 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:35:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f0192d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f019415f3c0 0x7f01940cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019259c700 1 -- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f019415f3c0 msgr2=0x7f01940cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:35:36.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019259c700 1 --2- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f019415f3c0 0x7f01940cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019259c700 1 -- 172.21.15.181:0/2248041986 >> v1:172.21.15.181:6789/0 conn(0x7f01940615f0 legacy=0x7f019415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019159a700 1 -- 172.21.15.181:0/2248041986 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903268959 0 0) 0x7f01940cccd0 con 0x7f01940615f0 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.513+0000 7f019259c700 1 -- 172.21.15.181:0/2248041986 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019415e2d0 con 0x7f0194156a90 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019259c700 1 --2- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0194156a90 0x7f0194061100 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f018400b9a0 tx=0x7f0184006df0).ready entity=mon.1 client_cookie=29aa606dba989630 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019159a700 1 -- 172.21.15.181:0/2248041986 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01840097b0 con 0x7f0194156a90 2020-11-16T22:35:36.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019159a700 1 -- 172.21.15.181:0/2248041986 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184009910 con 0x7f0194156a90 2020-11-16T22:35:36.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019159a700 1 -- 172.21.15.181:0/2248041986 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184009c90 con 0x7f0194156a90 2020-11-16T22:35:36.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0194156a90 msgr2=0x7f0194061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:36.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 --2- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0194156a90 0x7f0194061100 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f018400b9a0 tx=0x7f0184006df0).stop 2020-11-16T22:35:36.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- 172.21.15.181:0/2248041986 shutdown_connections 2020-11-16T22:35:36.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 --2- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0194156a90 0x7f0194061100 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:36.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 --2- 172.21.15.181:0/2248041986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f019415f3c0 0x7f01940cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:36.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- 172.21.15.181:0/2248041986 >> 172.21.15.181:0/2248041986 conn(0x7f01940cf630 msgr2=0x7f01940cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:36.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- 172.21.15.181:0/2248041986 shutdown_connections 2020-11-16T22:35:36.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- 172.21.15.181:0/2248041986 wait complete. 2020-11-16T22:35:36.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 Processor -- start 2020-11-16T22:35:36.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.517+0000 7f019359e700 1 -- start start 2020-11-16T22:35:36.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940615f0 0x7f01940b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:36.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:36.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019415f3c0 0x7f01941707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:36.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940d0f80 con 0x7f019415f3c0 2020-11-16T22:35:36.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019416d210 con 0x7f01940615f0 2020-11-16T22:35:36.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0194170fc0 con 0x7f0194156a90 2020-11-16T22:35:36.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:36.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940615f0 0x7f01940b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:36.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44696/0 (socket says 172.21.15.181:44696) 2020-11-16T22:35:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 -- 172.21.15.181:0/2254443840 learned_addr learned my addr 172.21.15.181:0/2254443840 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:35:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0192d9d700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019415f3c0 0x7f01941707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 -- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940615f0 msgr2=0x7f01940b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940615f0 0x7f01940b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 -- 172.21.15.181:0/2254443840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019415f3c0 msgr2=0x7f01941707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019415f3c0 0x7f01941707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 -- 172.21.15.181:0/2254443840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194171260 con 0x7f0194156a90 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f0191d9b700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f018800c740 tx=0x7f0188007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:37.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880072e0 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188007440 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0194171500 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188005800 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019416d6a0 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.521+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194044f60 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.525+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f018801a060 con 0x7f0194156a90 2020-11-16T22:35:37.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.525+0000 7f01827fc700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 0x7f0168057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.525+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f018806d4c0 con 0x7f0194156a90 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.525+0000 7f019259c700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 0x7f0168057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.525+0000 7f019259c700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 0x7f0168057520 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f0184006e30 tx=0x7f0184015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.529+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f01880437d0 con 0x7f0194156a90 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0194170cd0 con 0x7f0194156a90 2020-11-16T22:35:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f01827fc700 1 -- 172.21.15.181:0/2254443840 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f0188018070 con 0x7f0194156a90 2020-11-16T22:35:37.130 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:35:37.130 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:33.872772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:35:37.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 msgr2=0x7f0168057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 0x7f0168057520 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f0184006e30 tx=0x7f0184015000).stop 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 msgr2=0x7f01940d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f018800c740 tx=0x7f0188007ac0).stop 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 shutdown_connections 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0168055060 0x7f0168057520 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f019415f3c0 0x7f01941707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01940615f0 0x7f01940b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 --2- 172.21.15.181:0/2254443840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0194156a90 0x7f01940d0a90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:35:37.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 >> 172.21.15.181:0/2254443840 conn(0x7f01940cf630 msgr2=0x7f019415ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:35:37.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 shutdown_connections 2020-11-16T22:35:37.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:35:36.729+0000 7f019359e700 1 -- 172.21.15.181:0/2254443840 wait complete. 2020-11-16T22:35:37.337 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 119 2020-11-16T22:35:37.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:37.338 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:35:37.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:37.400 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:35:37.973 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:35:37.985+0000 7ff288533700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:35:42.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:42.114 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:35:42.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:42.153 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:35:42.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:35:42.194 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:35:42.235 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:42.369 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:35:42.370 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:42.370 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:44.371 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:44.512 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:35:44.513 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:44.513 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:46.514 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:46.650 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:35:46.650 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:46.651 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:48.652 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:48.790 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:35:48.791 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:48.792 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:50.793 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:50.929 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:35:50.929 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:50.930 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:52.931 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2020-11-16T22:35:53.065 INFO:teuthology.orchestra.run.smithi181.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:35:53.066 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:35:53.067 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:35:53.067 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:35:53.203 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:35:53.204 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:35:53.204 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:35:55.205 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:35:55.344 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:35:55.344 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:35:55.345 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:35:57.347 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:35:57.485 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:35:57.486 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:35:57.486 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:35:59.487 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok --format=json get subtrees 2020-11-16T22:35:59.617 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:35:59.618 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:35:59.619 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:35:59.619 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok --format=json get subtrees 2020-11-16T22:35:59.742 INFO:teuthology.orchestra.run.smithi175.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"15","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:35:59.947 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:35:59.948 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:35:59.948 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:00.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 Processor -- start 2020-11-16T22:36:00.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 -- start start 2020-11-16T22:36:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff294147270 0x7ff294060800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29415e090 con 0x7ff294146e10 2020-11-16T22:36:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29415e330 con 0x7ff29415d4e0 2020-11-16T22:36:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff29a4ae700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29415edb0 con 0x7ff294147270 2020-11-16T22:36:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff293fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff29415d4e0 0x7ff29415d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51162/0 (socket says 172.21.15.181:51162) 2020-11-16T22:36:00.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff293fff700 1 -- 172.21.15.181:0/1727415966 learned_addr learned my addr 172.21.15.181:0/1727415966 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931743560 0 0) 0x7ff29415e330 con 0x7ff29415d4e0 2020-11-16T22:36:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 --2- 172.21.15.181:0/1727415966 >> v2:172.21.15.175:3300/0 conn(0x7ff294147270 0x7ff294060800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003240 con 0x7ff29415d4e0 2020-11-16T22:36:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1383010291 0 0) 0x7ff280003240 con 0x7ff29415d4e0 2020-11-16T22:36:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 >> v2:172.21.15.175:3300/0 conn(0x7ff294147270 msgr2=0x7ff294060800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 --2- 172.21.15.181:0/1727415966 >> v2:172.21.15.175:3300/0 conn(0x7ff294147270 0x7ff294060800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 >> v1:172.21.15.175:6789/0 conn(0x7ff294146e10 legacy=0x7ff2940600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29415e5d0 con 0x7ff29415d4e0 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7ff28c001520 con 0x7ff29415d4e0 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 >> v1:172.21.15.181:6789/0 conn(0x7ff29415d4e0 legacy=0x7ff29415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280003730 0x7ff280003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800044e0 0x7ff280004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 0x7ff28000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29415edb0 con 0x7ff280003730 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280004330 con 0x7ff2800044e0 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000bc50 con 0x7ff280004e50 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff2937fe700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800044e0 0x7ff280004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 0x7ff28000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff293fff700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280003730 0x7ff280003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 -- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800044e0 msgr2=0x7ff280004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800044e0 0x7ff280004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 -- 172.21.15.181:0/1727415966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280003730 msgr2=0x7ff280003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:00.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280003730 0x7ff280003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 -- 172.21.15.181:0/1727415966 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280003240 con 0x7ff280004e50 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 -- 172.21.15.181:0/1727415966 reap_dead start 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff298a4b700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 0x7ff28000b730 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7ff284009ea0 tx=0x7ff2840091d0).ready entity=mon.2 client_cookie=2ba02edf944a9b27 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400ec40 con 0x7ff280004e50 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28400eda0 con 0x7ff280004e50 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.088+0000 7ff292ffd700 1 -- 172.21.15.181:0/1727415966 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284012640 con 0x7ff280004e50 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 msgr2=0x7ff28000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 0x7ff28000b730 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7ff284009ea0 tx=0x7ff2840091d0).stop 2020-11-16T22:36:00.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- 172.21.15.181:0/1727415966 shutdown_connections 2020-11-16T22:36:00.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/1727415966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff280004e50 0x7ff28000b730 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- 172.21.15.181:0/1727415966 >> 172.21.15.181:0/1727415966 conn(0x7ff2940d3680 msgr2=0x7ff2940d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:00.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- 172.21.15.181:0/1727415966 shutdown_connections 2020-11-16T22:36:00.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- 172.21.15.181:0/1727415966 wait complete. 2020-11-16T22:36:00.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 Processor -- start 2020-11-16T22:36:00.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- start start 2020-11-16T22:36:00.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff29416d280 0x7ff294142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 0x7ff294141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29416d760 con 0x7ff29416d280 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29416da00 con 0x7ff280004e50 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff29a4ae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29416dca0 con 0x7ff294143370 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff293fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 0x7ff294141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff293fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36870/0 (socket says 172.21.15.181:36870) 2020-11-16T22:36:00.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff293fff700 1 -- 172.21.15.181:0/44329711 learned_addr learned my addr 172.21.15.181:0/44329711 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:00.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff2937fe700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff29416d280 0x7ff294142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 -- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 msgr2=0x7ff29416cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 -- 172.21.15.181:0/44329711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff29416d280 msgr2=0x7ff294142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff29416d280 0x7ff294142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 -- 172.21.15.181:0/44329711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940caa70 con 0x7ff294143370 2020-11-16T22:36:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff293fff700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff298a4b700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 0x7ff294141db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff29405d710 tx=0x7ff284019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284009140 con 0x7ff294143370 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.092+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284006ca0 con 0x7ff294143370 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2940cad10 con 0x7ff294143370 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28401a840 con 0x7ff294143370 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2941422a0 con 0x7ff294143370 2020-11-16T22:36:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff294069eb0 con 0x7ff294143370 2020-11-16T22:36:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff28401ac30 con 0x7ff294143370 2020-11-16T22:36:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff273fff700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 0x7ff26c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff28406f660 con 0x7ff294143370 2020-11-16T22:36:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff293fff700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 0x7ff26c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:00.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.096+0000 7ff293fff700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 0x7ff26c057520 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff28c001e30 tx=0x7ff28c0077d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:00.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.100+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff2840458a0 con 0x7ff294143370 2020-11-16T22:36:00.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.304+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff294062210 con 0x7ff294143370 2020-11-16T22:36:00.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.304+0000 7ff273fff700 1 -- 172.21.15.181:0/44329711 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4668 (secure 0 0 0) 0x7ff284016090 con 0x7ff294143370 2020-11-16T22:36:00.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 msgr2=0x7ff26c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 0x7ff26c057520 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff28c001e30 tx=0x7ff28c0077d0).stop 2020-11-16T22:36:00.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 msgr2=0x7ff294141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 0x7ff294141db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff29405d710 tx=0x7ff284019000).stop 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 shutdown_connections 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff26c055060 0x7ff26c057520 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff294143370 0x7ff294141db0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff280004e50 0x7ff29416cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 --2- 172.21.15.181:0/44329711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff29416d280 0x7ff294142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:00.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 >> 172.21.15.181:0/44329711 conn(0x7ff2940d3680 msgr2=0x7ff29415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:00.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 shutdown_connections 2020-11-16T22:36:00.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:00.308+0000 7ff29a4ae700 1 -- 172.21.15.181:0/44329711 wait complete. 2020-11-16T22:36:00.910 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 121 2020-11-16T22:36:00.910 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:00.910 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":121,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:35:57.900685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:01.114 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d flush journal 2020-11-16T22:36:01.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 Processor -- start 2020-11-16T22:36:01.249 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 -- start start 2020-11-16T22:36:01.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2040142660 0x7f2040142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 0x7f2040061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2040062700 con 0x7f2040160530 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400629a0 con 0x7f2040142660 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20469bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2040062c40 con 0x7f2040063620 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2040142660 0x7f2040142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44720/0 (socket says 172.21.15.181:44720) 2020-11-16T22:36:01.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20451b9700 1 -- 172.21.15.181:0/528252050 learned_addr learned my addr 172.21.15.181:0/528252050 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:01.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20461bb700 1 --2- 172.21.15.181:0/528252050 >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 0x7f2040061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 -- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 msgr2=0x7f2040063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:01.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 -- 172.21.15.181:0/528252050 >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 msgr2=0x7f2040061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:01.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 --2- 172.21.15.181:0/528252050 >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 0x7f2040061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20459ba700 1 -- 172.21.15.181:0/528252050 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040062ee0 con 0x7f2040142660 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20451b9700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.260+0000 7f20461bb700 1 --2- 172.21.15.181:0/528252050 >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 0x7f2040061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20459ba700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f2040142a40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f2040066500 tx=0x7f203c006df0).ready entity=mon.1 client_cookie=5d912fc7d76c8e63 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20449b8700 1 -- 172.21.15.181:0/528252050 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c009840 con 0x7f2040142660 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20449b8700 1 -- 172.21.15.181:0/528252050 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c0099a0 con 0x7f2040142660 2020-11-16T22:36:01.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20449b8700 1 -- 172.21.15.181:0/528252050 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c009d20 con 0x7f2040142660 2020-11-16T22:36:01.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 -- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 msgr2=0x7f2040142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f2040142a40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f2040066500 tx=0x7f203c006df0).stop 2020-11-16T22:36:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 -- 172.21.15.181:0/528252050 shutdown_connections 2020-11-16T22:36:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 --2- 172.21.15.181:0/528252050 >> v2:172.21.15.175:3300/0 conn(0x7f2040160530 0x7f2040061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f2040142a40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 --2- 172.21.15.181:0/528252050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f2040063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 -- 172.21.15.181:0/528252050 >> 172.21.15.181:0/528252050 conn(0x7f20400d2dc0 msgr2=0x7f20400ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 -- 172.21.15.181:0/528252050 shutdown_connections 2020-11-16T22:36:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.264+0000 7f20469bc700 1 -- 172.21.15.181:0/528252050 wait complete. 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 Processor -- start 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 -- start start 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f20400cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040160530 0x7f20400b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400cdd10 con 0x7f2040160530 2020-11-16T22:36:01.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400b4e90 con 0x7f2040142660 2020-11-16T22:36:01.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20469bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2040138df0 con 0x7f2040063620 2020-11-16T22:36:01.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f20400cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36886/0 (socket says 172.21.15.181:36886) 2020-11-16T22:36:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20451b9700 1 -- 172.21.15.181:0/1899199290 learned_addr learned my addr 172.21.15.181:0/1899199290 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:01.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.268+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f20400cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44726/0 (socket says 172.21.15.181:44726) 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20461bb700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040160530 0x7f20400b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 -- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 msgr2=0x7f20400cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f20400cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 -- 172.21.15.181:0/1899199290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040160530 msgr2=0x7f20400b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040160530 0x7f20400b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 -- 172.21.15.181:0/1899199290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040173140 con 0x7f2040142660 2020-11-16T22:36:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20451b9700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f203400ed40 tx=0x7f2034006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:01.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 -- 172.21.15.181:0/1899199290 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400cb00 con 0x7f2040142660 2020-11-16T22:36:01.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 -- 172.21.15.181:0/1899199290 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203400cc60 con 0x7f2040142660 2020-11-16T22:36:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 -- 172.21.15.181:0/1899199290 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340123f0 con 0x7f2040142660 2020-11-16T22:36:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401733e0 con 0x7f2040142660 2020-11-16T22:36:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20400cd0e0 con 0x7f2040142660 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 -- 172.21.15.181:0/1899199290 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2034022070 con 0x7f2040142660 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 0x7f2018057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.272+0000 7f2031ffb700 1 -- 172.21.15.181:0/1899199290 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f203406e3a0 con 0x7f2040142660 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.276+0000 7f20459ba700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 0x7f2018057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.276+0000 7f20459ba700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 0x7f2018057520 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f203c006e30 tx=0x7f203c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.276+0000 7f20566af740 1 Processor -- start 2020-11-16T22:36:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.276+0000 7f20566af740 1 -- start start 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e464b0 con 0x1d9c7b0 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1efa930 con 0x1d2ebb0 2020-11-16T22:36:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dba6f0 con 0x1fbafd0 2020-11-16T22:36:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44738/0 (socket says 172.21.15.181:44738) 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36894/0 (socket says 172.21.15.181:36894) 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 -- 172.21.15.181:0/632854025 learned_addr learned my addr 172.21.15.181:0/632854025 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20459ba700 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 -- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 msgr2=0x1fbb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 -- 172.21.15.181:0/632854025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 msgr2=0x1fbd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 -- 172.21.15.181:0/632854025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1db55a0 con 0x1fbafd0 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20451b9700 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20461bb700 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f203800cd50 tx=0x7f2038008040).ready entity=mon.2 client_cookie=3ae8899b05a2d787 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f201effd700 1 -- 172.21.15.181:0/632854025 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800c780 con 0x1fbafd0 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f201effd700 1 -- 172.21.15.181:0/632854025 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038011bb0 con 0x1fbafd0 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f201effd700 1 -- 172.21.15.181:0/632854025 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038015480 con 0x1fbafd0 2020-11-16T22:36:02.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 msgr2=0x1fb4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f203800cd50 tx=0x7f2038008040).stop 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- 172.21.15.181:0/632854025 shutdown_connections 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1fbafd0 0x1fb4700 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 --2- 172.21.15.181:0/632854025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x1fbb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- 172.21.15.181:0/632854025 >> 172.21.15.181:0/632854025 conn(0x1f6ae40 msgr2=0x1fb73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:02.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- 172.21.15.181:0/632854025 shutdown_connections 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- 172.21.15.181:0/632854025 wait complete. 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 -- start start 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.280+0000 7f20566af740 1 Processor -- start 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x201a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1fbafd0 0x1fda1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ee76e0 con 0x1fbafd0 2020-11-16T22:36:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ee7820 con 0x1d2ebb0 2020-11-16T22:36:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x201a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x201a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36898/0 (socket says 172.21.15.181:36898) 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44742/0 (socket says 172.21.15.181:44742) 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20459ba700 1 -- 172.21.15.181:0/1898534683 learned_addr learned my addr 172.21.15.181:0/1898534683 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ee8a90 con 0x1d9c7b0 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 10 client.0 ll_register_callbacks cb 0x1eddad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20461bb700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1fbafd0 0x1fda1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 -- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 msgr2=0x201a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x201a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 -- 172.21.15.181:0/1898534683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1fbafd0 msgr2=0x1fda1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1fbafd0 0x1fda1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 -- 172.21.15.181:0/1898534683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ed8280 con 0x1d9c7b0 2020-11-16T22:36:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20451b9700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2034047fc0 tx=0x7f203401d040).ready entity=mon.2 client_cookie=a9c02bdf6802c86b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203406cd80 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20340440b0 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f9b6c0 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2034047220 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 10 client.7562 fetch_fsmap learned FSMap version 121 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x1d0dfd0 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f9b6c0 con 0x1d9c7b0 2020-11-16T22:36:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 121) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f2034047380 con 0x1d9c7b0 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 10 client.7562 fetch_fsmap finished waiting for FSMap version 121 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 10 client.7562 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 20 client.7562 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2008a40 0x1fbca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 4 client.7562 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1dd8c50 con 0x2008a40 2020-11-16T22:36:02.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20459ba700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2008a40 0x1fbca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f2013fff700 10 client.7562 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:02.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.284+0000 7f20459ba700 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2008a40 0x1fbca90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c2d9e370d261a860 server_cookie=4297be576644d50d in_seq=0 out_seq=0 2020-11-16T22:36:02.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f2013fff700 1 -- 172.21.15.181:0/1898534683 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1dd8c50 con 0x2008a40 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f2013fff700 10 client.7562 handle_command_reply: tid=0 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 client.7562 shutdown 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 msgr2=0x1fb7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2034047fc0 tx=0x7f203401d040).stop 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 shutdown_connections 2020-11-16T22:36:02.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2008a40 0x1fbca90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1fbafd0 0x1fda1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d2ebb0 0x201a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 --2- 172.21.15.181:0/1898534683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d9c7b0 0x1fb7530 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 >> 172.21.15.181:0/1898534683 conn(0x1f6ae40 msgr2=0x1fee270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f2013fff700 0 client.7562 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:02.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 shutdown_connections 2020-11-16T22:36:02.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 1 -- 172.21.15.181:0/1898534683 wait complete. 2020-11-16T22:36:02.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.288+0000 7f20566af740 20 client.7562 trim_cache size 0 max 16384 2020-11-16T22:36:02.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.304+0000 7f20566af740 1 Processor -- start 2020-11-16T22:36:02.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.304+0000 7f20566af740 1 -- start start 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f6ae40 0x1fbca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x1fd9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ebfe40 con 0x1f6ae40 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x201b520 con 0x1d9c7b0 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2007ec0 con 0x1d2ebb0 2020-11-16T22:36:02.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36908/0 (socket says 172.21.15.181:36908) 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 -- 172.21.15.181:0/2819706896 learned_addr learned my addr 172.21.15.181:0/2819706896 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20451b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x1fd9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20459ba700 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f6ae40 0x1fbca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 -- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 msgr2=0x1fd9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x1fd9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 -- 172.21.15.181:0/2819706896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f6ae40 msgr2=0x1fbca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f6ae40 0x1fbca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 -- 172.21.15.181:0/2819706896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c0081d0 con 0x1d9c7b0 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f203800c440 tx=0x7f2038008040).ready entity=mon.1 client_cookie=818a9dbf60cf78d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f201f7fe700 1 -- 172.21.15.181:0/2819706896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038026070 con 0x1d9c7b0 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f201f7fe700 1 -- 172.21.15.181:0/2819706896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038015480 con 0x1d9c7b0 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f201f7fe700 1 -- 172.21.15.181:0/2819706896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038005c60 con 0x1d9c7b0 2020-11-16T22:36:02.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 msgr2=0x1fb9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f203800c440 tx=0x7f2038008040).stop 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- 172.21.15.181:0/2819706896 shutdown_connections 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d9c7b0 0x1fb9da0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f6ae40 0x1fbca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- 172.21.15.181:0/2819706896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x1fd9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:02.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- 172.21.15.181:0/2819706896 >> 172.21.15.181:0/2819706896 conn(0x7f200c002320 msgr2=0x1fb4700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- 172.21.15.181:0/2819706896 shutdown_connections 2020-11-16T22:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- 172.21.15.181:0/2819706896 wait complete. 2020-11-16T22:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- start start 2020-11-16T22:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 Processor -- start 2020-11-16T22:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f6ae40 0x1fb9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x206d730 con 0x1d9c7b0 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2005630 con 0x1f6ae40 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x206ccb0 con 0x1d2ebb0 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20566af740 10 client.0 ll_register_callbacks cb 0x1fbbaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20461bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f6ae40 0x1fb9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20459ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44754/0 (socket says 172.21.15.181:44754) 2020-11-16T22:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20459ba700 1 -- 172.21.15.181:0/2344158429 learned_addr learned my addr 172.21.15.181:0/2344158429 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.308+0000 7f20451b9700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 -- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f6ae40 msgr2=0x1fb9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:03.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f6ae40 0x1fb9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 -- 172.21.15.181:0/2344158429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 msgr2=0x1fbd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:03.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 -- 172.21.15.181:0/2344158429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d4c720 con 0x1d2ebb0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20459ba700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f203c000dd0 tx=0x7f203c031280).ready entity=mon.2 client_cookie=9f56dbcb81971a4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011d50 con 0x1d2ebb0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f203c023030 con 0x1d2ebb0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2038011bb0 con 0x1d2ebb0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203c01e640 con 0x1d2ebb0 2020-11-16T22:36:03.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 10 client.7568 fetch_fsmap learned FSMap version 121 2020-11-16T22:36:03.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x1d57e00 con 0x1d2ebb0 2020-11-16T22:36:03.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038011bb0 con 0x1d2ebb0 2020-11-16T22:36:03.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 121) v1 ==== 2607+0+0 (secure 0 0 0) 0x7f203c028040 con 0x1d2ebb0 2020-11-16T22:36:03.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 10 client.7568 fetch_fsmap finished waiting for FSMap version 121 2020-11-16T22:36:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 10 client.7568 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 20 client.7568 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1fbafd0 0x1fba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 4 client.7568 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1ddff50 con 0x1fbafd0 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20451b9700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1fbafd0 0x1fba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f20451b9700 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1fbafd0 0x1fba290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7e9c60051e585c35 server_cookie=a748312a4c0a739 in_seq=0 out_seq=0 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.312+0000 7f2013fff700 10 client.7568 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f2013fff700 1 -- 172.21.15.181:0/2344158429 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1ddff50 con 0x1fbafd0 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f2013fff700 10 client.7568 handle_command_reply: tid=0 2020-11-16T22:36:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 client.7568 shutdown 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 msgr2=0x2005140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f203c000dd0 tx=0x7f203c031280).stop 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 shutdown_connections 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1fbafd0 0x1fba290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f6ae40 0x1fb9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d2ebb0 0x2005140 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 --2- 172.21.15.181:0/2344158429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d9c7b0 0x1fbd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f2013fff700 0 client.7568 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:03.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 >> 172.21.15.181:0/2344158429 conn(0x7f200c002320 msgr2=0x1fd9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 shutdown_connections 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.336+0000 7f20566af740 1 -- 172.21.15.181:0/2344158429 wait complete. 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20566af740 20 client.7568 trim_cache size 0 max 16384 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 msgr2=0x7f2018057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 0x7f2018057520 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f203c006e30 tx=0x7f203c015000).stop 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 msgr2=0x7f20400b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f203400ed40 tx=0x7f2034006eb0).stop 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 shutdown_connections 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2018055060 0x7f2018057520 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2040160530 0x7f20400b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2040063620 0x7f20400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 --2- 172.21.15.181:0/1899199290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2040142660 0x7f20400b49a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 >> 172.21.15.181:0/1899199290 conn(0x7f20400d2dc0 msgr2=0x7f2040065160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 shutdown_connections 2020-11-16T22:36:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:01.340+0000 7f20469bc700 1 -- 172.21.15.181:0/1899199290 wait complete. 2020-11-16T22:36:03.725 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:03.725 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:36:03.725 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:03.725 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:03.726 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:03.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 Processor -- start 2020-11-16T22:36:03.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 -- start start 2020-11-16T22:36:03.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e70062740 0x7f2e700644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:03.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e700c8db0 con 0x7f2e70161480 2020-11-16T22:36:03.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e70165030 con 0x7f2e70062740 2020-11-16T22:36:03.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700c8330 con 0x7f2e70161860 2020-11-16T22:36:03.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e70062740 0x7f2e700644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6e59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2e70161480 0x7f2e70164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51224/0 (socket says 172.21.15.181:51224) 2020-11-16T22:36:03.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6e59c700 1 -- 172.21.15.181:0/3360359870 learned_addr learned my addr 172.21.15.181:0/3360359870 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:03.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6dd9b700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:03.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 msgr2=0x7f2e70062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:03.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/3360359870 >> v1:172.21.15.181:6789/0 conn(0x7f2e70161480 legacy=0x7f2e70164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/3360359870 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e700c85d0 con 0x7f2e70062740 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6dd9b700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6ed9d700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e700644e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f2e68009c90 tx=0x7f2e680091d0).ready entity=mon.1 client_cookie=7012121f2480d529 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6d59a700 1 -- 172.21.15.181:0/3360359870 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6800bbc0 con 0x7f2e70062740 2020-11-16T22:36:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6d59a700 1 -- 172.21.15.181:0/3360359870 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6800bd20 con 0x7f2e70062740 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.868+0000 7f2e6d59a700 1 -- 172.21.15.181:0/3360359870 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68011610 con 0x7f2e70062740 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 msgr2=0x7f2e700644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e700644e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f2e68009c90 tx=0x7f2e680091d0).stop 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/3360359870 shutdown_connections 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e700644e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/3360359870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/3360359870 >> 172.21.15.181:0/3360359870 conn(0x7f2e700d2dc0 msgr2=0x7f2e700ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/3360359870 shutdown_connections 2020-11-16T22:36:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/3360359870 wait complete. 2020-11-16T22:36:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 Processor -- start 2020-11-16T22:36:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- start start 2020-11-16T22:36:04.068 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e70173650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 0x7f2e70143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70145e60 con 0x7f2e70161480 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70173c70 con 0x7f2e70062740 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70173f10 con 0x7f2e70161860 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e70173650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44772/0 (socket says 172.21.15.181:44772) 2020-11-16T22:36:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/1994003291 learned_addr learned my addr 172.21.15.181:0/1994003291 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6dd9b700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 0x7f2e70143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 msgr2=0x7f2e70173650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e70173650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/1994003291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 msgr2=0x7f2e70143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:04.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 0x7f2e70143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 -- 172.21.15.181:0/1994003291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e701741b0 con 0x7f2e70161860 2020-11-16T22:36:04.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6dd9b700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 0x7f2e70143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6ed9d700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2e68009f80 tx=0x7f2e68016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68010780 con 0x7f2e70161860 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680108e0 con 0x7f2e70161860 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e70174450 con 0x7f2e70161860 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68010c60 con 0x7f2e70161860 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.872+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e701444e0 con 0x7f2e70161860 2020-11-16T22:36:04.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e70044f60 con 0x7f2e70161860 2020-11-16T22:36:04.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2e68024030 con 0x7f2e70161860 2020-11-16T22:36:04.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e667fc700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 0x7f2e40057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e6806f7a0 con 0x7f2e70161860 2020-11-16T22:36:04.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e6e59c700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 0x7f2e40057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:04.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.876+0000 7f2e6e59c700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 0x7f2e40057550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2e700c91e0 tx=0x7f2e60009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:04.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:03.880+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2e68045ab0 con 0x7f2e70161860 2020-11-16T22:36:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e700671e0 con 0x7f2e70161860 2020-11-16T22:36:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e667fc700 1 -- 172.21.15.181:0/1994003291 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f2e68015070 con 0x7f2e70161860 2020-11-16T22:36:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 msgr2=0x7f2e40057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:04.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 0x7f2e40057550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2e700c91e0 tx=0x7f2e60009040).stop 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 msgr2=0x7f2e70146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2e68009f80 tx=0x7f2e68016000).stop 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 shutdown_connections 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2e40055090 0x7f2e40057550 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e70161860 0x7f2e70146530 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70062740 0x7f2e70173650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 --2- 172.21.15.181:0/1994003291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e70161480 0x7f2e70143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:04.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 >> 172.21.15.181:0/1994003291 conn(0x7f2e700d2dc0 msgr2=0x7f2e701472b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:04.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 shutdown_connections 2020-11-16T22:36:04.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.084+0000 7f2e6f59e700 1 -- 172.21.15.181:0/1994003291 wait complete. 2020-11-16T22:36:04.683 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:04.683 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:04.683 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:04.684 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i flush journal 2020-11-16T22:36:04.685 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:04.688 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:04.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 Processor -- start 2020-11-16T22:36:04.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 -- start start 2020-11-16T22:36:04.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a840cf770 0x7f1a840600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 0x7f1a840660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:04.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a840cfbf0 con 0x7f1a840cf390 2020-11-16T22:36:04.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a84065190 con 0x7f1a840cf770 2020-11-16T22:36:04.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84065430 con 0x7f1a840605e0 2020-11-16T22:36:04.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 0x7f1a840660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1a840cf390 0x7f1a84163970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51238/0 (socket says 172.21.15.181:51238) 2020-11-16T22:36:05.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8259c700 1 -- 172.21.15.181:0/3660301145 learned_addr learned my addr 172.21.15.181:0/3660301145 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:05.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/3660301145 >> v2:172.21.15.181:3300/0 conn(0x7f1a840cf770 0x7f1a840600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8159a700 1 -- 172.21.15.181:0/3660301145 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890540966 0 0) 0x7f1a840cfbf0 con 0x7f1a840cf390 2020-11-16T22:36:05.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8159a700 1 -- 172.21.15.181:0/3660301145 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a64003240 con 0x7f1a840cf390 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3660301145 >> v2:172.21.15.181:3300/0 conn(0x7f1a840cf770 msgr2=0x7f1a840600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/3660301145 >> v2:172.21.15.181:3300/0 conn(0x7f1a840cf770 0x7f1a840600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3660301145 >> v1:172.21.15.181:6789/0 conn(0x7f1a840cf390 legacy=0x7f1a84163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3660301145 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840656d0 con 0x7f1a840605e0 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/3660301145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 0x7f1a840660b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f1a7c009c90 tx=0x7f1a7c0091d0).ready entity=mon.2 client_cookie=e99bc93e3f06a7ae server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:05.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8159a700 1 -- 172.21.15.181:0/3660301145 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00b9c0 con 0x7f1a840605e0 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8159a700 1 -- 172.21.15.181:0/3660301145 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c00bb20 con 0x7f1a840605e0 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8159a700 1 -- 172.21.15.181:0/3660301145 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00bea0 con 0x7f1a840605e0 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 -- 172.21.15.181:0/3660301145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 msgr2=0x7f1a840660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.828+0000 7f1a8359e700 1 --2- 172.21.15.181:0/3660301145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 0x7f1a840660b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f1a7c009c90 tx=0x7f1a7c0091d0).stop 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- 172.21.15.181:0/3660301145 shutdown_connections 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 --2- 172.21.15.181:0/3660301145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840605e0 0x7f1a840660b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 --2- 172.21.15.181:0/3660301145 >> v2:172.21.15.181:3300/0 conn(0x7f1a840cf770 0x7f1a840600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- 172.21.15.181:0/3660301145 >> 172.21.15.181:0/3660301145 conn(0x7f1a840cfe90 msgr2=0x7f1a840d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- 172.21.15.181:0/3660301145 shutdown_connections 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- 172.21.15.181:0/3660301145 wait complete. 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 Processor -- start 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- start start 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840cf390 0x7f1a841656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a840cf770 0x7f1a84162990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8416fe50 con 0x7f1a840cf770 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84164a30 con 0x7f1a840605e0 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84164cd0 con 0x7f1a840cf390 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36940/0 (socket says 172.21.15.181:36940) 2020-11-16T22:36:05.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 -- 172.21.15.181:0/1571438487 learned_addr learned my addr 172.21.15.181:0/1571438487 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:05.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840cf390 0x7f1a841656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a840cf770 0x7f1a84162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 -- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840cf390 msgr2=0x7f1a841656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:05.238 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840cf390 0x7f1a841656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 -- 172.21.15.181:0/1571438487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a840cf770 msgr2=0x7f1a84162990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a840cf770 0x7f1a84162990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 -- 172.21.15.181:0/1571438487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84164f70 con 0x7f1a840605e0 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.832+0000 7f1a8259c700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f1a74001e90 tx=0x7f1a74007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 -- 172.21.15.181:0/1571438487 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740072e0 con 0x7f1a840605e0 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 -- 172.21.15.181:0/1571438487 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74007440 con 0x7f1a840605e0 2020-11-16T22:36:05.239 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a84165210 con 0x7f1a840605e0 2020-11-16T22:36:05.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 -- 172.21.15.181:0/1571438487 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74005820 con 0x7f1a840605e0 2020-11-16T22:36:05.240 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a84162e80 con 0x7f1a840605e0 2020-11-16T22:36:05.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 -- 172.21.15.181:0/1571438487 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1a74010070 con 0x7f1a840605e0 2020-11-16T22:36:05.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 0x7f1a60057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a7a7fc700 1 -- 172.21.15.181:0/1571438487 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a7406d2d0 con 0x7f1a840605e0 2020-11-16T22:36:05.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 0x7f1a60057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.836+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 0x7f1a60057550 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1a6c006fd0 tx=0x7f1a6c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.840+0000 7f1a979f2740 1 Processor -- start 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.840+0000 7f1a979f2740 1 -- start start 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1d6d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d6cfd0 0x1d66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bf84b0 con 0x1d6cfd0 2020-11-16T22:36:05.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cac930 con 0x1b4e7b0 2020-11-16T22:36:05.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b6c6f0 con 0x1ae0bb0 2020-11-16T22:36:05.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36948/0 (socket says 172.21.15.181:36948) 2020-11-16T22:36:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a8259c700 1 -- 172.21.15.181:0/2993535458 learned_addr learned my addr 172.21.15.181:0/2993535458 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1d6d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d6cfd0 0x1d66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 msgr2=0x1d6f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2993535458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d6cfd0 msgr2=0x1d66700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d6cfd0 0x1d66700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2993535458 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b675a0 con 0x1ae0bb0 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1d6d670 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1a6c014d30 tx=0x7f1a6c013000).ready entity=mon.2 client_cookie=b88916827e2813b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a5b7fe700 1 -- 172.21.15.181:0/2993535458 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c014020 con 0x1ae0bb0 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a5b7fe700 1 -- 172.21.15.181:0/2993535458 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c014180 con 0x1ae0bb0 2020-11-16T22:36:05.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a5b7fe700 1 -- 172.21.15.181:0/2993535458 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c011be0 con 0x1ae0bb0 2020-11-16T22:36:05.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 msgr2=0x1d6d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:05.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1d6d670 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1a6c014d30 tx=0x7f1a6c013000).stop 2020-11-16T22:36:05.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- 172.21.15.181:0/2993535458 shutdown_connections 2020-11-16T22:36:05.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d6cfd0 0x1d66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2993535458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1d6d670 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- 172.21.15.181:0/2993535458 >> 172.21.15.181:0/2993535458 conn(0x1d1ce40 msgr2=0x1d693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- 172.21.15.181:0/2993535458 shutdown_connections 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.844+0000 7f1a979f2740 1 -- 172.21.15.181:0/2993535458 wait complete. 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- start start 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 Processor -- start 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ae0bb0 0x1d69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 0x1dcc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c996e0 con 0x1ae0bb0 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c99820 con 0x1d6cfd0 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c9aa90 con 0x1b4e7b0 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 10 client.0 ll_register_callbacks cb 0x1c8fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 0x1dcc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36958/0 (socket says 172.21.15.181:36958) 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3126158555 learned_addr learned my addr 172.21.15.181:0/3126158555 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a8259c700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ae0bb0 0x1d69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 msgr2=0x1dcc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 0x1dcc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ae0bb0 msgr2=0x1d69490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:05.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ae0bb0 0x1d69490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 -- 172.21.15.181:0/3126158555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c8a280 con 0x1d6cfd0 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 0x1dcc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f1a84065570 tx=0x7f1a7c016000).ready entity=mon.1 client_cookie=547c1063e841462a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c022070 con 0x1d6cfd0 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d4d6c0 con 0x1d6cfd0 2020-11-16T22:36:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a7c009960 con 0x1d6cfd0 2020-11-16T22:36:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7c00bea0 con 0x1d6cfd0 2020-11-16T22:36:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 10 client.8362 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x1abffd0 con 0x1d6cfd0 2020-11-16T22:36:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d4d6c0 con 0x1d6cfd0 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1a7c0103a0 con 0x1d6cfd0 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.848+0000 7f1a979f2740 10 client.8362 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 10 client.8362 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 20 client.8362 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1dbfcb0 0x1d8c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 4 client.8362 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1b8ac50 con 0x1dbfcb0 2020-11-16T22:36:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a8259c700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1dbfcb0 0x1d8c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a8259c700 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1dbfcb0 0x1d8c6c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=9cf356ef63d61c44 server_cookie=687e90b978a6758d in_seq=0 out_seq=0 2020-11-16T22:36:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a58ff9700 10 client.8362 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a58ff9700 1 -- 172.21.15.181:0/3126158555 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1b8ac50 con 0x1dbfcb0 2020-11-16T22:36:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a58ff9700 10 client.8362 handle_command_reply: tid=0 2020-11-16T22:36:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 client.8362 shutdown 2020-11-16T22:36:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 msgr2=0x1dcc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f1a84065570 tx=0x7f1a7c016000).stop 2020-11-16T22:36:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 shutdown_connections 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1dbfcb0 0x1d8c6c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d6cfd0 0x1dcc7f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ae0bb0 0x1d69490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 --2- 172.21.15.181:0/3126158555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1b4e7b0 0x1dcc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a58ff9700 0 client.8362 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 >> 172.21.15.181:0/3126158555 conn(0x1d1ce40 msgr2=0x1d67600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 shutdown_connections 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 1 -- 172.21.15.181:0/3126158555 wait complete. 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.852+0000 7f1a979f2740 20 client.8362 trim_cache size 0 max 16384 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.872+0000 7f1a979f2740 1 Processor -- start 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.872+0000 7f1a979f2740 1 -- start start 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 0x1d61730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 0x1dcc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c71e40 con 0x1b4e7b0 2020-11-16T22:36:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d9dd50 con 0x1d1ce40 2020-11-16T22:36:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1db9820 con 0x1ae0bb0 2020-11-16T22:36:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44804/0 (socket says 172.21.15.181:44804) 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2349023588 learned_addr learned my addr 172.21.15.181:0/2349023588 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a8259c700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 0x1d61730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 0x1dcc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 msgr2=0x1d61730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 0x1d61730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2349023588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 msgr2=0x1dcc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:06.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 0x1dcc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 -- 172.21.15.181:0/2349023588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0081d0 con 0x1ae0bb0 2020-11-16T22:36:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a8259c700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 0x1d61730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 0x1dcc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:06.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f1a6c01d000 tx=0x7f1a6c01e000).ready entity=mon.2 client_cookie=2b994a088b326ffa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:06.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a5bfff700 1 -- 172.21.15.181:0/2349023588 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0220d0 con 0x1ae0bb0 2020-11-16T22:36:06.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a5bfff700 1 -- 172.21.15.181:0/2349023588 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c029040 con 0x1ae0bb0 2020-11-16T22:36:06.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a5bfff700 1 -- 172.21.15.181:0/2349023588 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c022760 con 0x1ae0bb0 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 msgr2=0x1da0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f1a6c01d000 tx=0x7f1a6c01e000).stop 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- 172.21.15.181:0/2349023588 shutdown_connections 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1b4e7b0 0x1dcc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1ce40 0x1d61730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- 172.21.15.181:0/2349023588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1da0c90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- 172.21.15.181:0/2349023588 >> 172.21.15.181:0/2349023588 conn(0x7f1a4c002320 msgr2=0x1d69490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- 172.21.15.181:0/2349023588 shutdown_connections 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- 172.21.15.181:0/2349023588 wait complete. 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- start start 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 Processor -- start 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d1ce40 0x1d61b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dbd970 con 0x1d1ce40 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d6d9a0 con 0x1b4e7b0 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd0400 con 0x1ae0bb0 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a979f2740 10 client.0 ll_register_callbacks cb 0x1dcce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44810/0 (socket says 172.21.15.181:44810) 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a8259c700 1 -- 172.21.15.181:0/68830507 learned_addr learned my addr 172.21.15.181:0/68830507 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.876+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d1ce40 0x1d61b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 -- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 msgr2=0x1d6d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 -- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d1ce40 msgr2=0x1d61b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:06.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d1ce40 0x1d61b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 -- 172.21.15.181:0/68830507 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1afe720 con 0x1ae0bb0 2020-11-16T22:36:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a8259c700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f1a740436f0 tx=0x7f1a7400cf90).ready entity=mon.2 client_cookie=d6367e824e80c571 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7404e040 con 0x1ae0bb0 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a740468f0 con 0x1ae0bb0 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6c014180 con 0x1ae0bb0 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a74005de0 con 0x1ae0bb0 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 10 client.7592 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x1b09e00 con 0x1ae0bb0 2020-11-16T22:36:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c014180 con 0x1ae0bb0 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f1a74053040 con 0x1ae0bb0 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 10 client.7592 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 10 client.7592 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 20 client.7592 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1d6cfd0 0x1dcfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 4 client.7592 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:36:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1b91f50 con 0x1d6cfd0 2020-11-16T22:36:07.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1d6cfd0 0x1dcfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a58ff9700 10 client.7592 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:07.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.880+0000 7f1a81d9b700 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1d6cfd0 0x1dcfa30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=2892cc51d8fd70f9 server_cookie=61dfddf4d94bf2d0 in_seq=0 out_seq=0 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a58ff9700 1 -- 172.21.15.181:0/68830507 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1b91f50 con 0x1d6cfd0 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a58ff9700 10 client.7592 handle_command_reply: tid=0 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 client.7592 shutdown 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 msgr2=0x1dbd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f1a740436f0 tx=0x7f1a7400cf90).stop 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 shutdown_connections 2020-11-16T22:36:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1d6cfd0 0x1dcfa30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d1ce40 0x1d61b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ae0bb0 0x1dbd480 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a58ff9700 0 client.7592 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:07.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 --2- 172.21.15.181:0/68830507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1b4e7b0 0x1d6d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 >> 172.21.15.181:0/68830507 conn(0x7f1a4c002320 msgr2=0x1d670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 shutdown_connections 2020-11-16T22:36:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 1 -- 172.21.15.181:0/68830507 wait complete. 2020-11-16T22:36:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a979f2740 20 client.7592 trim_cache size 0 max 16384 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 msgr2=0x7f1a60057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 0x7f1a60057550 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1a6c006fd0 tx=0x7f1a6c006df0).stop 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 msgr2=0x7f1a840b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f1a74001e90 tx=0x7f1a74007ac0).stop 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 shutdown_connections 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1a60055090 0x7f1a60057550 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a840cf770 0x7f1a84162990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a840605e0 0x7f1a840b6380 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 --2- 172.21.15.181:0/1571438487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a840cf390 0x7f1a841656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 >> 172.21.15.181:0/1571438487 conn(0x7f1a840cfe90 msgr2=0x7f1a841638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 shutdown_connections 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:04.900+0000 7f1a8359e700 1 -- 172.21.15.181:0/1571438487 wait complete. 2020-11-16T22:36:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:07.289 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:36:07.289 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:07.289 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:07.289 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:07.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 Processor -- start 2020-11-16T22:36:07.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 -- start start 2020-11-16T22:36:07.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f935c145ce0 0x7f935c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f935c0814c0 0x7f935c0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0cc060 con 0x7f935c145900 2020-11-16T22:36:07.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c15fdb0 con 0x7f935c145ce0 2020-11-16T22:36:07.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9361869700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c146140 con 0x7f935c0814c0 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f935c145900 0x7f935c0cb850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51280/0 (socket says 172.21.15.181:51280) 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935affd700 1 -- 172.21.15.181:0/2906226982 learned_addr learned my addr 172.21.15.181:0/2906226982 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 --2- 172.21.15.181:0/2906226982 >> v2:172.21.15.181:3300/0 conn(0x7f935c0814c0 0x7f935c0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935a7fc700 1 --2- 172.21.15.181:0/2906226982 >> v2:172.21.15.175:3300/0 conn(0x7f935c145ce0 0x7f935c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9359ffb700 1 -- 172.21.15.181:0/2906226982 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088366878 0 0) 0x7f935c0cc060 con 0x7f935c145900 2020-11-16T22:36:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9359ffb700 1 -- 172.21.15.181:0/2906226982 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9340003240 con 0x7f935c145900 2020-11-16T22:36:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 -- 172.21.15.181:0/2906226982 >> v2:172.21.15.175:3300/0 conn(0x7f935c145ce0 msgr2=0x7f935c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 --2- 172.21.15.181:0/2906226982 >> v2:172.21.15.175:3300/0 conn(0x7f935c145ce0 0x7f935c080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 -- 172.21.15.181:0/2906226982 >> v1:172.21.15.181:6789/0 conn(0x7f935c145900 legacy=0x7f935c0cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 -- 172.21.15.181:0/2906226982 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0cc300 con 0x7f935c0814c0 2020-11-16T22:36:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f935b7fe700 1 --2- 172.21.15.181:0/2906226982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c0814c0 0x7f935c0d1370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f935c083b40 tx=0x7f9344006df0).ready entity=mon.1 client_cookie=3ffc1263fc4741d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9359ffb700 1 -- 172.21.15.181:0/2906226982 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344009640 con 0x7f935c0814c0 2020-11-16T22:36:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9359ffb700 1 -- 172.21.15.181:0/2906226982 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440097a0 con 0x7f935c0814c0 2020-11-16T22:36:07.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.436+0000 7f9359ffb700 1 -- 172.21.15.181:0/2906226982 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344009b20 con 0x7f935c0814c0 2020-11-16T22:36:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- 172.21.15.181:0/2906226982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c0814c0 msgr2=0x7f935c0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 --2- 172.21.15.181:0/2906226982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c0814c0 0x7f935c0d1370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f935c083b40 tx=0x7f9344006df0).stop 2020-11-16T22:36:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- 172.21.15.181:0/2906226982 shutdown_connections 2020-11-16T22:36:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 --2- 172.21.15.181:0/2906226982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c0814c0 0x7f935c0d1370 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 --2- 172.21.15.181:0/2906226982 >> v2:172.21.15.175:3300/0 conn(0x7f935c145ce0 0x7f935c080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- 172.21.15.181:0/2906226982 >> 172.21.15.181:0/2906226982 conn(0x7f935c0d68f0 msgr2=0x7f935c0d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- 172.21.15.181:0/2906226982 shutdown_connections 2020-11-16T22:36:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- 172.21.15.181:0/2906226982 wait complete. 2020-11-16T22:36:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 Processor -- start 2020-11-16T22:36:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.440+0000 7f9361869700 1 -- start start 2020-11-16T22:36:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c0814c0 0x7f935c1611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 0x7f935c171a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c161b50 con 0x7f935c0814c0 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c161df0 con 0x7f935c145900 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c162090 con 0x7f935c145ce0 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 0x7f935c171a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44828/0 (socket says 172.21.15.181:44828) 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 -- 172.21.15.181:0/1277319820 learned_addr learned my addr 172.21.15.181:0/1277319820 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935affd700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c0814c0 0x7f935c1611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 -- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 msgr2=0x7f935c171a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 0x7f935c171a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 -- 172.21.15.181:0/1277319820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c0814c0 msgr2=0x7f935c1611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c0814c0 0x7f935c1611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 -- 172.21.15.181:0/1277319820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c162330 con 0x7f935c145ce0 2020-11-16T22:36:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935a7fc700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 0x7f935c171a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f935b7fe700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f9344006fd0 tx=0x7f9344015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344011cc0 con 0x7f935c145ce0 2020-11-16T22:36:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9344011e20 con 0x7f935c145ce0 2020-11-16T22:36:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934401d680 con 0x7f935c145ce0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c1625d0 con 0x7f935c145ce0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.444+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1616e0 con 0x7f935c145ce0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c170320 con 0x7f935c145ce0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f934400f070 con 0x7f935c145ce0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f9352ffd700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 0x7f932c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f935affd700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 0x7f932c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f934406fcb0 con 0x7f935c145ce0 2020-11-16T22:36:07.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.448+0000 7f935affd700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 0x7f932c057550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f9354005920 tx=0x7f9354009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:07.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.452+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9344045e00 con 0x7f935c145ce0 2020-11-16T22:36:07.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.656+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f935c069eb0 con 0x7f935c145ce0 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.656+0000 7f9352ffd700 1 -- 172.21.15.181:0/1277319820 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f9344049420 con 0x7f935c145ce0 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 msgr2=0x7f932c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 0x7f932c057550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f9354005920 tx=0x7f9354009040).stop 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 msgr2=0x7f935c171f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f9344006fd0 tx=0x7f9344015000).stop 2020-11-16T22:36:07.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 shutdown_connections 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f932c055090 0x7f932c057550 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f935c145ce0 0x7f935c171f80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c0814c0 0x7f935c1611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 --2- 172.21.15.181:0/1277319820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c145900 0x7f935c171a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 >> 172.21.15.181:0/1277319820 conn(0x7f935c0d68f0 msgr2=0x7f935c159830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 shutdown_connections 2020-11-16T22:36:07.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:07.660+0000 7f9361869700 1 -- 172.21.15.181:0/1277319820 wait complete. 2020-11-16T22:36:08.050 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:08.050 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:08.050 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:08.254 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e flush journal 2020-11-16T22:36:08.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.396+0000 7f035af66700 1 Processor -- start 2020-11-16T22:36:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.396+0000 7f035af66700 1 -- start start 2020-11-16T22:36:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f035415f5b0 0x7f03540d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f03540d1830 0x7f03540629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0354160e40 con 0x7f03540615f0 2020-11-16T22:36:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03541609a0 con 0x7f03540d1830 2020-11-16T22:36:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0354160ba0 con 0x7f035415f5b0 2020-11-16T22:36:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f035415f5b0 0x7f03540d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0353fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f03540615f0 0x7f03540622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51294/0 (socket says 172.21.15.181:51294) 2020-11-16T22:36:08.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0353fff700 1 -- 172.21.15.181:0/2600136095 learned_addr learned my addr 172.21.15.181:0/2600136095 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:08.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0359503700 1 --2- 172.21.15.181:0/2600136095 >> v2:172.21.15.175:3300/0 conn(0x7f03540d1830 0x7f03540629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 -- 172.21.15.181:0/2600136095 >> v2:172.21.15.175:3300/0 conn(0x7f03540d1830 msgr2=0x7f03540629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 --2- 172.21.15.181:0/2600136095 >> v2:172.21.15.175:3300/0 conn(0x7f03540d1830 0x7f03540629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 -- 172.21.15.181:0/2600136095 >> v1:172.21.15.181:6789/0 conn(0x7f03540615f0 legacy=0x7f03540622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 -- 172.21.15.181:0/2600136095 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354060970 con 0x7f035415f5b0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f03537fe700 1 -- 172.21.15.181:0/2600136095 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542517288 0 0) 0x7f0354160e40 con 0x7f03540615f0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f0358d02700 1 --2- 172.21.15.181:0/2600136095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f035415f5b0 0x7f03540d1340 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f0354083ad0 tx=0x7f0348006d90).ready entity=mon.1 client_cookie=18876dcf6457474b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f03537fe700 1 -- 172.21.15.181:0/2600136095 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348006030 con 0x7f035415f5b0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f03537fe700 1 -- 172.21.15.181:0/2600136095 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348006190 con 0x7f035415f5b0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f03537fe700 1 -- 172.21.15.181:0/2600136095 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348006510 con 0x7f035415f5b0 2020-11-16T22:36:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- 172.21.15.181:0/2600136095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f035415f5b0 msgr2=0x7f03540d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 --2- 172.21.15.181:0/2600136095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f035415f5b0 0x7f03540d1340 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f0354083ad0 tx=0x7f0348006d90).stop 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- 172.21.15.181:0/2600136095 shutdown_connections 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 --2- 172.21.15.181:0/2600136095 >> v2:172.21.15.175:3300/0 conn(0x7f03540d1830 0x7f03540629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 --2- 172.21.15.181:0/2600136095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f035415f5b0 0x7f03540d1340 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- 172.21.15.181:0/2600136095 >> 172.21.15.181:0/2600136095 conn(0x7f03540d69a0 msgr2=0x7f03540d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- 172.21.15.181:0/2600136095 shutdown_connections 2020-11-16T22:36:08.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.400+0000 7f035af66700 1 -- 172.21.15.181:0/2600136095 wait complete. 2020-11-16T22:36:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 Processor -- start 2020-11-16T22:36:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 -- start start 2020-11-16T22:36:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 0x7f03540ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03540d1830 0x7f035416f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 0x7f03541596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0354159f80 con 0x7f03540d1830 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035415a220 con 0x7f03540615f0 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f035af66700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035415a4c0 con 0x7f035415f5b0 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 0x7f03540ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0359503700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 0x7f03541596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 0x7f03540ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36996/0 (socket says 172.21.15.181:36996) 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0358d02700 1 -- 172.21.15.181:0/648969312 learned_addr learned my addr 172.21.15.181:0/648969312 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0353fff700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03540d1830 0x7f035416f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.404+0000 7f0359503700 1 -- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 msgr2=0x7f03540ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0359503700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 0x7f03540ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0359503700 1 -- 172.21.15.181:0/648969312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03540d1830 msgr2=0x7f035416f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0359503700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03540d1830 0x7f035416f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0359503700 1 -- 172.21.15.181:0/648969312 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03540b9930 con 0x7f035415f5b0 2020-11-16T22:36:08.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0359503700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 0x7f03541596c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f034c00ed20 tx=0x7f034c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 -- 172.21.15.181:0/648969312 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00eee0 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 -- 172.21.15.181:0/648969312 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00cb00 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03540b9bd0 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 -- 172.21.15.181:0/648969312 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00cea0 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03540d3820 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 -- 172.21.15.181:0/648969312 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f034c022070 con 0x7f035415f5b0 2020-11-16T22:36:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 0x7f032c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0350ff9700 1 -- 172.21.15.181:0/648969312 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f034c06e3b0 con 0x7f035415f5b0 2020-11-16T22:36:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.408+0000 7f0358d02700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 0x7f032c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0358d02700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 0x7f032c057520 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f0348006dd0 tx=0x7f0348015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 Processor -- start 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- start start 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x2599cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x25d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x242c4b0 con 0x2314bb0 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24e0930 con 0x259b470 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23a06f0 con 0x23827b0 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x2599cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37006/0 (socket says 172.21.15.181:37006) 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 -- 172.21.15.181:0/3464799413 learned_addr learned my addr 172.21.15.181:0/3464799413 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0353fff700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x25d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 -- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 msgr2=0x2599cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x2599cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 -- 172.21.15.181:0/3464799413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 msgr2=0x25d8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x25d8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 -- 172.21.15.181:0/3464799413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x239b5a0 con 0x259b470 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0358d02700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x2599cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:08.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0353fff700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x25d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0359503700 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f034c047c60 tx=0x7f034c015000).ready entity=mon.1 client_cookie=64774c8f96ab3f47 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0335ffb700 1 -- 172.21.15.181:0/3464799413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c046740 con 0x259b470 2020-11-16T22:36:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0335ffb700 1 -- 172.21.15.181:0/3464799413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0449e0 con 0x259b470 2020-11-16T22:36:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f0335ffb700 1 -- 172.21.15.181:0/3464799413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0479d0 con 0x259b470 2020-11-16T22:36:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 msgr2=0x25f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f034c047c60 tx=0x7f034c015000).stop 2020-11-16T22:36:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- 172.21.15.181:0/3464799413 shutdown_connections 2020-11-16T22:36:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25f5c90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x2599cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 --2- 172.21.15.181:0/3464799413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x25d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- 172.21.15.181:0/3464799413 >> 172.21.15.181:0/3464799413 conn(0x2550e40 msgr2=0x25f5580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- 172.21.15.181:0/3464799413 shutdown_connections 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.412+0000 7f03699f7740 1 -- 172.21.15.181:0/3464799413 wait complete. 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- start start 2020-11-16T22:36:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 Processor -- start 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23827b0 0x25fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 0x25c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cd6e0 con 0x259b470 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cd820 con 0x23827b0 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24cea90 con 0x2314bb0 2020-11-16T22:36:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 10 client.0 ll_register_callbacks cb 0x24c3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0353fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23827b0 0x25fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44850/0 (socket says 172.21.15.181:44850) 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 -- 172.21.15.181:0/1527594979 learned_addr learned my addr 172.21.15.181:0/1527594979 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0359503700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 0x25c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 -- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23827b0 msgr2=0x25fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23827b0 0x25fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 -- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 msgr2=0x25c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 0x25c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 -- 172.21.15.181:0/1527594979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x24be280 con 0x2314bb0 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0359503700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 0x25c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0358d02700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0348009a40 tx=0x7f034801d000).ready entity=mon.2 client_cookie=31d97110adef84 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034801d8b0 con 0x2314bb0 2020-11-16T22:36:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0348022040 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2595e10 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034801dc20 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 10 client.7604 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x22f3fd0 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2595e10 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.416+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f0348028410 con 0x2314bb0 2020-11-16T22:36:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 10 client.7604 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 10 client.7604 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 20 client.7604 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x2602720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 4 client.7604 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x23bec50 con 0x2600490 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0353fff700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x2602720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0322ffd700 10 client.7604 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0353fff700 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x2602720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=5f0466bb37d90cc3 server_cookie=4c5dea8e222dcbfa in_seq=0 out_seq=0 2020-11-16T22:36:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0322ffd700 1 -- 172.21.15.181:0/1527594979 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x23bec50 con 0x2600490 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0322ffd700 10 client.7604 handle_command_reply: tid=0 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 client.7604 shutdown 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 msgr2=0x25a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0348009a40 tx=0x7f034801d000).stop 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 shutdown_connections 2020-11-16T22:36:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x2602720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259b470 0x25c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2314bb0 0x25a0b30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f0322ffd700 0 client.7604 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 --2- 172.21.15.181:0/1527594979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23827b0 0x25fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 >> 172.21.15.181:0/1527594979 conn(0x2550e40 msgr2=0x25daf50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 shutdown_connections 2020-11-16T22:36:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 1 -- 172.21.15.181:0/1527594979 wait complete. 2020-11-16T22:36:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.420+0000 7f03699f7740 20 client.7604 trim_cache size 0 max 16384 2020-11-16T22:36:08.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 Processor -- start 2020-11-16T22:36:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 -- start start 2020-11-16T22:36:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2599e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x259a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24a5e40 con 0x2314bb0 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2649bf0 con 0x2550e40 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2598de0 con 0x23827b0 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37016/0 (socket says 172.21.15.181:37016) 2020-11-16T22:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 -- 172.21.15.181:0/1626027563 learned_addr learned my addr 172.21.15.181:0/1626027563 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0359503700 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x259a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0353fff700 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2599e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 -- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 msgr2=0x259a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x259a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 -- 172.21.15.181:0/1626027563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 msgr2=0x2599e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2599e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 -- 172.21.15.181:0/1626027563 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280081d0 con 0x2550e40 2020-11-16T22:36:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f0358d02700 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f034800a010 tx=0x7f0348028000).ready entity=mon.1 client_cookie=1cd071e77e8ac2a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03367fc700 1 -- 172.21.15.181:0/1626027563 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034803d030 con 0x2550e40 2020-11-16T22:36:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03367fc700 1 -- 172.21.15.181:0/1626027563 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348028990 con 0x2550e40 2020-11-16T22:36:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.440+0000 7f03367fc700 1 -- 172.21.15.181:0/1626027563 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348028bc0 con 0x2550e40 2020-11-16T22:36:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 msgr2=0x25a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f034800a010 tx=0x7f0348028000).stop 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- 172.21.15.181:0/1626027563 shutdown_connections 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x259a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2550e40 0x25a2140 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- 172.21.15.181:0/1626027563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2599e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- 172.21.15.181:0/1626027563 >> 172.21.15.181:0/1626027563 conn(0x7f0328002320 msgr2=0x25a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- 172.21.15.181:0/1626027563 shutdown_connections 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- 172.21.15.181:0/1626027563 wait complete. 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- start start 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 Processor -- start 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2645ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x25c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x262d550 con 0x2314bb0 2020-11-16T22:36:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2603310 con 0x259b470 2020-11-16T22:36:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2649a20 con 0x23827b0 2020-11-16T22:36:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f03699f7740 10 client.0 ll_register_callbacks cb 0x2600fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0353fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x25c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0359503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0353fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x25c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44866/0 (socket says 172.21.15.181:44866) 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0353fff700 1 -- 172.21.15.181:0/702143718 learned_addr learned my addr 172.21.15.181:0/702143718 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0358d02700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2645ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0359503700 1 -- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 msgr2=0x25c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0359503700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x25c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0359503700 1 -- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 msgr2=0x2645ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.444+0000 7f0359503700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2645ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0359503700 1 -- 172.21.15.181:0/702143718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2332720 con 0x259b470 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0359503700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25c3e70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x25da010 tx=0x7f034c027000).ready entity=mon.1 client_cookie=f11cdf34f8946dc7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0443a0 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f034c047230 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034c0449e0 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034c047600 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 10 client.8398 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x233de00 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0449e0 con 0x259b470 2020-11-16T22:36:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f034c046930 con 0x259b470 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 10 client.8398 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 10 client.8398 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 20 client.8398 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x25c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 4 client.8398 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x23c5f50 con 0x2600490 2020-11-16T22:36:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0358d02700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x25c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0358d02700 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x25c4360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=9597b12b5d84de10 server_cookie=3bd8995fa3193670 in_seq=0 out_seq=0 2020-11-16T22:36:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.448+0000 7f0322ffd700 10 client.8398 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f0322ffd700 1 -- 172.21.15.181:0/702143718 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x23c5f50 con 0x2600490 2020-11-16T22:36:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f0322ffd700 10 client.8398 handle_command_reply: tid=0 2020-11-16T22:36:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 client.8398 shutdown 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 msgr2=0x25c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25c3e70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x25da010 tx=0x7f034c027000).stop 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 shutdown_connections 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2600490 0x25c4360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x259b470 0x25c3e70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2314bb0 0x2645ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f0322ffd700 0 client.8398 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 --2- 172.21.15.181:0/702143718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23827b0 0x25c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.468+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 >> 172.21.15.181:0/702143718 conn(0x7f0328002320 msgr2=0x25f1210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 shutdown_connections 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f03699f7740 1 -- 172.21.15.181:0/702143718 wait complete. 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f03699f7740 20 client.8398 trim_cache size 0 max 16384 2020-11-16T22:36:08.461 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:08.462 INFO:teuthology.orchestra.run.smithi181.stdout: "message": "", 2020-11-16T22:36:08.462 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:08.462 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 msgr2=0x7f032c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 0x7f032c057520 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f0348006dd0 tx=0x7f0348015000).stop 2020-11-16T22:36:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 msgr2=0x7f03541596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 0x7f03541596c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f034c00ed20 tx=0x7f034c0091d0).stop 2020-11-16T22:36:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 shutdown_connections 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f032c055060 0x7f032c057520 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f035415f5b0 0x7f03541596c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03540615f0 0x7f03540ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 --2- 172.21.15.181:0/648969312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03540d1830 0x7f035416f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 >> 172.21.15.181:0/648969312 conn(0x7f03540d69a0 msgr2=0x7f035415bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 shutdown_connections 2020-11-16T22:36:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:08.472+0000 7f035af66700 1 -- 172.21.15.181:0/648969312 wait complete. 2020-11-16T22:36:08.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:36:08.478 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:36:08.535 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/ 2020-11-16T22:36:08.535 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4 2020-11-16T22:36:08.535 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5 2020-11-16T22:36:08.535 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:36:08.536 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:36:08.536 INFO:teuthology.orchestra.run.smithi106.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:36:08.540 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:08.541 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:08.541 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:08.541 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:08.542 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:36:08.542 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:08.653 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627778 2020-11-16T22:36:08.844 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:08.844 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:08.845 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:08.845 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:08.845 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:36:08.846 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:08.963 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627779 2020-11-16T22:36:09.152 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:09.153 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:09.153 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:09.153 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:09.154 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:36:09.154 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:09.268 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627780 2020-11-16T22:36:09.458 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:09.459 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:09.459 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:09.460 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:09.460 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:36:09.460 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:09.612 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627781 2020-11-16T22:36:09.802 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:09.803 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:09.803 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:09.803 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:09.803 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:36:09.804 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:09.912 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627782 2020-11-16T22:36:10.103 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:36:10.103 INFO:teuthology.orchestra.run.smithi106:> import os 2020-11-16T22:36:10.104 INFO:teuthology.orchestra.run.smithi106:> import stat 2020-11-16T22:36:10.104 INFO:teuthology.orchestra.run.smithi106:> 2020-11-16T22:36:10.104 INFO:teuthology.orchestra.run.smithi106:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:36:10.105 INFO:teuthology.orchestra.run.smithi106:> ' 2020-11-16T22:36:10.216 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627783 2020-11-16T22:36:10.405 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:36:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.448+0000 7f42c10d5c40 1 Processor -- start 2020-11-16T22:36:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 -- start start 2020-11-16T22:36:10.441 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d23297a690 0x55d2329fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2328d91f0 con 0x55d23292ba90 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2328a7be0 con 0x55d23297a690 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d232982010 con 0x55d232964cd0 2020-11-16T22:36:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b26db700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b2edc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d23297a690 0x55d2329fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44880/0 (socket says 172.21.15.181:44880) 2020-11-16T22:36:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 -- 172.21.15.181:0/2976542525 learned_addr learned my addr 172.21.15.181:0/2976542525 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 -- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d23297a690 msgr2=0x55d2329fb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d23297a690 0x55d2329fb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 -- 172.21.15.181:0/2976542525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 msgr2=0x55d2329816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329816b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 -- 172.21.15.181:0/2976542525 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2328d18a0 con 0x55d232964cd0 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b26db700 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:10.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b1eda700 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f429c009c50 tx=0x7f429c00c040).ready entity=mon.2 client_cookie=15652df5f5c482b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b0ed8700 1 -- 172.21.15.181:0/2976542525 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f040 con 0x55d232964cd0 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b0ed8700 1 -- 172.21.15.181:0/2976542525 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c008be0 con 0x55d232964cd0 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.452+0000 7f42b0ed8700 1 -- 172.21.15.181:0/2976542525 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c014430 con 0x55d232964cd0 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 msgr2=0x55d23292b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f429c009c50 tx=0x7f429c00c040).stop 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- 172.21.15.181:0/2976542525 shutdown_connections 2020-11-16T22:36:10.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d23297a690 0x55d2329fb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/2976542525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d232964cd0 0x55d23292b4c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- 172.21.15.181:0/2976542525 >> 172.21.15.181:0/2976542525 conn(0x55d2329821d0 msgr2=0x55d23292beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- 172.21.15.181:0/2976542525 shutdown_connections 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- 172.21.15.181:0/2976542525 wait complete. 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 Processor -- start 2020-11-16T22:36:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- start start 2020-11-16T22:36:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 0x55d232a0bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d23297a690 0x55d232a24e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d232980e80 con 0x55d23292ba90 2020-11-16T22:36:10.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d232a14f90 con 0x55d232964cd0 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42c10d5c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d232a15230 con 0x55d23297a690 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 0x55d232a0bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 0x55d232a0bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37044/0 (socket says 172.21.15.181:37044) 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 -- 172.21.15.181:0/1256278294 learned_addr learned my addr 172.21.15.181:0/1256278294 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b2edc700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d23297a690 0x55d232a24e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b26db700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 -- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d23297a690 msgr2=0x55d232a24e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:10.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d23297a690 0x55d232a24e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 -- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 msgr2=0x55d2329fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:10.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d23292ba90 0x55d2329fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:10.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.456+0000 7f42b1eda700 1 -- 172.21.15.181:0/1256278294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2328cd280 con 0x55d232964cd0 2020-11-16T22:36:10.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42b1eda700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 0x55d232a0bec0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f429c012000 tx=0x7f429c018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:10.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 -- 172.21.15.181:0/1256278294 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f070 con 0x55d232964cd0 2020-11-16T22:36:10.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 -- 172.21.15.181:0/1256278294 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c014e50 con 0x55d232964cd0 2020-11-16T22:36:10.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 -- 172.21.15.181:0/1256278294 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c013820 con 0x55d232964cd0 2020-11-16T22:36:10.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d2328ab620 con 0x55d232964cd0 2020-11-16T22:36:10.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d232980be0 con 0x55d232964cd0 2020-11-16T22:36:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 -- 172.21.15.181:0/1256278294 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f429c013c10 con 0x55d232964cd0 2020-11-16T22:36:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4294053ce0 0x7f42940561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42b26db700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4294053ce0 0x7f42940561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42aaffd700 1 -- 172.21.15.181:0/1256278294 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f429c06e6f0 con 0x55d232964cd0 2020-11-16T22:36:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55d232a0ed40 0x55d232a23ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- osd_op(unknown.0.0:1 15.2 15:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x55d2329686c0 con 0x55d232a0ed40 2020-11-16T22:36:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.460+0000 7f42b26db700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4294053ce0 0x7f42940561a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f42a400c010 tx=0x7f42a4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.464+0000 7f42b2edc700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55d232a0ed40 0x55d232a23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.464+0000 7f42b2edc700 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55d232a0ed40 0x55d232a23ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42b2edc700 1 -- 172.21.15.181:0/1256278294 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v111'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f42ac0014f0 con 0x55d232a0ed40 2020-11-16T22:36:11.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55d232a0ed40 msgr2=0x55d232a23ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x55d232a0ed40 0x55d232a23ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4294053ce0 msgr2=0x7f42940561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4294053ce0 0x7f42940561a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f42a400c010 tx=0x7f42a4015000).stop 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 msgr2=0x55d232a0bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 --2- 172.21.15.181:0/1256278294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d232964cd0 0x55d232a0bec0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f429c012000 tx=0x7f429c018000).stop 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42b2edc700 1 -- 172.21.15.181:0/1256278294 reap_dead start 2020-11-16T22:36:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 shutdown_connections 2020-11-16T22:36:11.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 >> 172.21.15.181:0/1256278294 conn(0x55d2329821d0 msgr2=0x55d232a11630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:11.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 shutdown_connections 2020-11-16T22:36:11.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:10.468+0000 7f42c10d5c40 1 -- 172.21.15.181:0/1256278294 wait complete. 2020-11-16T22:36:11.064 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:36:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 Processor -- start 2020-11-16T22:36:11.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 -- start start 2020-11-16T22:36:11.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 0x5583090004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830908d4a0 0x55830908aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558308f561f0 con 0x55830908d4a0 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558308f24be0 con 0x558309000ec0 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25ed815c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55830908c230 con 0x558309000a60 2020-11-16T22:36:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25de61a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25dee1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 0x5583090004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25dee1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 0x5583090004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44900/0 (socket says 172.21.15.181:44900) 2020-11-16T22:36:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25de61a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37060/0 (socket says 172.21.15.181:37060) 2020-11-16T22:36:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25dee1b700 1 -- 172.21.15.181:0/3689057137 learned_addr learned my addr 172.21.15.181:0/3689057137 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.108+0000 7f25df61c700 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830908d4a0 0x55830908aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 -- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 msgr2=0x5583090004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:11.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 0x5583090004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 -- 172.21.15.181:0/3689057137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830908d4a0 msgr2=0x55830908aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:11.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830908d4a0 0x55830908aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 -- 172.21.15.181:0/3689057137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558308f4e8a0 con 0x558309000ec0 2020-11-16T22:36:11.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f25c800b550 tx=0x7f25c80017a0).ready entity=mon.1 client_cookie=1d0bba4c65b95f0f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dd618700 1 -- 172.21.15.181:0/3689057137 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800d490 con 0x558309000ec0 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dd618700 1 -- 172.21.15.181:0/3689057137 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800d5f0 con 0x558309000ec0 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dd618700 1 -- 172.21.15.181:0/3689057137 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800d9b0 con 0x558309000ec0 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 msgr2=0x55830908c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f25c800b550 tx=0x7f25c80017a0).stop 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- 172.21.15.181:0/3689057137 shutdown_connections 2020-11-16T22:36:11.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55830908d4a0 0x55830908aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558309000a60 0x5583090004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3689057137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000ec0 0x55830908c8c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- 172.21.15.181:0/3689057137 >> 172.21.15.181:0/3689057137 conn(0x5583090013e0 msgr2=0x558308ffa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- 172.21.15.181:0/3689057137 shutdown_connections 2020-11-16T22:36:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- 172.21.15.181:0/3689057137 wait complete. 2020-11-16T22:36:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 Processor -- start 2020-11-16T22:36:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- start start 2020-11-16T22:36:11.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000a60 0x558308f92c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558309000ec0 0x558309082230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830908d4a0 0x558308fe3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x558308f93180 con 0x558309000ec0 2020-11-16T22:36:11.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558309000440 con 0x558309000a60 2020-11-16T22:36:11.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25ed815c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583090006e0 con 0x55830908d4a0 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dee1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000a60 0x558308f92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25df61c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830908d4a0 0x558308fe3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dee1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000a60 0x558308f92c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37064/0 (socket says 172.21.15.181:37064) 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25dee1b700 1 -- 172.21.15.181:0/3337037666 learned_addr learned my addr 172.21.15.181:0/3337037666 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.112+0000 7f25de61a700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558309000ec0 0x558309082230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 -- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000a60 msgr2=0x558308f92c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558309000a60 0x558308f92c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 -- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558309000ec0 msgr2=0x558309082230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558309000ec0 0x558309082230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 -- 172.21.15.181:0/3337037666 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558308f4a280 con 0x55830908d4a0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25df61c700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830908d4a0 0x558308fe3980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x558308fe73d0 tx=0x7f25d8005dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 -- 172.21.15.181:0/3337037666 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800eaa0 con 0x55830908d4a0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 -- 172.21.15.181:0/3337037666 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800ec00 con 0x55830908d4a0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558308f28620 con 0x55830908d4a0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 -- 172.21.15.181:0/3337037666 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8013440 con 0x55830908d4a0 2020-11-16T22:36:11.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55830909e5b0 con 0x55830908d4a0 2020-11-16T22:36:11.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 -- 172.21.15.181:0/3337037666 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f25d8013870 con 0x55830908d4a0 2020-11-16T22:36:11.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25c0053d10 0x7f25c00561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25dee1b700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25c0053d10 0x7f25c00561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25d77fe700 1 -- 172.21.15.181:0/3337037666 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25d806d570 con 0x55830908d4a0 2020-11-16T22:36:11.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x558309082720 0x55830908f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- osd_op(unknown.0.0:1 15.2 15:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x558309090fd0 con 0x558309082720 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.116+0000 7f25dee1b700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25c0053d10 0x7f25c00561d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f25d000a940 tx=0x7f25d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.120+0000 7f25de61a700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x558309082720 0x55830908f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.120+0000 7f25de61a700 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x558309082720 0x55830908f790 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25de61a700 1 -- 172.21.15.181:0/3337037666 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v111'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25c800dd60 con 0x558309082720 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x558309082720 msgr2=0x55830908f790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x558309082720 0x55830908f790 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25c0053d10 msgr2=0x7f25c00561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25c0053d10 0x7f25c00561d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f25d000a940 tx=0x7f25d0015000).stop 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830908d4a0 msgr2=0x558308fe3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 --2- 172.21.15.181:0/3337037666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830908d4a0 0x558308fe3980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x558308fe73d0 tx=0x7f25d8005dc0).stop 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25df61c700 1 -- 172.21.15.181:0/3337037666 reap_dead start 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 shutdown_connections 2020-11-16T22:36:11.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 >> 172.21.15.181:0/3337037666 conn(0x5583090013e0 msgr2=0x558309089ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 shutdown_connections 2020-11-16T22:36:11.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.124+0000 7f25ed815c40 1 -- 172.21.15.181:0/3337037666 wait complete. 2020-11-16T22:36:11.925 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:36:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 Processor -- start 2020-11-16T22:36:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 -- start start 2020-11-16T22:36:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67d6da0 0x559af67c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af66a6190 con 0x559af67d6da0 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af66a71f0 con 0x559af67e0450 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa370b81c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af6675be0 con 0x559af6735760 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa362988700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa362988700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37084/0 (socket says 172.21.15.181:37084) 2020-11-16T22:36:11.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa362988700 1 -- 172.21.15.181:0/3910632686 learned_addr learned my addr 172.21.15.181:0/3910632686 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:11.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa361986700 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67d6da0 0x559af67c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa362988700 1 -- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 msgr2=0x559af67493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.972+0000 7fa362988700 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:36:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa362988700 1 -- 172.21.15.181:0/3910632686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67d6da0 msgr2=0x559af67c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa362988700 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67d6da0 0x559af67c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa362988700 1 -- 172.21.15.181:0/3910632686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af669b280 con 0x559af67e0450 2020-11-16T22:36:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa362988700 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fa35c005d40 tx=0x7fa35c015000).ready entity=mon.1 client_cookie=3c54dd24ddb29ce1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa360984700 1 -- 172.21.15.181:0/3910632686 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c008340 con 0x559af67e0450 2020-11-16T22:36:11.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa360984700 1 -- 172.21.15.181:0/3910632686 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c005830 con 0x559af67e0450 2020-11-16T22:36:11.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.976+0000 7fa360984700 1 -- 172.21.15.181:0/3910632686 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c01b860 con 0x559af67e0450 2020-11-16T22:36:11.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 msgr2=0x559af67df1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fa35c005d40 tx=0x7fa35c015000).stop 2020-11-16T22:36:11.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- 172.21.15.181:0/3910632686 shutdown_connections 2020-11-16T22:36:11.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67e0450 0x559af67df1a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:36:11.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- 172.21.15.181:0/3910632686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67d6da0 0x559af67c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- 172.21.15.181:0/3910632686 >> 172.21.15.181:0/3910632686 conn(0x559af67501d0 msgr2=0x559af6737230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:11.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- 172.21.15.181:0/3910632686 shutdown_connections 2020-11-16T22:36:11.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- 172.21.15.181:0/3910632686 wait complete. 2020-11-16T22:36:11.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 Processor -- start 2020-11-16T22:36:11.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- start start 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 0x559af67ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67e0450 0x559af67c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af67de1d0 con 0x559af67e0450 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af67edb80 con 0x559af67d6da0 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa370b81c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af67f0840 con 0x559af6735760 2020-11-16T22:36:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa361986700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 0x559af67ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa361986700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 0x559af67ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37086/0 (socket says 172.21.15.181:37086) 2020-11-16T22:36:11.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa361986700 1 -- 172.21.15.181:0/815613575 learned_addr learned my addr 172.21.15.181:0/815613575 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:11.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa362187700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa362988700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67e0450 0x559af67c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa361986700 1 -- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 msgr2=0x559af67f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.980+0000 7fa361986700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa361986700 1 -- 172.21.15.181:0/815613575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67e0450 msgr2=0x559af67c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa361986700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af67e0450 0x559af67c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa361986700 1 -- 172.21.15.181:0/815613575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af6679620 con 0x559af67d6da0 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa362187700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x559af6735760 0x559af67f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa361986700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 0x559af67ed690 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fa34c00c840 tx=0x7fa34c0017f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 -- 172.21.15.181:0/815613575 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c010640 con 0x559af67d6da0 2020-11-16T22:36:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 -- 172.21.15.181:0/815613575 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c0107a0 con 0x559af67d6da0 2020-11-16T22:36:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559af67f0ae0 con 0x559af67d6da0 2020-11-16T22:36:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 -- 172.21.15.181:0/815613575 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c010b20 con 0x559af67d6da0 2020-11-16T22:36:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559af67de400 con 0x559af67d6da0 2020-11-16T22:36:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 -- 172.21.15.181:0/815613575 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa34c011410 con 0x559af67d6da0 2020-11-16T22:36:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa344053ce0 0x7fa3440561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa362187700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa344053ce0 0x7fa3440561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa35affd700 1 -- 172.21.15.181:0/815613575 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa34c00b030 con 0x559af67d6da0 2020-11-16T22:36:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa370b81c40 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x559af66fbfe0 0x559af6750ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 15.5 15:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x559af67de8a0 con 0x559af66fbfe0 2020-11-16T22:36:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa362187700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa344053ce0 0x7fa3440561a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa354004130 tx=0x7fa354000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa362988700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x559af66fbfe0 0x559af6750ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.984+0000 7fa362988700 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x559af66fbfe0 0x559af6750ac0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa362988700 1 -- 172.21.15.181:0/815613575 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v111'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa35c01b650 con 0x559af66fbfe0 2020-11-16T22:36:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x559af66fbfe0 msgr2=0x559af6750ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x559af66fbfe0 0x559af6750ac0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa344053ce0 msgr2=0x7fa3440561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa344053ce0 0x7fa3440561a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa354004130 tx=0x7fa354000b80).stop 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 msgr2=0x559af67ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 --2- 172.21.15.181:0/815613575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559af67d6da0 0x559af67ed690 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fa34c00c840 tx=0x7fa34c0017f0).stop 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa362988700 1 -- 172.21.15.181:0/815613575 reap_dead start 2020-11-16T22:36:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 shutdown_connections 2020-11-16T22:36:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 >> 172.21.15.181:0/815613575 conn(0x559af67501d0 msgr2=0x559af6738b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 shutdown_connections 2020-11-16T22:36:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:11.988+0000 7fa370b81c40 1 -- 172.21.15.181:0/815613575 wait complete. 2020-11-16T22:36:11.981 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:36:12.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.064+0000 7ffa70356c40 1 Processor -- start 2020-11-16T22:36:12.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.064+0000 7ffa70356c40 1 -- start start 2020-11-16T22:36:12.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.064+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.064+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab4a0 0x556ca191ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca19ab880 0x556ca191dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca1873190 con 0x556ca19ab880 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca18741f0 con 0x556ca1918fc0 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ca1842be0 con 0x556ca19ab4a0 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37102/0 (socket says 172.21.15.181:37102) 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 -- 172.21.15.181:0/3916298743 learned_addr learned my addr 172.21.15.181:0/3916298743 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6215d700 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca19ab880 0x556ca191dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6115b700 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab4a0 0x556ca191ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 -- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab4a0 msgr2=0x556ca191ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab4a0 0x556ca191ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 -- 172.21.15.181:0/3916298743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca19ab880 msgr2=0x556ca191dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca19ab880 0x556ca191dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 -- 172.21.15.181:0/3916298743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca1868280 con 0x556ca1918fc0 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7ffa5400b580 tx=0x7ffa54001820).ready entity=mon.1 client_cookie=e5ca900b944743ab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa5bfff700 1 -- 172.21.15.181:0/3916298743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400d550 con 0x556ca1918fc0 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa5bfff700 1 -- 172.21.15.181:0/3916298743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5400d6b0 con 0x556ca1918fc0 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa5bfff700 1 -- 172.21.15.181:0/3916298743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400da30 con 0x556ca1918fc0 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 msgr2=0x556ca18cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7ffa5400b580 tx=0x7ffa54001820).stop 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- 172.21.15.181:0/3916298743 shutdown_connections 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca19ab880 0x556ca191dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca1918fc0 0x556ca18cdf10 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3916298743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab4a0 0x556ca191ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- 172.21.15.181:0/3916298743 >> 172.21.15.181:0/3916298743 conn(0x556ca191f3e0 msgr2=0x556ca1901980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- 172.21.15.181:0/3916298743 shutdown_connections 2020-11-16T22:36:12.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- 172.21.15.181:0/3916298743 wait complete. 2020-11-16T22:36:12.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 Processor -- start 2020-11-16T22:36:12.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.068+0000 7ffa70356c40 1 -- start start 2020-11-16T22:36:12.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca1918fc0 0x556ca19ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca19ab4a0 0x556ca1921e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 0x556ca1991920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca18c6780 con 0x556ca1918fc0 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca19c03f0 con 0x556ca19ab4a0 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ca19c0690 con 0x556ca19ab880 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 0x556ca1991920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 0x556ca1991920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44952/0 (socket says 172.21.15.181:44952) 2020-11-16T22:36:12.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 -- 172.21.15.181:0/3112250917 learned_addr learned my addr 172.21.15.181:0/3112250917 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6115b700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca19ab4a0 0x556ca1921e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca1918fc0 0x556ca19ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 -- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca19ab4a0 msgr2=0x556ca1921e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ca19ab4a0 0x556ca1921e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 -- 172.21.15.181:0/3112250917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca1918fc0 msgr2=0x556ca19ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ca1918fc0 0x556ca19ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 -- 172.21.15.181:0/3112250917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca1846620 con 0x556ca19ab880 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa6215d700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 0x556ca1991920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x556ca18b2bb0 tx=0x7ffa5c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa5a7fc700 1 -- 172.21.15.181:0/3112250917 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c005910 con 0x556ca19ab880 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa5a7fc700 1 -- 172.21.15.181:0/3112250917 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c008e40 con 0x556ca19ab880 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556ca19c0930 con 0x556ca19ab880 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa5a7fc700 1 -- 172.21.15.181:0/3112250917 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c01d850 con 0x556ca19ab880 2020-11-16T22:36:12.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.072+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ca19c0e70 con 0x556ca19ab880 2020-11-16T22:36:12.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa5a7fc700 1 -- 172.21.15.181:0/3112250917 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ffa5c00e070 con 0x556ca19ab880 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa5a7fc700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffa44053ce0 0x7ffa440561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa5a7fc700 1 -- 172.21.15.181:0/3112250917 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa5c06f1b0 con 0x556ca19ab880 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffa44053ce0 0x7ffa440561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x556ca1922320 0x556ca18c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- osd_op(unknown.0.0:1 15.1 15:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x556ca1901980 con 0x556ca1922320 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa6115b700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x556ca1922320 0x556ca18c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa6195c700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffa44053ce0 0x7ffa440561a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ffa54000fb0 tx=0x7ffa54001820).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.076+0000 7ffa6115b700 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x556ca1922320 0x556ca18c6170 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa6115b700 1 -- 172.21.15.181:0/3112250917 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v111'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ffa4c00f490 con 0x556ca1922320 2020-11-16T22:36:12.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x556ca1922320 msgr2=0x556ca18c6170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x556ca1922320 0x556ca18c6170 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffa44053ce0 msgr2=0x7ffa440561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ffa44053ce0 0x7ffa440561a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7ffa54000fb0 tx=0x7ffa54001820).stop 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 msgr2=0x556ca1991920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 --2- 172.21.15.181:0/3112250917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556ca19ab880 0x556ca1991920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x556ca18b2bb0 tx=0x7ffa5c014000).stop 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa6215d700 1 -- 172.21.15.181:0/3112250917 reap_dead start 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 shutdown_connections 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 >> 172.21.15.181:0/3112250917 conn(0x556ca191f3e0 msgr2=0x556ca19185e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 shutdown_connections 2020-11-16T22:36:12.271 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.080+0000 7ffa70356c40 1 -- 172.21.15.181:0/3112250917 wait complete. 2020-11-16T22:36:12.273 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:36:12.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 Processor -- start 2020-11-16T22:36:12.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 -- start start 2020-11-16T22:36:12.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c110690 0x55ce0c11f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c11a450 0x55ce0c11fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0bfe11f0 con 0x55ce0c11a450 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0bfafbe0 con 0x55ce0c06f760 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce0c10f830 con 0x55ce0c110690 2020-11-16T22:36:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37124/0 (socket says 172.21.15.181:37124) 2020-11-16T22:36:12.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 -- 172.21.15.181:0/2620146824 learned_addr learned my addr 172.21.15.181:0/2620146824 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0d8fc700 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c110690 0x55ce0c11f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e8fe700 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c11a450 0x55ce0c11fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 -- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c110690 msgr2=0x55ce0c11f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c110690 0x55ce0c11f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 -- 172.21.15.181:0/2620146824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c11a450 msgr2=0x55ce0c11fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c11a450 0x55ce0c11fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.360+0000 7f8d0e0fd700 1 -- 172.21.15.181:0/2620146824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce0bfd98a0 con 0x55ce0c06f760 2020-11-16T22:36:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d0e0fd700 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f8cf8001a10 tx=0x7f8cf8006ab0).ready entity=mon.1 client_cookie=cfe4e20ab7d81c45 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d0c8fa700 1 -- 172.21.15.181:0/2620146824 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8006320 con 0x55ce0c06f760 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d0c8fa700 1 -- 172.21.15.181:0/2620146824 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8004500 con 0x55ce0c06f760 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d0c8fa700 1 -- 172.21.15.181:0/2620146824 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8004880 con 0x55ce0c06f760 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 msgr2=0x55ce0c113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f8cf8001a10 tx=0x7f8cf8006ab0).stop 2020-11-16T22:36:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/2620146824 shutdown_connections 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c11a450 0x55ce0c11fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c06f760 0x55ce0c113df0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/2620146824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c110690 0x55ce0c11f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/2620146824 >> 172.21.15.181:0/2620146824 conn(0x55ce0c08a1d0 msgr2=0x55ce0c08a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/2620146824 shutdown_connections 2020-11-16T22:36:12.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/2620146824 wait complete. 2020-11-16T22:36:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 Processor -- start 2020-11-16T22:36:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.364+0000 7f8d1caf7c40 1 -- start start 2020-11-16T22:36:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c06f760 0x55ce0c127560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c110690 0x55ce0c035af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 0x55ce0c0f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0c127a50 con 0x55ce0c110690 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0c1131b0 con 0x55ce0c11a450 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce0c0362d0 con 0x55ce0c06f760 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 0x55ce0c0f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e0fd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c06f760 0x55ce0c127560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 0x55ce0c0f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37134/0 (socket says 172.21.15.181:37134) 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e0fd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c06f760 0x55ce0c127560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44974/0 (socket says 172.21.15.181:44974) 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 -- 172.21.15.181:0/3367981314 learned_addr learned my addr 172.21.15.181:0/3367981314 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0d8fc700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c110690 0x55ce0c035af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 -- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c06f760 msgr2=0x55ce0c127560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ce0c06f760 0x55ce0c127560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.360 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 -- 172.21.15.181:0/3367981314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c110690 msgr2=0x55ce0c035af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0c110690 0x55ce0c035af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 -- 172.21.15.181:0/3367981314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce0bfd5280 con 0x55ce0c11a450 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d0e8fe700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 0x55ce0c0f85d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x55ce0c022a00 tx=0x7f8d08013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d06ffd700 1 -- 172.21.15.181:0/3367981314 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08023070 con 0x55ce0c11a450 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d06ffd700 1 -- 172.21.15.181:0/3367981314 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d080054b0 con 0x55ce0c11a450 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d06ffd700 1 -- 172.21.15.181:0/3367981314 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0801e7f0 con 0x55ce0c11a450 2020-11-16T22:36:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce0bfb3620 con 0x55ce0c11a450 2020-11-16T22:36:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.368+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce0c036810 con 0x55ce0c11a450 2020-11-16T22:36:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d06ffd700 1 -- 172.21.15.181:0/3367981314 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8d0801eba0 con 0x55ce0c11a450 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d06ffd700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8cf0053ce0 0x7f8cf00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d06ffd700 1 -- 172.21.15.181:0/3367981314 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8d0806f4e0 con 0x55ce0c11a450 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d0e0fd700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8cf0053ce0 0x7f8cf00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55ce0c10f1b0 0x55ce0c08aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 15.5 15:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x55ce0c073320 con 0x55ce0c10f1b0 2020-11-16T22:36:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d0d8fc700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55ce0c10f1b0 0x55ce0c08aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d0e0fd700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8cf0053ce0 0x7f8cf00561a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8cf80044d0 tx=0x7f8cf8014000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.372+0000 7f8d0d8fc700 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55ce0c10f1b0 0x55ce0c08aac0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d0d8fc700 1 -- 172.21.15.181:0/3367981314 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v111'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8d00012040 con 0x55ce0c10f1b0 2020-11-16T22:36:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55ce0c10f1b0 msgr2=0x55ce0c08aac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55ce0c10f1b0 0x55ce0c08aac0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8cf0053ce0 msgr2=0x7f8cf00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f8cf0053ce0 0x7f8cf00561a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8cf80044d0 tx=0x7f8cf8014000).stop 2020-11-16T22:36:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 msgr2=0x55ce0c0f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 --2- 172.21.15.181:0/3367981314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ce0c11a450 0x55ce0c0f85d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x55ce0c022a00 tx=0x7f8d08013000).stop 2020-11-16T22:36:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d0e8fe700 1 -- 172.21.15.181:0/3367981314 reap_dead start 2020-11-16T22:36:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 shutdown_connections 2020-11-16T22:36:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 >> 172.21.15.181:0/3367981314 conn(0x55ce0c08a1d0 msgr2=0x55ce0c1114b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 shutdown_connections 2020-11-16T22:36:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.376+0000 7f8d1caf7c40 1 -- 172.21.15.181:0/3367981314 wait complete. 2020-11-16T22:36:12.368 INFO:teuthology.orchestra.run.smithi181:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:36:12.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 Processor -- start 2020-11-16T22:36:12.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 -- start start 2020-11-16T22:36:12.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a358ba60 0x55d0a3535300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 0x55d0a358ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a34e11f0 con 0x55d0a36184a0 2020-11-16T22:36:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a34afbe0 con 0x55d0a358ba60 2020-11-16T22:36:12.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a35861c0 con 0x55d0a358bec0 2020-11-16T22:36:12.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44988/0 (socket says 172.21.15.181:44988) 2020-11-16T22:36:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 -- 172.21.15.181:0/2151073831 learned_addr learned my addr 172.21.15.181:0/2151073831 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1b094700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a358ba60 0x55d0a3535300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1b895700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 0x55d0a358ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 -- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a358ba60 msgr2=0x55d0a3535300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a358ba60 0x55d0a3535300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 -- 172.21.15.181:0/2151073831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 msgr2=0x55d0a358ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 0x55d0a358ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 -- 172.21.15.181:0/2151073831 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a34d98a0 con 0x55d0a358bec0 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1b895700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 0x55d0a358ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.456+0000 7f1f1a893700 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f1f0c00b610 tx=0x7f1f0c001740).ready entity=mon.2 client_cookie=855560e13b437859 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f19891700 1 -- 172.21.15.181:0/2151073831 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00be70 con 0x55d0a358bec0 2020-11-16T22:36:12.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f19891700 1 -- 172.21.15.181:0/2151073831 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c00d5a0 con 0x55d0a358bec0 2020-11-16T22:36:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f19891700 1 -- 172.21.15.181:0/2151073831 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00d920 con 0x55d0a358bec0 2020-11-16T22:36:12.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 msgr2=0x55d0a358c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f1f0c00b610 tx=0x7f1f0c001740).stop 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/2151073831 shutdown_connections 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a36184a0 0x55d0a358ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a358ba60 0x55d0a3535300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/2151073831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a358c980 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/2151073831 >> 172.21.15.181:0/2151073831 conn(0x55d0a358c3e0 msgr2=0x55d0a361bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/2151073831 shutdown_connections 2020-11-16T22:36:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/2151073831 wait complete. 2020-11-16T22:36:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 Processor -- start 2020-11-16T22:36:12.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.460+0000 7f1f29a8ec40 1 -- start start 2020-11-16T22:36:12.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a358ba60 0x55d0a3614220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a3614710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a36184a0 0x55d0a36112a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a358a9c0 con 0x55d0a358ba60 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a36164a0 con 0x55d0a36184a0 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a3612e10 con 0x55d0a358bec0 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a3614710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1b895700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a36184a0 0x55d0a36112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a3614710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:44992/0 (socket says 172.21.15.181:44992) 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 -- 172.21.15.181:0/1726533093 learned_addr learned my addr 172.21.15.181:0/1726533093 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1b094700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a358ba60 0x55d0a3614220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 -- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a36184a0 msgr2=0x55d0a36112a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d0a36184a0 0x55d0a36112a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 -- 172.21.15.181:0/1726533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a358ba60 msgr2=0x55d0a3614220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0a358ba60 0x55d0a3614220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 -- 172.21.15.181:0/1726533093 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a34d5280 con 0x55d0a358bec0 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f1a893700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a3614710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f1f0c00b5e0 tx=0x7f1f0c001740).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f13fff700 1 -- 172.21.15.181:0/1726533093 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00eb70 con 0x55d0a358bec0 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f13fff700 1 -- 172.21.15.181:0/1726533093 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c00ecd0 con 0x55d0a358bec0 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f13fff700 1 -- 172.21.15.181:0/1726533093 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01b650 con 0x55d0a358bec0 2020-11-16T22:36:12.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d0a34b3620 con 0x55d0a358bec0 2020-11-16T22:36:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0a358abf0 con 0x55d0a358bec0 2020-11-16T22:36:12.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.464+0000 7f1f13fff700 1 -- 172.21.15.181:0/1726533093 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1f0c01ba00 con 0x55d0a358bec0 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f13fff700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1efc053ce0 0x7f1efc0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f13fff700 1 -- 172.21.15.181:0/1726533093 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1f0c06f370 con 0x55d0a358bec0 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f1b094700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1efc053ce0 0x7f1efc0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55d0a3574450 0x55d0a362c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- osd_op(unknown.0.0:1 15.5 15:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x55d0a3585c40 con 0x55d0a3574450 2020-11-16T22:36:12.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f1b895700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55d0a3574450 0x55d0a362c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f1b094700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1efc053ce0 0x7f1efc0561a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f1f04005fd0 tx=0x7f1f04007040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.468+0000 7f1f1b895700 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55d0a3574450 0x55d0a362c410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f1b895700 1 -- 172.21.15.181:0/1726533093 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v111'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1f1400a540 con 0x55d0a3574450 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55d0a3574450 msgr2=0x55d0a362c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x55d0a3574450 0x55d0a362c410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1efc053ce0 msgr2=0x7f1efc0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1efc053ce0 0x7f1efc0561a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f1f04005fd0 tx=0x7f1f04007040).stop 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 msgr2=0x55d0a3614710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 --2- 172.21.15.181:0/1726533093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d0a358bec0 0x55d0a3614710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f1f0c00b5e0 tx=0x7f1f0c001740).stop 2020-11-16T22:36:12.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f1b895700 1 -- 172.21.15.181:0/1726533093 reap_dead start 2020-11-16T22:36:12.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 shutdown_connections 2020-11-16T22:36:12.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 >> 172.21.15.181:0/1726533093 conn(0x55d0a358c3e0 msgr2=0x55d0a358c980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 shutdown_connections 2020-11-16T22:36:12.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.472+0000 7f1f29a8ec40 1 -- 172.21.15.181:0/1726533093 wait complete. 2020-11-16T22:36:12.463 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.644+0000 7f083e494700 1 Processor -- start 2020-11-16T22:36:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.644+0000 7f083e494700 1 -- start start 2020-11-16T22:36:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0838140550 0x7f0838140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380cf390 con 0x7f08380d07e0 2020-11-16T22:36:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083815ebb0 con 0x7f0838140550 2020-11-16T22:36:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380cf5d0 con 0x7f08380cef10 2020-11-16T22:36:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45002/0 (socket says 172.21.15.181:45002) 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 -- 172.21.15.181:0/1499697004 learned_addr learned my addr 172.21.15.181:0/1499697004 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083d492700 1 --2- 172.21.15.181:0/1499697004 >> v2:172.21.15.175:3300/0 conn(0x7f0838140550 0x7f0838140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 -- 172.21.15.181:0/1499697004 >> v2:172.21.15.175:3300/0 conn(0x7f0838140550 msgr2=0x7f0838140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 --2- 172.21.15.181:0/1499697004 >> v2:172.21.15.175:3300/0 conn(0x7f0838140550 0x7f0838140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 -- 172.21.15.181:0/1499697004 >> v1:172.21.15.175:6789/0 conn(0x7f08380d07e0 legacy=0x7f0838063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 -- 172.21.15.181:0/1499697004 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380cf870 con 0x7f08380cef10 2020-11-16T22:36:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083cc91700 1 --2- 172.21.15.181:0/1499697004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f0828006fd0 tx=0x7f0828006eb0).ready entity=mon.2 client_cookie=720f7c2c6506679b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f082ffff700 1 -- 172.21.15.181:0/1499697004 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828009890 con 0x7f08380cef10 2020-11-16T22:36:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f082ffff700 1 -- 172.21.15.181:0/1499697004 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08280099f0 con 0x7f08380cef10 2020-11-16T22:36:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f082ffff700 1 -- 172.21.15.181:0/1499697004 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828009d70 con 0x7f08380cef10 2020-11-16T22:36:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- 172.21.15.181:0/1499697004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 msgr2=0x7f08380d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 --2- 172.21.15.181:0/1499697004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f0828006fd0 tx=0x7f0828006eb0).stop 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- 172.21.15.181:0/1499697004 shutdown_connections 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 --2- 172.21.15.181:0/1499697004 >> v2:172.21.15.175:3300/0 conn(0x7f0838140550 0x7f0838140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 --2- 172.21.15.181:0/1499697004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08380cef10 0x7f08380d02f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- 172.21.15.181:0/1499697004 >> 172.21.15.181:0/1499697004 conn(0x7f08380d4570 msgr2=0x7f08380d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- 172.21.15.181:0/1499697004 shutdown_connections 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 -- 172.21.15.181:0/1499697004 wait complete. 2020-11-16T22:36:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.648+0000 7f083e494700 1 Processor -- start 2020-11-16T22:36:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- start start 2020-11-16T22:36:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08380d07e0 0x7f083815f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 0x7f08380d1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083816d350 con 0x7f08380d07e0 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083816d5f0 con 0x7f08380cef10 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083816d890 con 0x7f0838140550 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083d492700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083d492700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37166/0 (socket says 172.21.15.181:37166) 2020-11-16T22:36:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083d492700 1 -- 172.21.15.181:0/2842992462 learned_addr learned my addr 172.21.15.181:0/2842992462 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 0x7f08380d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083cc91700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08380d07e0 0x7f083815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 -- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 msgr2=0x7f083815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 -- 172.21.15.181:0/2842992462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08380d07e0 msgr2=0x7f083815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08380d07e0 0x7f083815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 -- 172.21.15.181:0/2842992462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083816db30 con 0x7f0838140550 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083d492700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083dc93700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 0x7f08380d1010 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f083000c860 tx=0x7f0830007e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083000a570 con 0x7f0838140550 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083000a6d0 con 0x7f0838140550 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083816ddd0 con 0x7f0838140550 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083000aa70 con 0x7f0838140550 2020-11-16T22:36:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.652+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083815c9e0 con 0x7f0838140550 2020-11-16T22:36:12.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083816d1b0 con 0x7f0838140550 2020-11-16T22:36:12.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0830010070 con 0x7f0838140550 2020-11-16T22:36:12.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f082d7fa700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 0x7f0810057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:12.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f083006dab0 con 0x7f0838140550 2020-11-16T22:36:12.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f083d492700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 0x7f0810057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:12.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.656+0000 7f083d492700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 0x7f0810057520 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f083400bac0 tx=0x7f0834008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:12.655 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.664+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f083816d1b0 con 0x7f0838140550 2020-11-16T22:36:12.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.864+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08380cb5b0 con 0x7f0838140550 2020-11-16T22:36:12.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f082d7fa700 1 -- 172.21.15.181:0/2842992462 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f08380cb5b0 con 0x7f0838140550 2020-11-16T22:36:12.856 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:12.856 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 msgr2=0x7f0810057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 0x7f0810057520 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f083400bac0 tx=0x7f0834008040).stop 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 msgr2=0x7f08380d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 0x7f08380d1010 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f083000c860 tx=0x7f0830007e60).stop 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 shutdown_connections 2020-11-16T22:36:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0810055060 0x7f0810057520 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0838140550 0x7f08380d1010 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08380cef10 0x7f083815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 --2- 172.21.15.181:0/2842992462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08380d07e0 0x7f083815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 >> 172.21.15.181:0/2842992462 conn(0x7f08380d4570 msgr2=0x7f08380caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.868+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 shutdown_connections 2020-11-16T22:36:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:12.872+0000 7f083e494700 1 -- 172.21.15.181:0/2842992462 wait complete. 2020-11-16T22:36:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:12.875 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub start /d1/d2/d3 recursive force 2020-11-16T22:36:13.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 Processor -- start 2020-11-16T22:36:13.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 -- start start 2020-11-16T22:36:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 0x7fea98081550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 0x7fea981423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea981428b0 0x7fea98161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea98081a40 con 0x7fea98081170 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea980d1800 con 0x7fea981428b0 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea97fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980d1aa0 con 0x7fea980d1060 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea981428b0 0x7fea98161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea967fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 0x7fea981423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea981428b0 0x7fea98161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37178/0 (socket says 172.21.15.181:37178) 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea967fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 0x7fea981423c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45016/0 (socket says 172.21.15.181:45016) 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea977fe700 1 -- 172.21.15.181:0/2783453479 learned_addr learned my addr 172.21.15.181:0/2783453479 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.012+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2783453479 >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 0x7fea98081550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 -- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 msgr2=0x7fea981423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 --2- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 0x7fea981423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 -- 172.21.15.181:0/2783453479 >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 msgr2=0x7fea98081550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 --2- 172.21.15.181:0/2783453479 >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 0x7fea98081550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 -- 172.21.15.181:0/2783453479 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980d1d40 con 0x7fea981428b0 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2783453479 >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 0x7fea98081550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea977fe700 1 --2- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea981428b0 0x7fea98161630 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7fea98062b00 tx=0x7fea90006eb0).ready entity=mon.1 client_cookie=14ea8d7712312081 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea95ffb700 1 -- 172.21.15.181:0/2783453479 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90009810 con 0x7fea981428b0 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea95ffb700 1 -- 172.21.15.181:0/2783453479 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea90009970 con 0x7fea981428b0 2020-11-16T22:36:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea95ffb700 1 -- 172.21.15.181:0/2783453479 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90009cf0 con 0x7fea981428b0 2020-11-16T22:36:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea981428b0 msgr2=0x7fea98161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 --2- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea981428b0 0x7fea98161630 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7fea98062b00 tx=0x7fea90006eb0).stop 2020-11-16T22:36:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- 172.21.15.181:0/2783453479 shutdown_connections 2020-11-16T22:36:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 --2- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea981428b0 0x7fea98161630 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 --2- 172.21.15.181:0/2783453479 >> v2:172.21.15.175:3300/0 conn(0x7fea98081170 0x7fea98081550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 --2- 172.21.15.181:0/2783453479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea980d1060 0x7fea981423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- 172.21.15.181:0/2783453479 >> 172.21.15.181:0/2783453479 conn(0x7fea980682f0 msgr2=0x7fea980d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- 172.21.15.181:0/2783453479 shutdown_connections 2020-11-16T22:36:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- 172.21.15.181:0/2783453479 wait complete. 2020-11-16T22:36:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 Processor -- start 2020-11-16T22:36:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.016+0000 7fea97fff700 1 -- start start 2020-11-16T22:36:13.010 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea980d1060 0x7fea98170990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 0x7fea98160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980cde30 con 0x7fea981428b0 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ce0d0 con 0x7fea980d1060 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ce370 con 0x7fea98081170 2020-11-16T22:36:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45022/0 (socket says 172.21.15.181:45022) 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 -- 172.21.15.181:0/25354473 learned_addr learned my addr 172.21.15.181:0/25354473 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea980d1060 0x7fea98170990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea977fe700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 0x7fea98160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 -- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea980d1060 msgr2=0x7fea98170990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea980d1060 0x7fea98170990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 -- 172.21.15.181:0/25354473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 msgr2=0x7fea98160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 0x7fea98160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 -- 172.21.15.181:0/25354473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980ce610 con 0x7fea98081170 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea977fe700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 0x7fea98160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea96ffd700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fea8800dcb0 tx=0x7fea88008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea8effd700 1 -- 172.21.15.181:0/25354473 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800b9f0 con 0x7fea98081170 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea8effd700 1 -- 172.21.15.181:0/25354473 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800bb50 con 0x7fea98081170 2020-11-16T22:36:13.013 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea980ce8b0 con 0x7fea98081170 2020-11-16T22:36:13.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea8effd700 1 -- 172.21.15.181:0/25354473 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88012420 con 0x7fea98081170 2020-11-16T22:36:13.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea98170e80 con 0x7fea98081170 2020-11-16T22:36:13.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.020+0000 7fea8effd700 1 -- 172.21.15.181:0/25354473 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fea88012790 con 0x7fea98081170 2020-11-16T22:36:13.014 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.024+0000 7fea8effd700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 0x7fea74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.024+0000 7fea8effd700 1 -- 172.21.15.181:0/25354473 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea8806e090 con 0x7fea98081170 2020-11-16T22:36:13.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.024+0000 7fea967fc700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 0x7fea74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.024+0000 7fea967fc700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 0x7fea74057520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fea980d7350 tx=0x7fea80006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.015 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 Processor -- start 2020-11-16T22:36:13.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- start start 2020-11-16T22:36:13.016 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x253c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x253c080 0x25357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c7560 con 0x253c080 2020-11-16T22:36:13.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x247b9e0 con 0x22afc60 2020-11-16T22:36:13.017 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x233b7a0 con 0x231d860 2020-11-16T22:36:13.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45030/0 (socket says 172.21.15.181:45030) 2020-11-16T22:36:13.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 -- 172.21.15.181:0/3698894802 learned_addr learned my addr 172.21.15.181:0/3698894802 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea967fc700 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x253c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea977fe700 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x253c080 0x25357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 -- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 msgr2=0x253c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x253c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 -- 172.21.15.181:0/3698894802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x253c080 msgr2=0x25357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x253c080 0x25357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 -- 172.21.15.181:0/3698894802 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2336650 con 0x231d860 2020-11-16T22:36:13.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fea88047b10 tx=0x7fea88017000).ready entity=mon.2 client_cookie=f4b1aab325b8bddf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea6ffff700 1 -- 172.21.15.181:0/3698894802 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88029070 con 0x231d860 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea6ffff700 1 -- 172.21.15.181:0/3698894802 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88044660 con 0x231d860 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7fea6ffff700 1 -- 172.21.15.181:0/3698894802 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88047650 con 0x231d860 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 msgr2=0x253e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fea88047b10 tx=0x7fea88017000).stop 2020-11-16T22:36:13.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- 172.21.15.181:0/3698894802 shutdown_connections 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x253c080 0x25357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x231d860 0x253e7d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 --2- 172.21.15.181:0/3698894802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x253c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- 172.21.15.181:0/3698894802 >> 172.21.15.181:0/3698894802 conn(0x24ebef0 msgr2=0x2538450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- 172.21.15.181:0/3698894802 shutdown_connections 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.028+0000 7feaac044740 1 -- 172.21.15.181:0/3698894802 wait complete. 2020-11-16T22:36:13.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- start start 2020-11-16T22:36:13.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 Processor -- start 2020-11-16T22:36:13.025 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x231d860 0x253b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 0x2555ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2468790 con 0x231d860 2020-11-16T22:36:13.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x24688d0 con 0x22afc60 2020-11-16T22:36:13.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2469b40 con 0x253c080 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 10 client.0 ll_register_callbacks cb 0x245eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 0x2555ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37194/0 (socket says 172.21.15.181:37194) 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 -- 172.21.15.181:0/1258212951 learned_addr learned my addr 172.21.15.181:0/1258212951 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea967fc700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x231d860 0x253b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 -- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 msgr2=0x2555ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 0x2555ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 -- 172.21.15.181:0/1258212951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x231d860 msgr2=0x253b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x231d860 0x253b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 -- 172.21.15.181:0/1258212951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2459330 con 0x22afc60 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea977fe700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 0x2555ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea96ffd700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fea88047b10 tx=0x7fea88007f90).ready entity=mon.1 client_cookie=6683f410280a6e06 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88029040 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x251c770 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea880470c0 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea88047280 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 10 client.8476 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x228f080 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x251c770 con 0x22afc60 2020-11-16T22:36:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fea88005a50 con 0x22afc60 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 10 client.8476 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 10 client.8476 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 20 client.8476 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x258c680 0x259f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 4 client.8476 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2359d00 con 0x258c680 2020-11-16T22:36:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea967fc700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x258c680 0x259f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea6d7fa700 10 client.8476 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.032+0000 7fea967fc700 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x258c680 0x259f050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=65bfc75d8c762a24 server_cookie=3f4f42195bbcec1c in_seq=0 out_seq=0 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7fea6d7fa700 1 -- 172.21.15.181:0/1258212951 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2359d00 con 0x258c680 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7fea6d7fa700 10 client.8476 handle_command_reply: tid=0 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 client.8476 shutdown 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 msgr2=0x25385e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fea88047b10 tx=0x7fea88007f90).stop 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 shutdown_connections 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x258c680 0x259f050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x253c080 0x2555ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22afc60 0x25385e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 --2- 172.21.15.181:0/1258212951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x231d860 0x253b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 >> 172.21.15.181:0/1258212951 conn(0x24ebef0 msgr2=0x256f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7fea6d7fa700 0 client.8476 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 shutdown_connections 2020-11-16T22:36:13.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 1 -- 172.21.15.181:0/1258212951 wait complete. 2020-11-16T22:36:13.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.036+0000 7feaac044740 20 client.8476 trim_cache size 0 max 16384 2020-11-16T22:36:13.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.056+0000 7feaac044740 1 Processor -- start 2020-11-16T22:36:13.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.056+0000 7feaac044740 1 -- start start 2020-11-16T22:36:13.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2585f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x258bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2440ef0 con 0x22afc60 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2530920 con 0x231d860 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x253d910 con 0x24ebef0 2020-11-16T22:36:13.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea977fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x258bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45044/0 (socket says 172.21.15.181:45044) 2020-11-16T22:36:13.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 -- 172.21.15.181:0/3022474511 learned_addr learned my addr 172.21.15.181:0/3022474511 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea967fc700 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2585f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 -- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 msgr2=0x258bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x258bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 -- 172.21.15.181:0/3022474511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 msgr2=0x2585f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2585f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 -- 172.21.15.181:0/3022474511 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640081d0 con 0x24ebef0 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea96ffd700 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fea88000f40 tx=0x7fea8802c000).ready entity=mon.2 client_cookie=eba568e44280c1c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea8cff9700 1 -- 172.21.15.181:0/3022474511 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88029040 con 0x24ebef0 2020-11-16T22:36:13.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea8cff9700 1 -- 172.21.15.181:0/3022474511 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8801dea0 con 0x24ebef0 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7fea8cff9700 1 -- 172.21.15.181:0/3022474511 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880447d0 con 0x24ebef0 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 msgr2=0x253d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fea88000f40 tx=0x7fea8802c000).stop 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- 172.21.15.181:0/3022474511 shutdown_connections 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x258bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x253d250 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 --2- 172.21.15.181:0/3022474511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2585f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- 172.21.15.181:0/3022474511 >> 172.21.15.181:0/3022474511 conn(0x7fea64002320 msgr2=0x2586e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- 172.21.15.181:0/3022474511 shutdown_connections 2020-11-16T22:36:13.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.060+0000 7feaac044740 1 -- 172.21.15.181:0/3022474511 wait complete. 2020-11-16T22:36:13.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 -- start start 2020-11-16T22:36:13.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 Processor -- start 2020-11-16T22:36:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2586c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x25dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25394f0 con 0x22afc60 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e5bc0 con 0x231d860 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25871e0 con 0x24ebef0 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x25dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45054/0 (socket says 172.21.15.181:45054) 2020-11-16T22:36:13.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 -- 172.21.15.181:0/2430856983 learned_addr learned my addr 172.21.15.181:0/2430856983 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2586c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7feaac044740 10 client.0 ll_register_callbacks cb 0x253cc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 -- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 msgr2=0x25dfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x25dfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 -- 172.21.15.181:0/2430856983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 msgr2=0x2586c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.064+0000 7fea977fe700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2586c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea977fe700 1 -- 172.21.15.181:0/2430856983 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x22cd7d0 con 0x24ebef0 2020-11-16T22:36:13.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2586c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:13.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea977fe700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fea90010000 tx=0x7fea90015000).ready entity=mon.2 client_cookie=3349f4d6abab4169 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90024070 con 0x24ebef0 2020-11-16T22:36:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea88044660 con 0x24ebef0 2020-11-16T22:36:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea9001f440 con 0x24ebef0 2020-11-16T22:36:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea90009cf0 con 0x24ebef0 2020-11-16T22:36:13.061 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 10 client.7673 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x22d8eb0 con 0x24ebef0 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88044660 con 0x24ebef0 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fea9001f810 con 0x24ebef0 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 10 client.7673 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 10 client.7673 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:13.062 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 20 client.7673 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x253c080 0x259e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 4 client.7673 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x2361000 con 0x253c080 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x253c080 0x259e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 10 client.7673 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea96ffd700 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x253c080 0x259e750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c03b17f1ce58d5fa server_cookie=b60a0d8bc3d9567 in_seq=0 out_seq=0 2020-11-16T22:36:13.063 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 1 -- 172.21.15.181:0/2430856983 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x2361000 con 0x253c080 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7fea6d7fa700 10 client.7673 handle_command_reply: tid=0 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 client.7673 shutdown 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 msgr2=0x258c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.068+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fea90010000 tx=0x7fea90015000).stop 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 shutdown_connections 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x253c080 0x259e750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24ebef0 0x258c160 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.064 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x22afc60 0x2586c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea6d7fa700 0 client.7673 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 --2- 172.21.15.181:0/2430856983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x231d860 0x25dfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 >> 172.21.15.181:0/2430856983 conn(0x7fea64002320 msgr2=0x2530770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 shutdown_connections 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 1 -- 172.21.15.181:0/2430856983 wait complete. 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7feaac044740 20 client.7673 trim_cache size 0 max 16384 2020-11-16T22:36:13.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 msgr2=0x7fea74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 0x7fea74057520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fea980d7350 tx=0x7fea80006df0).stop 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 msgr2=0x7fea980ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fea8800dcb0 tx=0x7fea88008040).stop 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 shutdown_connections 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fea74055060 0x7fea74057520 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea981428b0 0x7fea98160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea98081170 0x7fea980ceff0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 --2- 172.21.15.181:0/25354473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea980d1060 0x7fea98170990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 >> 172.21.15.181:0/25354473 conn(0x7fea980682f0 msgr2=0x7fea98142190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 shutdown_connections 2020-11-16T22:36:13.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.072+0000 7fea97fff700 1 -- 172.21.15.181:0/25354473 wait complete. 2020-11-16T22:36:13.067 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:36:13.072+0000 7ff284d2c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:35:37.398734+0000 1=0+1) n(v4 rc2020-11-16T22:35:42.242633+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={8284=pAsLsXsFsx/pAsLsXsFsx@5},l=8284 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558897a3c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:35:37.398734+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:35:42.242633+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:35:37.398734+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:35:42.242633+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.067 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:13.068 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2020-11-16T22:36:13.068 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:13.068 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2020-11-16T22:36:13.068 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:13.069 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:36:13.071+0000 7f279768e700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1) n(v2 rc2020-11-16T22:35:42.242633+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={8284=pAsLsXsFsx/pAsLsXsFsx@4},l=8284 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x562e8b68cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:35:42.242633+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:35:42.242633+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.069 INFO:tasks.ceph.mds.i.smithi175.stderr:2020-11-16T22:36:13.075+0000 7f279d69a700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1) n(v1 rc2020-11-16T22:35:42.242633+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={8284=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x562e8b68d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:35:37.406734+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:35:42.242633+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.069 INFO:tasks.ceph.mds.e.smithi175.stderr:2020-11-16T22:36:13.071+0000 7f7c3cea1700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1) n(v0 rc2020-11-16T22:35:37.406734+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={8284=pAsLsXsFsx/pAsLsXsFsx@4},l=8284 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x558957339180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:35:37.406734+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:35:37.406734+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:35:37.406734+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.069 INFO:tasks.ceph.mds.e.smithi175.stderr:2020-11-16T22:36:13.071+0000 7f7c3cea1700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={8284=pAsLsXsFsx/AsLsXsFsx@2},l=8284 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x558957339700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.070 INFO:tasks.ceph.mds.d.smithi181.stderr:2020-11-16T22:36:13.072+0000 7ff28ad38700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1) n(v3 rc2020-11-16T22:35:42.242633+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={8284=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558897a3c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v23>,<0x10000000000/d2 v27>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:35:42.242633+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:35:37.402734+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:35:42.242633+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:36:13.275 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 Processor -- start 2020-11-16T22:36:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 -- start start 2020-11-16T22:36:13.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 0x7fbb381492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb38061b50 con 0x7fbb380816e0 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380618b0 con 0x7fbb38145a30 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb37fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380cb930 con 0x7fbb38145e10 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb377fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 0x7fbb381492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb36ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbb38145a30 0x7fbb38080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51520/0 (socket says 172.21.15.181:51520) 2020-11-16T22:36:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.416+0000 7fbb36ffd700 1 -- 172.21.15.181:0/3169628959 learned_addr learned my addr 172.21.15.181:0/3169628959 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb377fe700 1 -- 172.21.15.181:0/3169628959 >> v1:172.21.15.181:6789/0 conn(0x7fbb38145a30 legacy=0x7fbb38080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb377fe700 1 -- 172.21.15.181:0/3169628959 >> v1:172.21.15.175:6789/0 conn(0x7fbb380816e0 legacy=0x7fbb380cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb35ffb700 1 -- 172.21.15.181:0/3169628959 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456134696 0 0) 0x7fbb380618b0 con 0x7fbb38145a30 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb377fe700 1 -- 172.21.15.181:0/3169628959 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380cbbd0 con 0x7fbb38145e10 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb377fe700 1 --2- 172.21.15.181:0/3169628959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 0x7fbb381492c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fbb28009cc0 tx=0x7fbb280091d0).ready entity=mon.2 client_cookie=76e3f4df0434a7a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb35ffb700 1 -- 172.21.15.181:0/3169628959 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800bbc0 con 0x7fbb38145e10 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb35ffb700 1 -- 172.21.15.181:0/3169628959 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2800bd20 con 0x7fbb38145e10 2020-11-16T22:36:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb35ffb700 1 -- 172.21.15.181:0/3169628959 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28011610 con 0x7fbb38145e10 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- 172.21.15.181:0/3169628959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 msgr2=0x7fbb381492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 --2- 172.21.15.181:0/3169628959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 0x7fbb381492c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fbb28009cc0 tx=0x7fbb280091d0).stop 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- 172.21.15.181:0/3169628959 shutdown_connections 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 --2- 172.21.15.181:0/3169628959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb38145e10 0x7fbb381492c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- 172.21.15.181:0/3169628959 >> 172.21.15.181:0/3169628959 conn(0x7fbb380d68c0 msgr2=0x7fbb380d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- 172.21.15.181:0/3169628959 shutdown_connections 2020-11-16T22:36:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- 172.21.15.181:0/3169628959 wait complete. 2020-11-16T22:36:13.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 Processor -- start 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.420+0000 7fbb37fff700 1 -- start start 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 0x7fbb38147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 0x7fbb38144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb38145e10 0x7fbb38141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3813c780 con 0x7fbb38145e10 2020-11-16T22:36:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380610e0 con 0x7fbb38145a30 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3816f390 con 0x7fbb380816e0 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 0x7fbb38147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 0x7fbb38144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 0x7fbb38147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45064/0 (socket says 172.21.15.181:45064) 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb36ffd700 1 -- 172.21.15.181:0/2727885845 learned_addr learned my addr 172.21.15.181:0/2727885845 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb377fe700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb38145e10 0x7fbb38141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 -- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 msgr2=0x7fbb38147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 0x7fbb38147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 -- 172.21.15.181:0/2727885845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb38145e10 msgr2=0x7fbb38141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb38145e10 0x7fbb38141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 -- 172.21.15.181:0/2727885845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3816f630 con 0x7fbb38145a30 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb367fc700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 0x7fbb38144330 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fbb2000c710 tx=0x7fbb20007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20007020 con 0x7fbb38145a30 2020-11-16T22:36:13.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20007180 con 0x7fbb38145a30 2020-11-16T22:36:13.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3816f8d0 con 0x7fbb38145a30 2020-11-16T22:36:13.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200064f0 con 0x7fbb38145a30 2020-11-16T22:36:13.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb38061400 con 0x7fbb38145a30 2020-11-16T22:36:13.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.424+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb38044f60 con 0x7fbb38145a30 2020-11-16T22:36:13.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.428+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb20010070 con 0x7fbb38145a30 2020-11-16T22:36:13.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.428+0000 7fbb2effd700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 0x7fbb08057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.428+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb2006d860 con 0x7fbb38145a30 2020-11-16T22:36:13.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.428+0000 7fbb36ffd700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 0x7fbb08057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.428+0000 7fbb36ffd700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 0x7fbb08057520 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fbb30007120 tx=0x7fbb30007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.432+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbb20043b40 con 0x7fbb38145a30 2020-11-16T22:36:13.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.632+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb38068e30 con 0x7fbb38145a30 2020-11-16T22:36:13.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb2effd700 1 -- 172.21.15.181:0/2727885845 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7fbb20020020 con 0x7fbb38145a30 2020-11-16T22:36:13.624 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:13.624 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:13.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 msgr2=0x7fbb08057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 0x7fbb08057520 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fbb30007120 tx=0x7fbb30007a20).stop 2020-11-16T22:36:13.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 msgr2=0x7fbb38144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 0x7fbb38144330 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fbb2000c710 tx=0x7fbb20007b90).stop 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 shutdown_connections 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbb08055060 0x7fbb08057520 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb38145e10 0x7fbb38141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb380816e0 0x7fbb38147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 --2- 172.21.15.181:0/2727885845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb38145a30 0x7fbb38144330 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 >> 172.21.15.181:0/2727885845 conn(0x7fbb380d68c0 msgr2=0x7fbb38141320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 shutdown_connections 2020-11-16T22:36:13.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.636+0000 7fbb37fff700 1 -- 172.21.15.181:0/2727885845 wait complete. 2020-11-16T22:36:13.627 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:13.638 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub pause 2020-11-16T22:36:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.772+0000 7f2583364700 1 Processor -- start 2020-11-16T22:36:13.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.772+0000 7f2583364700 1 -- start start 2020-11-16T22:36:13.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c145a30 0x7f257c145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f257c1492c0 0x7f257c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.765 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f257c0814c0 0x7f257c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c061b50 con 0x7f257c0814c0 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c0618b0 con 0x7f257c1492c0 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0cb930 con 0x7f257c145a30 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f257c1492c0 0x7f257c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f257c1492c0 0x7f257c080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37230/0 (socket says 172.21.15.181:37230) 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 -- 172.21.15.181:0/2459643903 learned_addr learned my addr 172.21.15.181:0/2459643903 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.766 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2582362700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c145a30 0x7f257c145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2582b63700 1 --2- 172.21.15.181:0/2459643903 >> v2:172.21.15.175:3300/0 conn(0x7f257c0814c0 0x7f257c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 -- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c145a30 msgr2=0x7f257c145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c145a30 0x7f257c145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 -- 172.21.15.181:0/2459643903 >> v2:172.21.15.175:3300/0 conn(0x7f257c0814c0 msgr2=0x7f257c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 --2- 172.21.15.181:0/2459643903 >> v2:172.21.15.175:3300/0 conn(0x7f257c0814c0 0x7f257c0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 -- 172.21.15.181:0/2459643903 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0cbbd0 con 0x7f257c1492c0 2020-11-16T22:36:13.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581b61700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c1492c0 0x7f257c080fd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f257c0cdbd0 tx=0x7f2578006eb0).ready entity=mon.1 client_cookie=ad8d5072fb06d974 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581360700 1 -- 172.21.15.181:0/2459643903 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578009720 con 0x7f257c1492c0 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581360700 1 -- 172.21.15.181:0/2459643903 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578009880 con 0x7f257c1492c0 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2581360700 1 -- 172.21.15.181:0/2459643903 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578009c00 con 0x7f257c1492c0 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c1492c0 msgr2=0x7f257c080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c1492c0 0x7f257c080fd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f257c0cdbd0 tx=0x7f2578006eb0).stop 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- 172.21.15.181:0/2459643903 shutdown_connections 2020-11-16T22:36:13.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- 172.21.15.181:0/2459643903 >> v2:172.21.15.175:3300/0 conn(0x7f257c0814c0 0x7f257c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c145a30 0x7f257c145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 --2- 172.21.15.181:0/2459643903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c1492c0 0x7f257c080fd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- 172.21.15.181:0/2459643903 >> 172.21.15.181:0/2459643903 conn(0x7f257c0d68c0 msgr2=0x7f257c0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- 172.21.15.181:0/2459643903 shutdown_connections 2020-11-16T22:36:13.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.776+0000 7f2583364700 1 -- 172.21.15.181:0/2459643903 wait complete. 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 Processor -- start 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- start start 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 0x7f257c13c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 0x7f257c147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.771 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c13c9f0 con 0x7f257c145a30 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c13cc90 con 0x7f257c0814c0 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c13cf30 con 0x7f257c1492c0 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 0x7f257c13c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45082/0 (socket says 172.21.15.181:45082) 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 0x7f257c13c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37236/0 (socket says 172.21.15.181:37236) 2020-11-16T22:36:13.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 -- 172.21.15.181:0/2195570311 learned_addr learned my addr 172.21.15.181:0/2195570311 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2581b61700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 0x7f257c147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 -- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 msgr2=0x7f257c13c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 0x7f257c13c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 -- 172.21.15.181:0/2195570311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 msgr2=0x7f257c147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 0x7f257c147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 -- 172.21.15.181:0/2195570311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c13d1d0 con 0x7f257c1492c0 2020-11-16T22:36:13.773 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2581b61700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 0x7f257c147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2582b63700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f257400ecf0 tx=0x7f25740091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f25727fc700 1 -- 172.21.15.181:0/2195570311 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574009520 con 0x7f257c1492c0 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f25727fc700 1 -- 172.21.15.181:0/2195570311 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257400ca00 con 0x7f257c1492c0 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f25727fc700 1 -- 172.21.15.181:0/2195570311 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400cda0 con 0x7f257c1492c0 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c0d13e0 con 0x7f257c1492c0 2020-11-16T22:36:13.774 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.780+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c13c7c0 con 0x7f257c1492c0 2020-11-16T22:36:13.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.784+0000 7f25727fc700 1 -- 172.21.15.181:0/2195570311 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2574022070 con 0x7f257c1492c0 2020-11-16T22:36:13.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.784+0000 7f25727fc700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 0x7f2554057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.775 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.784+0000 7f25727fc700 1 -- 172.21.15.181:0/2195570311 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f257406e450 con 0x7f257c1492c0 2020-11-16T22:36:13.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.784+0000 7f2582362700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 0x7f2554057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.784+0000 7f2582362700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 0x7f2554057550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f256c006fd0 tx=0x7f256c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 Processor -- start 2020-11-16T22:36:13.776 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- start start 2020-11-16T22:36:13.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.777 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 0x21406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd24a0 con 0x1ebaba0 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2086920 con 0x1f287a0 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f466e0 con 0x2146fc0 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.778 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37244/0 (socket says 172.21.15.181:37244) 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 -- 172.21.15.181:0/1060262853 learned_addr learned my addr 172.21.15.181:0/1060262853 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 0x21406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2581b61700 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 -- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 msgr2=0x21406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 0x21406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 -- 172.21.15.181:0/1060262853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 msgr2=0x2147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 -- 172.21.15.181:0/1060262853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f41590 con 0x1f287a0 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582b63700 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 0x21406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2582362700 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f256c014e50 tx=0x7f256c013000).ready entity=mon.1 client_cookie=ecf57da59e45c789 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f255b7fe700 1 -- 172.21.15.181:0/1060262853 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c011b20 con 0x1f287a0 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f255b7fe700 1 -- 172.21.15.181:0/1060262853 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c011c80 con 0x1f287a0 2020-11-16T22:36:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f255b7fe700 1 -- 172.21.15.181:0/1060262853 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c020410 con 0x1f287a0 2020-11-16T22:36:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 msgr2=0x2149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f256c014e50 tx=0x7f256c013000).stop 2020-11-16T22:36:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- 172.21.15.181:0/1060262853 shutdown_connections 2020-11-16T22:36:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2146fc0 0x21406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x2149710 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 --2- 172.21.15.181:0/1060262853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- 172.21.15.181:0/1060262853 >> 172.21.15.181:0/1060262853 conn(0x20f6e30 msgr2=0x2143390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- 172.21.15.181:0/1060262853 shutdown_connections 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.788+0000 7f2593057740 1 -- 172.21.15.181:0/1060262853 wait complete. 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- start start 2020-11-16T22:36:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 Processor -- start 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 0x21a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2146fc0 0x21693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x20736d0 con 0x2146fc0 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2073810 con 0x1ebaba0 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2074a80 con 0x1f287a0 2020-11-16T22:36:13.785 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 10 client.0 ll_register_callbacks cb 0x2069ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2581b61700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 0x21a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37250/0 (socket says 172.21.15.181:37250) 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 -- 172.21.15.181:0/2402729742 learned_addr learned my addr 172.21.15.181:0/2402729742 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2581b61700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 0x21a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45094/0 (socket says 172.21.15.181:45094) 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582b63700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2146fc0 0x21693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.786 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 -- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 msgr2=0x21a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 0x21a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 -- 172.21.15.181:0/2402729742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2146fc0 msgr2=0x21693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2146fc0 0x21693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 -- 172.21.15.181:0/2402729742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2064270 con 0x1ebaba0 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2582362700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x217c3a0 tx=0x7f256c01e000).ready entity=mon.1 client_cookie=6b22f9b9dde9df8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:13.787 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c021920 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f256c021a80 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21276b0 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f256c021e50 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 10 client.8500 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x1e99fc0 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21276b0 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f256c02a410 con 0x1ebaba0 2020-11-16T22:36:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 10 client.8500 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 10 client.8500 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 20 client.8500 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x21a66c0 0x2194c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 4 client.8500 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1f64c40 con 0x21a66c0 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.792+0000 7f2581b61700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x21a66c0 0x2194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2558ff9700 10 client.8500 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.790 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2581b61700 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x21a66c0 0x2194c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=39b667d485dc7f6c server_cookie=b4b1f011494b2019 in_seq=0 out_seq=0 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2558ff9700 1 -- 172.21.15.181:0/2402729742 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1f64c40 con 0x21a66c0 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2558ff9700 10 client.8500 handle_command_reply: tid=0 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 client.8500 shutdown 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 msgr2=0x21434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x217c3a0 tx=0x7f256c01e000).stop 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 shutdown_connections 2020-11-16T22:36:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x21a66c0 0x2194c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2146fc0 0x21693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21434a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2558ff9700 0 client.8500 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 --2- 172.21.15.181:0/2402729742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f287a0 0x21a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 >> 172.21.15.181:0/2402729742 conn(0x20f6e30 msgr2=0x2148330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 shutdown_connections 2020-11-16T22:36:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 1 -- 172.21.15.181:0/2402729742 wait complete. 2020-11-16T22:36:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.796+0000 7f2593057740 20 client.8500 trim_cache size 0 max 16384 2020-11-16T22:36:13.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 Processor -- start 2020-11-16T22:36:13.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 -- start start 2020-11-16T22:36:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2191010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x204be30 con 0x1ebaba0 2020-11-16T22:36:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x21917f0 con 0x1f287a0 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21434a0 con 0x20f6e30 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.816+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37260/0 (socket says 172.21.15.181:37260) 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45100/0 (socket says 172.21.15.181:45100) 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 -- 172.21.15.181:0/586315757 learned_addr learned my addr 172.21.15.181:0/586315757 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2581b61700 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2191010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 -- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 msgr2=0x21797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 -- 172.21.15.181:0/586315757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 msgr2=0x2191010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2191010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 -- 172.21.15.181:0/586315757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25480081d0 con 0x1f287a0 2020-11-16T22:36:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2581b61700 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2191010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2582b63700 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f2574006dc0 tx=0x7f2574028040).ready entity=mon.1 client_cookie=33fbfb48786d7514 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f255bfff700 1 -- 172.21.15.181:0/586315757 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257402b040 con 0x1f287a0 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f255bfff700 1 -- 172.21.15.181:0/586315757 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574044c30 con 0x1f287a0 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f255bfff700 1 -- 172.21.15.181:0/586315757 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574047a70 con 0x1f287a0 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 msgr2=0x21410c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f2574006dc0 tx=0x7f2574028040).stop 2020-11-16T22:36:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- 172.21.15.181:0/586315757 shutdown_connections 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1f287a0 0x21410c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 --2- 172.21.15.181:0/586315757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ebaba0 0x2191010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- 172.21.15.181:0/586315757 >> 172.21.15.181:0/586315757 conn(0x7f2548002320 msgr2=0x219bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- 172.21.15.181:0/586315757 shutdown_connections 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- 172.21.15.181:0/586315757 wait complete. 2020-11-16T22:36:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 -- start start 2020-11-16T22:36:13.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.820+0000 7f2593057740 1 Processor -- start 2020-11-16T22:36:13.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 0x217ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x21999b0 con 0x1f287a0 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x217b080 con 0x1ebaba0 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x21f9c50 con 0x20f6e30 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 10 client.0 ll_register_callbacks cb 0x21479d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:13.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37266/0 (socket says 172.21.15.181:37266) 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582362700 1 -- 172.21.15.181:0/1094815095 learned_addr learned my addr 172.21.15.181:0/1094815095 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2581b61700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 0x217ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 -- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 msgr2=0x21994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.816 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 -- 172.21.15.181:0/1094815095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 msgr2=0x217ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 0x217ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 -- 172.21.15.181:0/1094815095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ed8710 con 0x20f6e30 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2581b61700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 0x217ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582362700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582b63700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21f2f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f257406e010 tx=0x7f257402c000).ready entity=mon.2 client_cookie=ce2ba20bec8bdc3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:13.817 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574034030 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2574035070 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f256c011c80 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257402b040 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 10 client.7697 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x1ee3df0 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574042020 con 0x20f6e30 2020-11-16T22:36:13.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f25740472d0 con 0x20f6e30 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 10 client.7697 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 10 client.7697 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 20 client.7697 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2146fc0 0x2196460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 4 client.7697 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0x1f6bf40 con 0x2146fc0 2020-11-16T22:36:13.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2582362700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2146fc0 0x2196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.824+0000 7f2558ff9700 10 client.7697 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2582362700 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2146fc0 0x2196460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1a076f2b5409c16d server_cookie=700a32ae1ae36b77 in_seq=0 out_seq=0 2020-11-16T22:36:13.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2558ff9700 1 -- 172.21.15.181:0/1094815095 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x1f6bf40 con 0x2146fc0 2020-11-16T22:36:13.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2558ff9700 10 client.7697 handle_command_reply: tid=0 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 client.7697 shutdown 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 msgr2=0x21f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21f2f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f257406e010 tx=0x7f257402c000).stop 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 shutdown_connections 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2146fc0 0x2196460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x20f6e30 0x21f2f20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ebaba0 0x21994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 --2- 172.21.15.181:0/1094815095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f287a0 0x217ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2558ff9700 0 client.7697 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 >> 172.21.15.181:0/1094815095 conn(0x7f2548002320 msgr2=0x219bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 shutdown_connections 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 1 -- 172.21.15.181:0/1094815095 wait complete. 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2593057740 20 client.7697 trim_cache size 0 max 16384 2020-11-16T22:36:13.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 msgr2=0x7f2554057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 0x7f2554057550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f256c006fd0 tx=0x7f256c006df0).stop 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 msgr2=0x7f257c1475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f257400ecf0 tx=0x7f25740091d0).stop 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 shutdown_connections 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2554055090 0x7f2554057550 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c1492c0 0x7f257c1475f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c0814c0 0x7f257c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 --2- 172.21.15.181:0/2195570311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f257c145a30 0x7f257c147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 >> 172.21.15.181:0/2195570311 conn(0x7f257c0d68c0 msgr2=0x7f257c13ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 shutdown_connections 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.828+0000 7f2583364700 1 -- 172.21.15.181:0/2195570311 wait complete. 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:13.825 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:13.834 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 Processor -- start 2020-11-16T22:36:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- start start 2020-11-16T22:36:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10200c8b80 0x7f1020164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1020063d00 con 0x7f1020066230 2020-11-16T22:36:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1020063fa0 con 0x7f1020066610 2020-11-16T22:36:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1020064240 con 0x7f10200c8b80 2020-11-16T22:36:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f102533f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10200c8b80 0x7f1020164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1020066610 0x7f10200c8470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51578/0 (socket says 172.21.15.181:51578) 2020-11-16T22:36:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101ffff700 1 -- 172.21.15.181:0/4036085889 learned_addr learned my addr 172.21.15.181:0/4036085889 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101f7fe700 1 -- 172.21.15.181:0/4036085889 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817408695 0 0) 0x7f1020063d00 con 0x7f1020066230 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f102533f700 1 -- 172.21.15.181:0/4036085889 >> v1:172.21.15.181:6789/0 conn(0x7f1020066610 legacy=0x7f10200c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f102533f700 1 -- 172.21.15.181:0/4036085889 >> v1:172.21.15.175:6789/0 conn(0x7f1020066230 legacy=0x7f10200632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f102533f700 1 -- 172.21.15.181:0/4036085889 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200644e0 con 0x7f10200c8b80 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f102533f700 1 --2- 172.21.15.181:0/4036085889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10200c8b80 0x7f1020164230 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f100800b920 tx=0x7f1008006df0).ready entity=mon.1 client_cookie=51f02c2016027be5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101f7fe700 1 -- 172.21.15.181:0/4036085889 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008009730 con 0x7f10200c8b80 2020-11-16T22:36:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101f7fe700 1 -- 172.21.15.181:0/4036085889 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1008009890 con 0x7f10200c8b80 2020-11-16T22:36:13.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f101f7fe700 1 -- 172.21.15.181:0/4036085889 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008009c10 con 0x7f10200c8b80 2020-11-16T22:36:13.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- 172.21.15.181:0/4036085889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10200c8b80 msgr2=0x7f1020164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 --2- 172.21.15.181:0/4036085889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10200c8b80 0x7f1020164230 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f100800b920 tx=0x7f1008006df0).stop 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- 172.21.15.181:0/4036085889 shutdown_connections 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 --2- 172.21.15.181:0/4036085889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10200c8b80 0x7f1020164230 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.972+0000 7f1025b40700 1 -- 172.21.15.181:0/4036085889 >> 172.21.15.181:0/4036085889 conn(0x7f10200cfe90 msgr2=0x7f10200d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- 172.21.15.181:0/4036085889 shutdown_connections 2020-11-16T22:36:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- 172.21.15.181:0/4036085889 wait complete. 2020-11-16T22:36:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 Processor -- start 2020-11-16T22:36:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- start start 2020-11-16T22:36:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1020066230 0x7f1020174ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10200c8b80 0x7f1020176be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1020177af0 con 0x7f10200c8b80 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10201770d0 con 0x7f1020066610 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1020177310 con 0x7f1020066230 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1024b3e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1020066230 0x7f1020174ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37278/0 (socket says 172.21.15.181:37278) 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 -- 172.21.15.181:0/495193158 learned_addr learned my addr 172.21.15.181:0/495193158 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f102533f700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10200c8b80 0x7f1020176be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 -- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1020066230 msgr2=0x7f1020174ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1020066230 0x7f1020174ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 -- 172.21.15.181:0/495193158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10200c8b80 msgr2=0x7f1020176be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10200c8b80 0x7f1020176be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 -- 172.21.15.181:0/495193158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10201775b0 con 0x7f1020066610 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101ffff700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f101800cac0 tx=0x7f1018007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018005b80 con 0x7f1020066610 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018005ce0 con 0x7f1020066610 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1020177850 con 0x7f1020066610 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018007360 con 0x7f1020066610 2020-11-16T22:36:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.976+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200b4580 con 0x7f1020066610 2020-11-16T22:36:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1020044f60 con 0x7f1020066610 2020-11-16T22:36:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1018010030 con 0x7f1020066610 2020-11-16T22:36:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f101cff9700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 0x7f0ff0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:13.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f101806de20 con 0x7f1020066610 2020-11-16T22:36:13.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f1024b3e700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 0x7f0ff0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:13.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.980+0000 7f1024b3e700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 0x7f0ff0057520 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f10100011a0 tx=0x7f1010007a00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:13.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:13.984+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1018044020 con 0x7f1020066610 2020-11-16T22:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.188+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10200d0d10 con 0x7f1020066610 2020-11-16T22:36:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.188+0000 7f101cff9700 1 -- 172.21.15.181:0/495193158 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f1018047640 con 0x7f1020066610 2020-11-16T22:36:14.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:14.178 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 msgr2=0x7f0ff0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 0x7f0ff0057520 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f10100011a0 tx=0x7f1010007a00).stop 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 msgr2=0x7f10200cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f101800cac0 tx=0x7f1018007ef0).stop 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 shutdown_connections 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0ff0055060 0x7f0ff0057520 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10200c8b80 0x7f1020176be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1020066230 0x7f1020174ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 --2- 172.21.15.181:0/495193158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1020066610 0x7f10200cc670 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 >> 172.21.15.181:0/495193158 conn(0x7f10200cfe90 msgr2=0x7f10200c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 shutdown_connections 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.192+0000 7f1025b40700 1 -- 172.21.15.181:0/495193158 wait complete. 2020-11-16T22:36:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:14.200 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:36:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.332+0000 7f69c759e700 1 Processor -- start 2020-11-16T22:36:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.332+0000 7f69c759e700 1 -- start start 2020-11-16T22:36:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 0x7f69c8061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c80cccd0 con 0x7f69c815f1a0 2020-11-16T22:36:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c80ccf70 con 0x7f69c8156a90 2020-11-16T22:36:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815e030 con 0x7f69c80615f0 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 0x7f69c8061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45130/0 (socket says 172.21.15.181:45130) 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f69c815f1a0 0x7f69c8160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51594/0 (socket says 172.21.15.181:51594) 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 0x7f69c8061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37286/0 (socket says 172.21.15.181:37286) 2020-11-16T22:36:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 -- 172.21.15.181:0/133550292 learned_addr learned my addr 172.21.15.181:0/133550292 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c559a700 1 -- 172.21.15.181:0/133550292 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194893334 0 0) 0x7f69c80cccd0 con 0x7f69c815f1a0 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c559a700 1 -- 172.21.15.181:0/133550292 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69a8003240 con 0x7f69c815f1a0 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 -- 172.21.15.181:0/133550292 >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 msgr2=0x7f69c8061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/133550292 >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 0x7f69c8061100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 -- 172.21.15.181:0/133550292 >> v1:172.21.15.181:6789/0 conn(0x7f69c815f1a0 legacy=0x7f69c8160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 -- 172.21.15.181:0/133550292 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c815e2d0 con 0x7f69c80615f0 2020-11-16T22:36:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/133550292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f69b0006fd0 tx=0x7f69b0006eb0).ready entity=mon.2 client_cookie=eb45842894bcb0ac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c559a700 1 -- 172.21.15.181:0/133550292 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0009720 con 0x7f69c80615f0 2020-11-16T22:36:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c559a700 1 -- 172.21.15.181:0/133550292 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0009880 con 0x7f69c80615f0 2020-11-16T22:36:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.336+0000 7f69c559a700 1 -- 172.21.15.181:0/133550292 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0009c00 con 0x7f69c80615f0 2020-11-16T22:36:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- 172.21.15.181:0/133550292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 msgr2=0x7f69c815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 --2- 172.21.15.181:0/133550292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f69b0006fd0 tx=0x7f69b0006eb0).stop 2020-11-16T22:36:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- 172.21.15.181:0/133550292 shutdown_connections 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 --2- 172.21.15.181:0/133550292 >> v2:172.21.15.181:3300/0 conn(0x7f69c8156a90 0x7f69c8061100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 --2- 172.21.15.181:0/133550292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c815ecb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- 172.21.15.181:0/133550292 >> 172.21.15.181:0/133550292 conn(0x7f69c80cf630 msgr2=0x7f69c80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- 172.21.15.181:0/133550292 shutdown_connections 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- 172.21.15.181:0/133550292 wait complete. 2020-11-16T22:36:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 Processor -- start 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.340+0000 7f69c759e700 1 -- start start 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c8156a90 0x7f69c813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69c815f1a0 0x7f69c816d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815a820 con 0x7f69c815f1a0 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c80cbdb0 con 0x7f69c8156a90 2020-11-16T22:36:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c815aa80 con 0x7f69c80615f0 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45134/0 (socket says 172.21.15.181:45134) 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 -- 172.21.15.181:0/1894657903 learned_addr learned my addr 172.21.15.181:0/1894657903 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c8156a90 0x7f69c813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c6d9d700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69c815f1a0 0x7f69c816d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 -- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c8156a90 msgr2=0x7f69c813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c8156a90 0x7f69c813ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 -- 172.21.15.181:0/1894657903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69c815f1a0 msgr2=0x7f69c816d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69c815f1a0 0x7f69c816d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 -- 172.21.15.181:0/1894657903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c815ad20 con 0x7f69c80615f0 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c659c700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f69b800ef80 tx=0x7f69b80091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 -- 172.21.15.181:0/1894657903 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b800cd40 con 0x7f69c80615f0 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 -- 172.21.15.181:0/1894657903 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b800cea0 con 0x7f69c80615f0 2020-11-16T22:36:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c815afc0 con 0x7f69c80615f0 2020-11-16T22:36:14.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 -- 172.21.15.181:0/1894657903 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8012600 con 0x7f69c80615f0 2020-11-16T22:36:14.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c816cd70 con 0x7f69c80615f0 2020-11-16T22:36:14.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 -- 172.21.15.181:0/1894657903 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f69b8022070 con 0x7f69c80615f0 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 0x7f69a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69be7fc700 1 -- 172.21.15.181:0/1894657903 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69b806e5c0 con 0x7f69c80615f0 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 0x7f69a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.344+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 0x7f69a4057520 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f69b0009420 tx=0x7f69b0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.348+0000 7f69db571740 1 Processor -- start 2020-11-16T22:36:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.348+0000 7f69db571740 1 -- start start 2020-11-16T22:36:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x25f1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25eefd0 0x25e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x247a4b0 con 0x23d07b0 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x252e930 con 0x25eefd0 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23ee6f0 con 0x2362bb0 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45144/0 (socket says 172.21.15.181:45144) 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25eefd0 0x25e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 -- 172.21.15.181:0/4150603740 learned_addr learned my addr 172.21.15.181:0/4150603740 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c659c700 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x25f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 -- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25eefd0 msgr2=0x25e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25eefd0 0x25e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 -- 172.21.15.181:0/4150603740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 msgr2=0x25f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x25f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 -- 172.21.15.181:0/4150603740 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23e95a0 con 0x2362bb0 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c659c700 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x25f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f69b0009210 tx=0x7f69b001f000).ready entity=mon.2 client_cookie=3beff0551b57f9ff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f699f7fe700 1 -- 172.21.15.181:0/4150603740 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b001eb70 con 0x2362bb0 2020-11-16T22:36:14.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f699f7fe700 1 -- 172.21.15.181:0/4150603740 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b001ecd0 con 0x2362bb0 2020-11-16T22:36:14.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f699f7fe700 1 -- 172.21.15.181:0/4150603740 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b002d690 con 0x2362bb0 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 msgr2=0x25ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f69b0009210 tx=0x7f69b001f000).stop 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- 172.21.15.181:0/4150603740 shutdown_connections 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x25eefd0 0x25e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x25f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 --2- 172.21.15.181:0/4150603740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x25ef670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- 172.21.15.181:0/4150603740 >> 172.21.15.181:0/4150603740 conn(0x259ee40 msgr2=0x25eb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- 172.21.15.181:0/4150603740 shutdown_connections 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- 172.21.15.181:0/4150603740 wait complete. 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.352+0000 7f69db571740 1 -- start start 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 Processor -- start 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x2641070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 0x263b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x251b6e0 con 0x23d07b0 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x251b820 con 0x2362bb0 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x251ca90 con 0x25eefd0 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 10 client.0 ll_register_callbacks cb 0x2511ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37306/0 (socket says 172.21.15.181:37306) 2020-11-16T22:36:14.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 -- 172.21.15.181:0/4150076772 learned_addr learned my addr 172.21.15.181:0/4150076772 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 0x263b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x2641070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 -- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 msgr2=0x263b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 0x263b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 -- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 msgr2=0x2641070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x2641070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 -- 172.21.15.181:0/4150076772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x250c280 con 0x2362bb0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c6d9d700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 0x263b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c659c700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f69b800ef50 tx=0x7f69b806e040).ready entity=mon.1 client_cookie=3b5ede48352c1aa8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b806e0b0 con 0x2362bb0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69b806e210 con 0x2362bb0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25cf6c0 con 0x2362bb0 2020-11-16T22:36:14.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b806e3d0 con 0x2362bb0 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 10 client.8530 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x2341fd0 con 0x2362bb0 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25cf6c0 con 0x2362bb0 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f69b80444e0 con 0x2362bb0 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 10 client.8530 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 10 client.8530 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 20 client.8530 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:14.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25e9960 0x264e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 4 client.8530 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:14.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x240cc50 con 0x25e9960 2020-11-16T22:36:14.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25e9960 0x264e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f699cff9700 10 client.8530 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:14.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.356+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25e9960 0x264e310 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=4b441ed61d79bfa9 server_cookie=b56a730d69a7f1ae in_seq=0 out_seq=0 2020-11-16T22:36:14.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f699cff9700 1 -- 172.21.15.181:0/4150076772 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x240cc50 con 0x25e9960 2020-11-16T22:36:14.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f699cff9700 10 client.8530 handle_command_reply: tid=0 2020-11-16T22:36:14.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 client.8530 shutdown 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 msgr2=0x25eb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f69b800ef50 tx=0x7f69b806e040).stop 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 shutdown_connections 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25e9960 0x264e310 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25eefd0 0x263b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2362bb0 0x25eb590 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 --2- 172.21.15.181:0/4150076772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23d07b0 0x2641070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f699cff9700 0 client.8530 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:14.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 >> 172.21.15.181:0/4150076772 conn(0x259ee40 msgr2=0x2624990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 shutdown_connections 2020-11-16T22:36:14.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 1 -- 172.21.15.181:0/4150076772 wait complete. 2020-11-16T22:36:14.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.360+0000 7f69db571740 20 client.8530 trim_cache size 0 max 16384 2020-11-16T22:36:14.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 Processor -- start 2020-11-16T22:36:14.365 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 -- start start 2020-11-16T22:36:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259ee40 0x2638e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.366 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x25e3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24f3e40 con 0x259ee40 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2643b90 con 0x23d07b0 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26240a0 con 0x2362bb0 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x25e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45156/0 (socket says 172.21.15.181:45156) 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 -- 172.21.15.181:0/851135551 learned_addr learned my addr 172.21.15.181:0/851135551 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c659c700 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259ee40 0x2638e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 -- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 msgr2=0x25e3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x25e3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 -- 172.21.15.181:0/851135551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259ee40 msgr2=0x2638e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259ee40 0x2638e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 -- 172.21.15.181:0/851135551 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900081d0 con 0x2362bb0 2020-11-16T22:36:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x25ef480 tx=0x7f69b0013000).ready entity=mon.2 client_cookie=b906f7510c94bb65 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f699ffff700 1 -- 172.21.15.181:0/851135551 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b002da40 con 0x2362bb0 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f699ffff700 1 -- 172.21.15.181:0/851135551 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b002dba0 con 0x2362bb0 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.376+0000 7f699ffff700 1 -- 172.21.15.181:0/851135551 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0037430 con 0x2362bb0 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 msgr2=0x2622b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x25ef480 tx=0x7f69b0013000).stop 2020-11-16T22:36:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- 172.21.15.181:0/851135551 shutdown_connections 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x25e3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x259ee40 0x2638e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 --2- 172.21.15.181:0/851135551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2362bb0 0x2622b90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- 172.21.15.181:0/851135551 >> 172.21.15.181:0/851135551 conn(0x7f6990002320 msgr2=0x25eb590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- 172.21.15.181:0/851135551 shutdown_connections 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- 172.21.15.181:0/851135551 wait complete. 2020-11-16T22:36:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 -- start start 2020-11-16T22:36:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.380+0000 7f69db571740 1 Processor -- start 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2362bb0 0x26482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x259ee40 0x2640c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26a1900 con 0x2362bb0 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26391b0 con 0x23d07b0 2020-11-16T22:36:14.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2622c00 con 0x259ee40 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 10 client.0 ll_register_callbacks cb 0x264f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37322/0 (socket says 172.21.15.181:37322) 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x259ee40 0x2640c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 -- 172.21.15.181:0/3102156612 learned_addr learned my addr 172.21.15.181:0/3102156612 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c659c700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2362bb0 0x26482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 -- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x259ee40 msgr2=0x2640c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x259ee40 0x2640c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 -- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2362bb0 msgr2=0x26482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2362bb0 0x26482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 -- 172.21.15.181:0/3102156612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2380720 con 0x23d07b0 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c5d9b700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f69b0000dd0 tx=0x7f69b003a000).ready entity=mon.1 client_cookie=5e40266912347e46 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b001ea50 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69b0034040 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69b001ecd0 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b0037a70 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 10 client.8536 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x238be00 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b001ecd0 con 0x23d07b0 2020-11-16T22:36:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f69b0037bd0 con 0x23d07b0 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 10 client.8536 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 10 client.8536 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 20 client.8536 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25eefd0 0x2647240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 4 client.8536 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x2413f50 con 0x25eefd0 2020-11-16T22:36:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c659c700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25eefd0 0x2647240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 10 client.8536 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69c659c700 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25eefd0 0x2647240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=f8212ef134a0f6f7 server_cookie=2721f54db0583c57 in_seq=0 out_seq=0 2020-11-16T22:36:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 1 -- 172.21.15.181:0/3102156612 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x2413f50 con 0x25eefd0 2020-11-16T22:36:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 10 client.8536 handle_command_reply: tid=0 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 client.8536 shutdown 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 msgr2=0x26223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f69b0000dd0 tx=0x7f69b003a000).stop 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 shutdown_connections 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x25eefd0 0x2647240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x259ee40 0x2640c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2362bb0 0x26482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 --2- 172.21.15.181:0/3102156612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23d07b0 0x26223b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f699cff9700 0 client.8536 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 >> 172.21.15.181:0/3102156612 conn(0x7f6990002320 msgr2=0x263fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.384+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 shutdown_connections 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69db571740 1 -- 172.21.15.181:0/3102156612 wait complete. 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69db571740 20 client.8536 trim_cache size 0 max 16384 2020-11-16T22:36:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 msgr2=0x7f69a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 0x7f69a4057520 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f69b0009420 tx=0x7f69b0015000).stop 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 msgr2=0x7f69c80cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f69b800ef80 tx=0x7f69b80091d0).stop 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 shutdown_connections 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f69a4055060 0x7f69a4057520 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69c815f1a0 0x7f69c816d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69c80615f0 0x7f69c80cbfc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 --2- 172.21.15.181:0/1894657903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c8156a90 0x7f69c813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 >> 172.21.15.181:0/1894657903 conn(0x7f69c80cf630 msgr2=0x7f69c80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 shutdown_connections 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.388+0000 7f69c759e700 1 -- 172.21.15.181:0/1894657903 wait complete. 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:14.384 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:14.385 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:14.390 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.520+0000 7f292699d700 1 Processor -- start 2020-11-16T22:36:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.520+0000 7f292699d700 1 -- start start 2020-11-16T22:36:14.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 0x7f2920145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2920061b50 con 0x7f29200814c0 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200618b0 con 0x7f2920145a30 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200cb930 con 0x7f29201492c0 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292599b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 0x7f2920145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45172/0 (socket says 172.21.15.181:45172) 2020-11-16T22:36:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 -- 172.21.15.181:0/1177729083 learned_addr learned my addr 172.21.15.181:0/1177729083 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 -- 172.21.15.181:0/1177729083 >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 msgr2=0x7f2920145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 --2- 172.21.15.181:0/1177729083 >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 0x7f2920145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 -- 172.21.15.181:0/1177729083 >> v1:172.21.15.175:6789/0 conn(0x7f29200814c0 legacy=0x7f29200cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 -- 172.21.15.181:0/1177729083 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200cbbd0 con 0x7f29201492c0 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292599b700 1 --2- 172.21.15.181:0/1177729083 >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 0x7f2920145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292519a700 1 --2- 172.21.15.181:0/1177729083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f2908006fd0 tx=0x7f2908006eb0).ready entity=mon.2 client_cookie=b0a1cd3d5ace8ea6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f2924999700 1 -- 172.21.15.181:0/1177729083 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908009810 con 0x7f29201492c0 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f2924999700 1 -- 172.21.15.181:0/1177729083 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908009970 con 0x7f29201492c0 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f2924999700 1 -- 172.21.15.181:0/1177729083 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908009cf0 con 0x7f29201492c0 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- 172.21.15.181:0/1177729083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 msgr2=0x7f2920080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 --2- 172.21.15.181:0/1177729083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f2908006fd0 tx=0x7f2908006eb0).stop 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- 172.21.15.181:0/1177729083 shutdown_connections 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 --2- 172.21.15.181:0/1177729083 >> v2:172.21.15.181:3300/0 conn(0x7f2920145a30 0x7f2920145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 --2- 172.21.15.181:0/1177729083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29201492c0 0x7f2920080fd0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- 172.21.15.181:0/1177729083 >> 172.21.15.181:0/1177729083 conn(0x7f29200d68c0 msgr2=0x7f29200d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- 172.21.15.181:0/1177729083 shutdown_connections 2020-11-16T22:36:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.524+0000 7f292699d700 1 -- 172.21.15.181:0/1177729083 wait complete. 2020-11-16T22:36:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 Processor -- start 2020-11-16T22:36:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- start start 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920145a30 0x7f2920146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29201492c0 0x7f2920146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292015f490 con 0x7f2920145a30 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292015f6f0 con 0x7f29201492c0 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292015f990 con 0x7f29200814c0 2020-11-16T22:36:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292619c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29201492c0 0x7f2920146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45176/0 (socket says 172.21.15.181:45176) 2020-11-16T22:36:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 -- 172.21.15.181:0/1947805229 learned_addr learned my addr 172.21.15.181:0/1947805229 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292519a700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920145a30 0x7f2920146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 -- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29201492c0 msgr2=0x7f2920146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29201492c0 0x7f2920146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 -- 172.21.15.181:0/1947805229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920145a30 msgr2=0x7f2920146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920145a30 0x7f2920146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 -- 172.21.15.181:0/1947805229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292015fc30 con 0x7f29200814c0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292599b700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f291000c150 tx=0x7f29100091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000ccc0 con 0x7f29200814c0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000ce20 con 0x7f29200814c0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292015fed0 con 0x7f29200814c0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910012600 con 0x7f29200814c0 2020-11-16T22:36:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.528+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920140290 con 0x7f29200814c0 2020-11-16T22:36:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2910022070 con 0x7f29200814c0 2020-11-16T22:36:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920160370 con 0x7f29200814c0 2020-11-16T22:36:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f291dffb700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 0x7f28fc057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f292519a700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 0x7f28fc057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f291006e780 con 0x7f29200814c0 2020-11-16T22:36:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.532+0000 7f292519a700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 0x7f28fc057550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f290800b1b0 tx=0x7f2908015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:14.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.536+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2920160370 con 0x7f29200814c0 2020-11-16T22:36:14.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.736+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2920160370 con 0x7f29200814c0 2020-11-16T22:36:14.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.736+0000 7f291dffb700 1 -- 172.21.15.181:0/1947805229 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f2920160370 con 0x7f29200814c0 2020-11-16T22:36:14.725 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:14.725 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:14.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 msgr2=0x7f28fc057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 0x7f28fc057550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f290800b1b0 tx=0x7f2908015000).stop 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 msgr2=0x7f292013fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f291000c150 tx=0x7f29100091d0).stop 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 shutdown_connections 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28fc055090 0x7f28fc057550 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29201492c0 0x7f2920146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.729 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29200814c0 0x7f292013fd20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 --2- 172.21.15.181:0/1947805229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920145a30 0x7f2920146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:14.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 >> 172.21.15.181:0/1947805229 conn(0x7f29200d68c0 msgr2=0x7f292015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:14.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 shutdown_connections 2020-11-16T22:36:14.730 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.740+0000 7f292699d700 1 -- 172.21.15.181:0/1947805229 wait complete. 2020-11-16T22:36:14.730 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:14.745 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:36:14.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 Processor -- start 2020-11-16T22:36:14.869 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 -- start start 2020-11-16T22:36:14.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb454145770 0x7fb454145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4540d2a30 0x7fb4541608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:14.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb454081f20 con 0x7fb454160450 2020-11-16T22:36:14.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb454060470 con 0x7fb454145770 2020-11-16T22:36:14.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb458ee6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb454060710 con 0x7fb4540d2a30 2020-11-16T22:36:14.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb453fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4540d2a30 0x7fb4541608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:14.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb44bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb454160450 0x7fb4540d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51646/0 (socket says 172.21.15.181:51646) 2020-11-16T22:36:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb44bfff700 1 -- 172.21.15.181:0/3247363059 learned_addr learned my addr 172.21.15.181:0/3247363059 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.880+0000 7fb4537fe700 1 --2- 172.21.15.181:0/3247363059 >> v2:172.21.15.175:3300/0 conn(0x7fb454145770 0x7fb454145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb452ffd700 1 -- 172.21.15.181:0/3247363059 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496353780 0 0) 0x7fb454081f20 con 0x7fb454160450 2020-11-16T22:36:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb452ffd700 1 -- 172.21.15.181:0/3247363059 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb434003240 con 0x7fb454160450 2020-11-16T22:36:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb453fff700 1 -- 172.21.15.181:0/3247363059 >> v2:172.21.15.175:3300/0 conn(0x7fb454145770 msgr2=0x7fb454145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:15.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb453fff700 1 --2- 172.21.15.181:0/3247363059 >> v2:172.21.15.175:3300/0 conn(0x7fb454145770 0x7fb454145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb453fff700 1 -- 172.21.15.181:0/3247363059 >> v1:172.21.15.181:6789/0 conn(0x7fb454160450 legacy=0x7fb4540d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:15.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb453fff700 1 -- 172.21.15.181:0/3247363059 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540609b0 con 0x7fb4540d2a30 2020-11-16T22:36:15.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb453fff700 1 --2- 172.21.15.181:0/3247363059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540d2a30 0x7fb4541608b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fb4540bc9e0 tx=0x7fb44c0091d0).ready entity=mon.1 client_cookie=29406987e29fc6ed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:15.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb452ffd700 1 -- 172.21.15.181:0/3247363059 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00b9c0 con 0x7fb4540d2a30 2020-11-16T22:36:15.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb452ffd700 1 -- 172.21.15.181:0/3247363059 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c00bb20 con 0x7fb4540d2a30 2020-11-16T22:36:15.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb452ffd700 1 -- 172.21.15.181:0/3247363059 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00bea0 con 0x7fb4540d2a30 2020-11-16T22:36:15.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- 172.21.15.181:0/3247363059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540d2a30 msgr2=0x7fb4541608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:15.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 --2- 172.21.15.181:0/3247363059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540d2a30 0x7fb4541608b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fb4540bc9e0 tx=0x7fb44c0091d0).stop 2020-11-16T22:36:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- 172.21.15.181:0/3247363059 shutdown_connections 2020-11-16T22:36:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 --2- 172.21.15.181:0/3247363059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540d2a30 0x7fb4541608b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 --2- 172.21.15.181:0/3247363059 >> v2:172.21.15.175:3300/0 conn(0x7fb454145770 0x7fb454145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- 172.21.15.181:0/3247363059 >> 172.21.15.181:0/3247363059 conn(0x7fb4540d6760 msgr2=0x7fb4540d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- 172.21.15.181:0/3247363059 shutdown_connections 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- 172.21.15.181:0/3247363059 wait complete. 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 Processor -- start 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.884+0000 7fb458ee6700 1 -- start start 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb454145770 0x7fb4541440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb454144a90 con 0x7fb454145770 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45416f500 con 0x7fb454160450 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45416f7a0 con 0x7fb4540d2a30 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45190/0 (socket says 172.21.15.181:45190) 2020-11-16T22:36:15.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37352/0 (socket says 172.21.15.181:37352) 2020-11-16T22:36:15.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb4537fe700 1 -- 172.21.15.181:0/2605171962 learned_addr learned my addr 172.21.15.181:0/2605171962 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:15.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb44bfff700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb454145770 0x7fb4541440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 -- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 msgr2=0x7fb45413e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 -- 172.21.15.181:0/2605171962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb454145770 msgr2=0x7fb4541440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb454145770 0x7fb4541440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 -- 172.21.15.181:0/2605171962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45416fa40 con 0x7fb454160450 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:15.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb453fff700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fb4540bc9e0 tx=0x7fb44c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:15.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb44b7fe700 1 -- 172.21.15.181:0/2605171962 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c021070 con 0x7fb454160450 2020-11-16T22:36:15.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb44b7fe700 1 -- 172.21.15.181:0/2605171962 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c01c5f0 con 0x7fb454160450 2020-11-16T22:36:15.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45416fce0 con 0x7fb454160450 2020-11-16T22:36:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb44b7fe700 1 -- 172.21.15.181:0/2605171962 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c006ea0 con 0x7fb454160450 2020-11-16T22:36:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.888+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45416f1a0 con 0x7fb454160450 2020-11-16T22:36:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.892+0000 7fb44b7fe700 1 -- 172.21.15.181:0/2605171962 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb44c017070 con 0x7fb454160450 2020-11-16T22:36:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.892+0000 7fb44b7fe700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 0x7fb430057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.892+0000 7fb44b7fe700 1 -- 172.21.15.181:0/2605171962 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb44c06f530 con 0x7fb454160450 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.892+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 0x7fb430057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.892+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 0x7fb430057520 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb444006a90 tx=0x7fb444006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 Processor -- start 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- start start 2020-11-16T22:36:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2527670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2526fd0 0x2520700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x23b24b0 con 0x229abb0 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2466930 con 0x2526fd0 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23266f0 con 0x23087b0 2020-11-16T22:36:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45198/0 (socket says 172.21.15.181:45198) 2020-11-16T22:36:15.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 -- 172.21.15.181:0/2677033543 learned_addr learned my addr 172.21.15.181:0/2677033543 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:15.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb453fff700 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2526fd0 0x2520700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb44bfff700 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2527670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 -- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2526fd0 msgr2=0x2520700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2526fd0 0x2520700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 -- 172.21.15.181:0/2677033543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 msgr2=0x2527670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2527670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 -- 172.21.15.181:0/2677033543 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23215a0 con 0x23087b0 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb4537fe700 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb4440135b0 tx=0x7fb444014000).ready entity=mon.2 client_cookie=988e6c8c7ab9ae24 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:15.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb448ff9700 1 -- 172.21.15.181:0/2677033543 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444013dd0 con 0x23087b0 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb448ff9700 1 -- 172.21.15.181:0/2677033543 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44401e400 con 0x23087b0 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb448ff9700 1 -- 172.21.15.181:0/2677033543 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44401e780 con 0x23087b0 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 msgr2=0x2529720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb4440135b0 tx=0x7fb444014000).stop 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- 172.21.15.181:0/2677033543 shutdown_connections 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2526fd0 0x2520700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2529720 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 --2- 172.21.15.181:0/2677033543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2527670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.896+0000 7fb468bd9740 1 -- 172.21.15.181:0/2677033543 >> 172.21.15.181:0/2677033543 conn(0x24d6e40 msgr2=0x25233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- 172.21.15.181:0/2677033543 shutdown_connections 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- 172.21.15.181:0/2677033543 wait complete. 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- start start 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 Processor -- start 2020-11-16T22:36:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2523790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 0x2540e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24536e0 con 0x2526fd0 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2453820 con 0x229abb0 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2454a90 con 0x23087b0 2020-11-16T22:36:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 10 client.0 ll_register_callbacks cb 0x2449ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:16.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37362/0 (socket says 172.21.15.181:37362) 2020-11-16T22:36:16.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 -- 172.21.15.181:0/588873722 learned_addr learned my addr 172.21.15.181:0/588873722 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:16.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb44bfff700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2523790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb453fff700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 0x2540e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 -- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 msgr2=0x2523790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2523790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 -- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 msgr2=0x2540e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 0x2540e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 -- 172.21.15.181:0/588873722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2444280 con 0x229abb0 2020-11-16T22:36:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb453fff700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 0x2540e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb4537fe700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fb44401fee0 tx=0x7fb444021000).ready entity=mon.1 client_cookie=e760cbd2ea4ba377 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb429ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440126a0 con 0x229abb0 2020-11-16T22:36:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb444012800 con 0x229abb0 2020-11-16T22:36:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.900+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25076c0 con 0x229abb0 2020-11-16T22:36:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb444012bd0 con 0x229abb0 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 10 client.8557 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x2279fd0 con 0x229abb0 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25076c0 con 0x229abb0 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fb444038020 con 0x229abb0 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 10 client.8557 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 10 client.8557 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 20 client.8557 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526330 0x2557250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 4 client.8557 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2344c50 con 0x2526330 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb44bfff700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526330 0x2557250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 10 client.8557 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb44bfff700 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526330 0x2557250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=16960aba4edce23b server_cookie=c3932db658f2b1b5 in_seq=0 out_seq=0 2020-11-16T22:36:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 1 -- 172.21.15.181:0/588873722 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2344c50 con 0x2526330 2020-11-16T22:36:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 10 client.8557 handle_command_reply: tid=0 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 client.8557 shutdown 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 msgr2=0x25232a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fb44401fee0 tx=0x7fb444021000).stop 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 shutdown_connections 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526330 0x2557250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2526fd0 0x2540e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x229abb0 0x25232a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb429ffb700 0 client.8557 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 --2- 172.21.15.181:0/588873722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23087b0 0x2523790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 >> 172.21.15.181:0/588873722 conn(0x24d6e40 msgr2=0x2521630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 shutdown_connections 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 1 -- 172.21.15.181:0/588873722 wait complete. 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.904+0000 7fb468bd9740 20 client.8557 trim_cache size 0 max 16384 2020-11-16T22:36:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 Processor -- start 2020-11-16T22:36:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 -- start start 2020-11-16T22:36:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x255a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x25d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x242be40 con 0x229abb0 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x251c140 con 0x23087b0 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2577540 con 0x24d6e40 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.924+0000 7fb4537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45212/0 (socket says 172.21.15.181:45212) 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 -- 172.21.15.181:0/4102480812 learned_addr learned my addr 172.21.15.181:0/4102480812 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb453fff700 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x25d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb44bfff700 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x255a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 -- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 msgr2=0x25d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x25d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 -- 172.21.15.181:0/4102480812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 msgr2=0x255a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x255a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 -- 172.21.15.181:0/4102480812 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4200081d0 con 0x24d6e40 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4537fe700 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fb444006c50 tx=0x7fb444029000).ready entity=mon.2 client_cookie=d8f22eb8a3ed9b80 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:16.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4497fa700 1 -- 172.21.15.181:0/4102480812 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444012bd0 con 0x24d6e40 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4497fa700 1 -- 172.21.15.181:0/4102480812 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444012d30 con 0x24d6e40 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb4497fa700 1 -- 172.21.15.181:0/4102480812 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444029630 con 0x24d6e40 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 msgr2=0x255a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fb444006c50 tx=0x7fb444029000).stop 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- 172.21.15.181:0/4102480812 shutdown_connections 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x25d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x255a150 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 --2- 172.21.15.181:0/4102480812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x255a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- 172.21.15.181:0/4102480812 >> 172.21.15.181:0/4102480812 conn(0x7fb420002320 msgr2=0x25257c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- 172.21.15.181:0/4102480812 shutdown_connections 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- 172.21.15.181:0/4102480812 wait complete. 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 -- start start 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.928+0000 7fb468bd9740 1 Processor -- start 2020-11-16T22:36:16.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2521570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x2528c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2577430 con 0x229abb0 2020-11-16T22:36:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d9e80 con 0x23087b0 2020-11-16T22:36:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2589530 con 0x24d6e40 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 10 client.0 ll_register_callbacks cb 0x2527b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb453fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x2528c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37378/0 (socket says 172.21.15.181:37378) 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 -- 172.21.15.181:0/1514746843 learned_addr learned my addr 172.21.15.181:0/1514746843 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb4537fe700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2521570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 -- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 msgr2=0x2528c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x2528c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 -- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 msgr2=0x2521570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2521570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 -- 172.21.15.181:0/1514746843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x22b8720 con 0x23087b0 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb44bfff700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fb45415fbe0 tx=0x7fb43c007f90).ready entity=mon.1 client_cookie=3f30d3d8c0860bd8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb429ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00c070 con 0x23087b0 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb43c005020 con 0x23087b0 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb44401e400 con 0x23087b0 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb43c0053f0 con 0x23087b0 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 10 client.8563 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x22c3e00 con 0x23087b0 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.932+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44401e400 con 0x23087b0 2020-11-16T22:36:17.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fb43c005a70 con 0x23087b0 2020-11-16T22:36:17.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 10 client.8563 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:17.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 10 client.8563 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:17.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 20 client.8563 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:17.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526fd0 0x25783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 4 client.8563 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x234bf50 con 0x2526fd0 2020-11-16T22:36:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb4537fe700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526fd0 0x25783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb4537fe700 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526fd0 0x25783c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=c8f6e8ccf08390e3 server_cookie=1b6b0cfbb497129f in_seq=0 out_seq=0 2020-11-16T22:36:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb429ffb700 10 client.8563 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb429ffb700 1 -- 172.21.15.181:0/1514746843 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x234bf50 con 0x2526fd0 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb429ffb700 10 client.8563 handle_command_reply: tid=0 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 client.8563 shutdown 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 msgr2=0x2576f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fb45415fbe0 tx=0x7fb43c007f90).stop 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 shutdown_connections 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2526fd0 0x25783c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24d6e40 0x2528c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb429ffb700 0 client.8563 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x229abb0 0x2521570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 --2- 172.21.15.181:0/1514746843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23087b0 0x2576f40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 >> 172.21.15.181:0/1514746843 conn(0x7fb420002320 msgr2=0x2574720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:17.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 shutdown_connections 2020-11-16T22:36:17.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 1 -- 172.21.15.181:0/1514746843 wait complete. 2020-11-16T22:36:17.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb468bd9740 20 client.8563 trim_cache size 0 max 16384 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 msgr2=0x7fb430057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 0x7fb430057520 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb444006a90 tx=0x7fb444006df0).stop 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 msgr2=0x7fb4541445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fb4540bc9e0 tx=0x7fb44c016000).stop 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 shutdown_connections 2020-11-16T22:36:17.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb430055060 0x7fb430057520 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb454160450 0x7fb4541445a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4540d2a30 0x7fb45413e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 --2- 172.21.15.181:0/2605171962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb454145770 0x7fb4541440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 >> 172.21.15.181:0/2605171962 conn(0x7fb4540d6760 msgr2=0x7fb4540d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 shutdown_connections 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:14.936+0000 7fb458ee6700 1 -- 172.21.15.181:0/2605171962 wait complete. 2020-11-16T22:36:17.544 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:17.545 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:17.546 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:17.546 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:17.546 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:17.546 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:17.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.684+0000 7f40a7256700 1 Processor -- start 2020-11-16T22:36:17.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.684+0000 7f40a7256700 1 -- start start 2020-11-16T22:36:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f40a00b6e40 0x7f40a00b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a013fca0 con 0x7f40a00ce030 2020-11-16T22:36:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00c8700 con 0x7f40a00b6a60 2020-11-16T22:36:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00c89a0 con 0x7f40a00b6e40 2020-11-16T22:36:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a4ff2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40a00b6a60 0x7f40a0160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51690/0 (socket says 172.21.15.181:51690) 2020-11-16T22:36:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a4ff2700 1 -- 172.21.15.181:0/3731795445 learned_addr learned my addr 172.21.15.181:0/3731795445 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a57f3700 1 --2- 172.21.15.181:0/3731795445 >> v2:172.21.15.175:3300/0 conn(0x7f40a00b6e40 0x7f40a00b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f409f7fe700 1 -- 172.21.15.181:0/3731795445 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041515837 0 0) 0x7f40a00c8700 con 0x7f40a00b6a60 2020-11-16T22:36:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f409f7fe700 1 -- 172.21.15.181:0/3731795445 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4080003240 con 0x7f40a00b6a60 2020-11-16T22:36:17.678 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a57f3700 1 -- 172.21.15.181:0/3731795445 >> v1:172.21.15.181:6789/0 conn(0x7f40a00b6a60 legacy=0x7f40a0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a57f3700 1 -- 172.21.15.181:0/3731795445 >> v1:172.21.15.175:6789/0 conn(0x7f40a00ce030 legacy=0x7f40a00ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a57f3700 1 -- 172.21.15.181:0/3731795445 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00c8c40 con 0x7f40a00b6e40 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a57f3700 1 --2- 172.21.15.181:0/3731795445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00b6e40 0x7f40a00b7240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f4098009c90 tx=0x7f40980091d0).ready entity=mon.0 client_cookie=f386ace1cf696b43 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f409f7fe700 1 -- 172.21.15.181:0/3731795445 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800ba70 con 0x7f40a00b6e40 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f409f7fe700 1 -- 172.21.15.181:0/3731795445 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409800bbd0 con 0x7f40a00b6e40 2020-11-16T22:36:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f409f7fe700 1 -- 172.21.15.181:0/3731795445 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098011400 con 0x7f40a00b6e40 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- 172.21.15.181:0/3731795445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00b6e40 msgr2=0x7f40a00b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 --2- 172.21.15.181:0/3731795445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00b6e40 0x7f40a00b7240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f4098009c90 tx=0x7f40980091d0).stop 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- 172.21.15.181:0/3731795445 shutdown_connections 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 --2- 172.21.15.181:0/3731795445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00b6e40 0x7f40a00b7240 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- 172.21.15.181:0/3731795445 >> 172.21.15.181:0/3731795445 conn(0x7f40a0060bc0 msgr2=0x7f40a00d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- 172.21.15.181:0/3731795445 shutdown_connections 2020-11-16T22:36:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.688+0000 7f40a7256700 1 -- 172.21.15.181:0/3731795445 wait complete. 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 Processor -- start 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- start start 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 0x7f40a00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 0x7f40a015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cc850 con 0x7f40a00ce030 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a016fdf0 con 0x7f40a00b6e40 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a016f8b0 con 0x7f40a00b6a60 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a4ff2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 0x7f40a015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a4ff2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45234/0 (socket says 172.21.15.181:45234) 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a4ff2700 1 -- 172.21.15.181:0/2734871662 learned_addr learned my addr 172.21.15.181:0/2734871662 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f409ffff700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 0x7f40a00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 -- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 msgr2=0x7f40a00cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 -- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 msgr2=0x7f40a00cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 0x7f40a00cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 -- 172.21.15.181:0/2734871662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a016fb50 con 0x7f40a00ce030 2020-11-16T22:36:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f409ffff700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 0x7f40a00cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a4ff2700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a57f3700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 0x7f40a015ae80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f4098009c90 tx=0x7f4098016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801c5f0 con 0x7f40a00ce030 2020-11-16T22:36:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409801c750 con 0x7f40a00ce030 2020-11-16T22:36:17.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00cab30 con 0x7f40a00ce030 2020-11-16T22:36:17.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801cad0 con 0x7f40a00ce030 2020-11-16T22:36:17.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.692+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00cb070 con 0x7f40a00ce030 2020-11-16T22:36:17.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0069eb0 con 0x7f40a00ce030 2020-11-16T22:36:17.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4098024030 con 0x7f40a00ce030 2020-11-16T22:36:17.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f409cff9700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 0x7f407c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:17.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f409806f750 con 0x7f40a00ce030 2020-11-16T22:36:17.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f40a4ff2700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 0x7f407c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.696+0000 7f40a4ff2700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 0x7f407c057520 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f4090001890 tx=0x7f4090009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.708+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4098045a60 con 0x7f40a00ce030 2020-11-16T22:36:17.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40a0061100 con 0x7f40a00ce030 2020-11-16T22:36:17.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f409cff9700 1 -- 172.21.15.181:0/2734871662 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f4098022020 con 0x7f40a00ce030 2020-11-16T22:36:17.901 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:17.901 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":122,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:02.942140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:18.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 msgr2=0x7f407c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 0x7f407c057520 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f4090001890 tx=0x7f4090009040).stop 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 msgr2=0x7f40a015ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 0x7f40a015ae80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f4098009c90 tx=0x7f4098016000).stop 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 shutdown_connections 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f407c055060 0x7f407c057520 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40a00ce030 0x7f40a015ae80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a00b6a60 0x7f40a00cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 --2- 172.21.15.181:0/2734871662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a00b6e40 0x7f40a00cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.912+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 >> 172.21.15.181:0/2734871662 conn(0x7f40a0060bc0 msgr2=0x7f40a0065990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:18.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.916+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 shutdown_connections 2020-11-16T22:36:18.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:17.916+0000 7f40a7256700 1 -- 172.21.15.181:0/2734871662 wait complete. 2020-11-16T22:36:18.105 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 122 2020-11-16T22:36:18.106 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:36:18.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.252+0000 7fa77a7fc700 1 Processor -- start 2020-11-16T22:36:18.243 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.252+0000 7fa77a7fc700 1 -- start start 2020-11-16T22:36:18.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa774161860 0x7fa774062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.245 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 0x7fa7740644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740c8db0 con 0x7fa774161480 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa774165030 con 0x7fa774062740 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740c8330 con 0x7fa774161860 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa774161860 0x7fa774062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa774161860 0x7fa774062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37398/0 (socket says 172.21.15.181:37398) 2020-11-16T22:36:18.246 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 -- 172.21.15.181:0/4206552878 learned_addr learned my addr 172.21.15.181:0/4206552878 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa7797fa700 1 --1- 172.21.15.181:0/4206552878 >> v1:172.21.15.181:6789/0 conn(0x7fa774161480 0x7fa774164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51704/0 (socket says 172.21.15.181:51704) 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa779ffb700 1 --2- 172.21.15.181:0/4206552878 >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 0x7fa7740644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 -- 172.21.15.181:0/4206552878 >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 msgr2=0x7fa7740644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 --2- 172.21.15.181:0/4206552878 >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 0x7fa7740644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 -- 172.21.15.181:0/4206552878 >> v1:172.21.15.181:6789/0 conn(0x7fa774161480 legacy=0x7fa774164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.247 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 -- 172.21.15.181:0/4206552878 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740c85d0 con 0x7fa774161860 2020-11-16T22:36:18.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa779ffb700 1 --2- 172.21.15.181:0/4206552878 >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 0x7fa7740644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:18.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa778ff9700 1 --2- 172.21.15.181:0/4206552878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774161860 0x7fa774062250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fa7740ced60 tx=0x7fa768006eb0).ready entity=mon.1 client_cookie=9dd78ea7ee51a203 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:18.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa773fff700 1 -- 172.21.15.181:0/4206552878 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680098d0 con 0x7fa774161860 2020-11-16T22:36:18.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa773fff700 1 -- 172.21.15.181:0/4206552878 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768009a30 con 0x7fa774161860 2020-11-16T22:36:18.248 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa773fff700 1 -- 172.21.15.181:0/4206552878 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768009db0 con 0x7fa774161860 2020-11-16T22:36:18.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- 172.21.15.181:0/4206552878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774161860 msgr2=0x7fa774062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/4206552878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774161860 0x7fa774062250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fa7740ced60 tx=0x7fa768006eb0).stop 2020-11-16T22:36:18.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- 172.21.15.181:0/4206552878 shutdown_connections 2020-11-16T22:36:18.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/4206552878 >> v2:172.21.15.175:3300/0 conn(0x7fa774062740 0x7fa7740644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/4206552878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774161860 0x7fa774062250 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- 172.21.15.181:0/4206552878 >> 172.21.15.181:0/4206552878 conn(0x7fa7740d2dc0 msgr2=0x7fa7740ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:18.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- 172.21.15.181:0/4206552878 shutdown_connections 2020-11-16T22:36:18.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.256+0000 7fa77a7fc700 1 -- 172.21.15.181:0/4206552878 wait complete. 2020-11-16T22:36:18.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 Processor -- start 2020-11-16T22:36:18.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- start start 2020-11-16T22:36:18.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774062740 0x7fa7740b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 0x7fa774134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b5a60 con 0x7fa774161480 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b5d00 con 0x7fa774062740 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b5fa0 con 0x7fa774161860 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45250/0 (socket says 172.21.15.181:45250) 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa7797fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774062740 0x7fa7740b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 -- 172.21.15.181:0/1335625630 learned_addr learned my addr 172.21.15.181:0/1335625630 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa778ff9700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 0x7fa774134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 -- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774062740 msgr2=0x7fa7740b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774062740 0x7fa7740b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 -- 172.21.15.181:0/1335625630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 msgr2=0x7fa774134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 0x7fa774134970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 -- 172.21.15.181:0/1335625630 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740b6240 con 0x7fa774161860 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa778ff9700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 0x7fa774134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa779ffb700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fa76c00cbb0 tx=0x7fa76c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa7717fa700 1 -- 172.21.15.181:0/1335625630 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0078a0 con 0x7fa774161860 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa7717fa700 1 -- 172.21.15.181:0/1335625630 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c007a00 con 0x7fa774161860 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7740b64e0 con 0x7fa774161860 2020-11-16T22:36:18.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa7717fa700 1 -- 172.21.15.181:0/1335625630 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c007d80 con 0x7fa774161860 2020-11-16T22:36:18.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.260+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7740b5770 con 0x7fa774161860 2020-11-16T22:36:18.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.264+0000 7fa7717fa700 1 -- 172.21.15.181:0/1335625630 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa76c011620 con 0x7fa774161860 2020-11-16T22:36:18.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.264+0000 7fa7717fa700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 0x7fa74c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.264+0000 7fa7717fa700 1 -- 172.21.15.181:0/1335625630 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa76c06e290 con 0x7fa774161860 2020-11-16T22:36:18.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.264+0000 7fa7797fa700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 0x7fa74c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.264+0000 7fa7797fa700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 0x7fa74c057520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa7740c91e0 tx=0x7fa764000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:18.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 Processor -- start 2020-11-16T22:36:18.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 -- start start 2020-11-16T22:36:18.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 0x1261720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125efd0 0x1258700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ea4b0 con 0x10407b0 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x119e930 con 0xfd2bb0 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x105e6f0 con 0x125efd0 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa779ffb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125efd0 0x1258700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37414/0 (socket says 172.21.15.181:37414) 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.268+0000 7fa778ff9700 1 -- 172.21.15.181:0/104558118 learned_addr learned my addr 172.21.15.181:0/104558118 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7797fa700 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 0x1261720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 -- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125efd0 msgr2=0x1258700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125efd0 0x1258700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 -- 172.21.15.181:0/104558118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 msgr2=0x1261720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:18.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 0x1261720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 -- 172.21.15.181:0/104558118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10595a0 con 0xfd2bb0 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7797fa700 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 0x1261720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7fa768010000 tx=0x7fa768015000).ready entity=mon.1 client_cookie=57745979d16cab92 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7527fc700 1 -- 172.21.15.181:0/104558118 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680098d0 con 0xfd2bb0 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7527fc700 1 -- 172.21.15.181:0/104558118 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768006920 con 0xfd2bb0 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7527fc700 1 -- 172.21.15.181:0/104558118 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768011610 con 0xfd2bb0 2020-11-16T22:36:18.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 msgr2=0x125f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7fa768010000 tx=0x7fa768015000).stop 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- 172.21.15.181:0/104558118 shutdown_connections 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x125efd0 0x1258700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x10407b0 0x1261720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/104558118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x125f670 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- 172.21.15.181:0/104558118 >> 172.21.15.181:0/104558118 conn(0x120ee40 msgr2=0x125b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:18.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- 172.21.15.181:0/104558118 shutdown_connections 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- 172.21.15.181:0/104558118 wait complete. 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- start start 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 Processor -- start 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x12adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 0x128f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x118b6e0 con 0x125efd0 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x118b820 con 0xfd2bb0 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x118ca90 con 0x10407b0 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45262/0 (socket says 172.21.15.181:45262) 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa778ff9700 1 -- 172.21.15.181:0/3567978996 learned_addr learned my addr 172.21.15.181:0/3567978996 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa7797fa700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x12adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.272+0000 7fa78a4ef740 10 client.0 ll_register_callbacks cb 0x1181ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:18.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa779ffb700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 0x128f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 -- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 msgr2=0x12adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:18.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x12adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 -- 172.21.15.181:0/3567978996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 msgr2=0x128f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:18.878 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 0x128f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 -- 172.21.15.181:0/3567978996 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x117c280 con 0x10407b0 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa779ffb700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 0x128f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7797fa700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x12adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa778ff9700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa76800b1b0 tx=0x7fa768020000).ready entity=mon.2 client_cookie=6327a35f72108469 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768011610 con 0x10407b0 2020-11-16T22:36:18.879 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa768028670 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x123f6c0 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa768028a40 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 10 client.7751 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0xfb1fd0 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x123f6c0 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fa768032070 con 0x10407b0 2020-11-16T22:36:18.880 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 10 client.7751 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 10 client.7751 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 20 client.7751 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x12b1880 0x1280ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 4 client.7751 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x107cc50 con 0x12b1880 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7797fa700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x12b1880 0x1280ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 10 client.7751 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:18.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7797fa700 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x12b1880 0x1280ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a5936ee34e5eeda5 server_cookie=80e35892980ce199 in_seq=0 out_seq=0 2020-11-16T22:36:18.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 1 -- 172.21.15.181:0/3567978996 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x107cc50 con 0x12b1880 2020-11-16T22:36:18.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 10 client.7751 handle_command_reply: tid=0 2020-11-16T22:36:18.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 client.7751 shutdown 2020-11-16T22:36:18.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 msgr2=0x1259630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fa76800b1b0 tx=0x7fa768020000).stop 2020-11-16T22:36:19.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 shutdown_connections 2020-11-16T22:36:19.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x12b1880 0x1280ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x125efd0 0x128f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfd2bb0 0x12adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/3567978996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x10407b0 0x1259630 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa7437fe700 0 client.7751 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 >> 172.21.15.181:0/3567978996 conn(0x120ee40 msgr2=0x12948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 shutdown_connections 2020-11-16T22:36:19.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 1 -- 172.21.15.181:0/3567978996 wait complete. 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.276+0000 7fa78a4ef740 20 client.7751 trim_cache size 0 max 16384 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.296+0000 7fa78a4ef740 1 Processor -- start 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.296+0000 7fa78a4ef740 1 -- start start 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x12ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x1294900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1290070 con 0x120ee40 2020-11-16T22:36:19.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1163e40 con 0x10407b0 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12943b0 con 0xfd2bb0 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x1294900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37430/0 (socket says 172.21.15.181:37430) 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 -- 172.21.15.181:0/2891544917 learned_addr learned my addr 172.21.15.181:0/2891544917 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa7797fa700 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x12ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 -- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 msgr2=0x1294900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x1294900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 -- 172.21.15.181:0/2891544917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 msgr2=0x12ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x12ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 -- 172.21.15.181:0/2891544917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa748008200 con 0x10407b0 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa779ffb700 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fa76c0446b0 tx=0x7fa76c06e040).ready entity=mon.1 client_cookie=e6e8157e02196f69 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa752ffd700 1 -- 172.21.15.181:0/2891544917 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c046620 con 0x10407b0 2020-11-16T22:36:19.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa752ffd700 1 -- 172.21.15.181:0/2891544917 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c02a070 con 0x10407b0 2020-11-16T22:36:19.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa752ffd700 1 -- 172.21.15.181:0/2891544917 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0478b0 con 0x10407b0 2020-11-16T22:36:19.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 msgr2=0x1291f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fa76c0446b0 tx=0x7fa76c06e040).stop 2020-11-16T22:36:19.294 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2891544917 shutdown_connections 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x1291f50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x12ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2891544917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x1294900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2891544917 >> 172.21.15.181:0/2891544917 conn(0x7fa748002320 msgr2=0x125ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2891544917 shutdown_connections 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2891544917 wait complete. 2020-11-16T22:36:19.295 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 -- start start 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.300+0000 7fa78a4ef740 1 Processor -- start 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x125e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x128ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x12cebf0 con 0x120ee40 2020-11-16T22:36:19.296 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x12ab7d0 con 0x10407b0 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12cede0 con 0xfd2bb0 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x125e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 10 client.0 ll_register_callbacks cb 0x125f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa778ff9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x125e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37434/0 (socket says 172.21.15.181:37434) 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45274/0 (socket says 172.21.15.181:45274) 2020-11-16T22:36:19.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa778ff9700 1 -- 172.21.15.181:0/2924783311 learned_addr learned my addr 172.21.15.181:0/2924783311 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa779ffb700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x128ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 -- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 msgr2=0x125e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x125e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 -- 172.21.15.181:0/2924783311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 msgr2=0x128ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x128ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 -- 172.21.15.181:0/2924783311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xff0720 con 0xfd2bb0 2020-11-16T22:36:19.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa779ffb700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x128ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:19.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7797fa700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x12ac860 tx=0x7fa764019040).ready entity=mon.2 client_cookie=e94a789e8b43bac0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:19.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:19.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76401c040 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa768006920 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa764021c40 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa764021e00 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 10 client.7757 fetch_fsmap learned FSMap version 122 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0xffbe00 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768006920 con 0xfd2bb0 2020-11-16T22:36:19.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 122) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fa764028020 con 0xfd2bb0 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 10 client.7757 fetch_fsmap finished waiting for FSMap version 122 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 10 client.7757 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 20 client.7757 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x125efd0 0x1291d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 4 client.7757 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1083f50 con 0x125efd0 2020-11-16T22:36:19.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa778ff9700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x125efd0 0x1291d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.304+0000 7fa7437fe700 10 client.7757 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa778ff9700 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x125efd0 0x1291d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d4462ca045227220 server_cookie=74d854a879dc724e in_seq=0 out_seq=0 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa7437fe700 1 -- 172.21.15.181:0/2924783311 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1083f50 con 0x125efd0 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa7437fe700 10 client.7757 handle_command_reply: tid=0 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 client.7757 shutdown 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 msgr2=0x12ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x12ac860 tx=0x7fa764019040).stop 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 shutdown_connections 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x125efd0 0x1291d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x120ee40 0x128ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xfd2bb0 0x12ab2e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa7437fe700 0 client.7757 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 --2- 172.21.15.181:0/2924783311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x10407b0 0x125e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 >> 172.21.15.181:0/2924783311 conn(0x7fa748002320 msgr2=0x12948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:19.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 shutdown_connections 2020-11-16T22:36:19.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 1 -- 172.21.15.181:0/2924783311 wait complete. 2020-11-16T22:36:19.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa78a4ef740 20 client.7757 trim_cache size 0 max 16384 2020-11-16T22:36:19.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 msgr2=0x7fa74c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 0x7fa74c057520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa7740c91e0 tx=0x7fa764000b80).stop 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 msgr2=0x7fa774134e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fa76c00cbb0 tx=0x7fa76c008040).stop 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 shutdown_connections 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa74c055060 0x7fa74c057520 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa774161860 0x7fa774134e60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa774062740 0x7fa7740b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 --2- 172.21.15.181:0/1335625630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774161480 0x7fa774134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 >> 172.21.15.181:0/1335625630 conn(0x7fa7740d2dc0 msgr2=0x7fa774064400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 shutdown_connections 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:18.308+0000 7fa77a7fc700 1 -- 172.21.15.181:0/1335625630 wait complete. 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:19.712 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:19.713 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:19.714 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:36:19.714 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:19.716 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:36:18.999+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:36:19.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 Processor -- start 2020-11-16T22:36:19.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 -- start start 2020-11-16T22:36:19.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 0x7f0f5c142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.846 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f5c160530 0x7f0f5c061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c062700 con 0x7f0f5c063620 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0629a0 con 0x7f0f5c142660 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61b68700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c062c40 con 0x7f0f5c160530 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f5c160530 0x7f0f5c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f5bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0f5c063620 0x7f0f5c063ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51744/0 (socket says 172.21.15.181:51744) 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f5bfff700 1 -- 172.21.15.181:0/2126161576 learned_addr learned my addr 172.21.15.181:0/2126161576 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:19.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f60b66700 1 --2- 172.21.15.181:0/2126161576 >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 0x7f0f5c142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 -- 172.21.15.181:0/2126161576 >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 msgr2=0x7f0f5c142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 --2- 172.21.15.181:0/2126161576 >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 0x7f0f5c142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 -- 172.21.15.181:0/2126161576 >> v1:172.21.15.181:6789/0 conn(0x7f0f5c063620 legacy=0x7f0f5c063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 -- 172.21.15.181:0/2126161576 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c062ee0 con 0x7f0f5c160530 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f60b66700 1 --2- 172.21.15.181:0/2126161576 >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 0x7f0f5c142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f61367700 1 --2- 172.21.15.181:0/2126161576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c061810 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f0f5c146160 tx=0x7f0f54006df0).ready entity=mon.1 client_cookie=99d35b1d9ebb1a01 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f5b7fe700 1 -- 172.21.15.181:0/2126161576 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54009840 con 0x7f0f5c160530 2020-11-16T22:36:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f5b7fe700 1 -- 172.21.15.181:0/2126161576 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f540099a0 con 0x7f0f5c160530 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.856+0000 7f0f5b7fe700 1 -- 172.21.15.181:0/2126161576 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54009d20 con 0x7f0f5c160530 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- 172.21.15.181:0/2126161576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 msgr2=0x7f0f5c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2126161576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c061810 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f0f5c146160 tx=0x7f0f54006df0).stop 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- 172.21.15.181:0/2126161576 shutdown_connections 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2126161576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c061810 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2126161576 >> v2:172.21.15.175:3300/0 conn(0x7f0f5c142660 0x7f0f5c142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- 172.21.15.181:0/2126161576 >> 172.21.15.181:0/2126161576 conn(0x7f0f5c0d2dc0 msgr2=0x7f0f5c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- 172.21.15.181:0/2126161576 shutdown_connections 2020-11-16T22:36:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- 172.21.15.181:0/2126161576 wait complete. 2020-11-16T22:36:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 Processor -- start 2020-11-16T22:36:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- start start 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c142660 0x7f0f5c1728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c172da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15d660 con 0x7f0f5c142660 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c143150 con 0x7f0f5c160530 2020-11-16T22:36:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.860+0000 7f0f61b68700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c1433f0 con 0x7f0f5c063620 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f61367700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c172da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45286/0 (socket says 172.21.15.181:45286) 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 -- 172.21.15.181:0/2921696785 learned_addr learned my addr 172.21.15.181:0/2921696785 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f5bfff700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c142660 0x7f0f5c1728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 -- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 msgr2=0x7f0f5c172da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c172da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 -- 172.21.15.181:0/2921696785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c142660 msgr2=0x7f0f5c1728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c142660 0x7f0f5c1728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 -- 172.21.15.181:0/2921696785 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c143690 con 0x7f0f5c063620 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f60b66700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f0f4c00cc20 tx=0x7f0f4c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00a860 con 0x7f0f5c063620 2020-11-16T22:36:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c00a9c0 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c143930 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00ad40 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c0d0b20 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c044f60 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0f4c011620 con 0x7f0f5c063620 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 0x7f0f38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f5bfff700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 0x7f0f38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.864+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0f4c06e270 con 0x7f0f5c063620 2020-11-16T22:36:20.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.868+0000 7f0f5bfff700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 0x7f0f38057520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f0f44007fd0 tx=0x7f0f44009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:19.872+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0f4c0443f0 con 0x7f0f5c063620 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f5c05d550 con 0x7f0f5c063620 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f58ff9700 1 -- 172.21.15.181:0/2921696785 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4671 (secure 0 0 0) 0x7f0f4c047a10 con 0x7f0f5c063620 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 msgr2=0x7f0f38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 0x7f0f38057520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f0f44007fd0 tx=0x7f0f44009040).stop 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 msgr2=0x7f0f5c0d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f0f4c00cc20 tx=0x7f0f4c008040).stop 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 shutdown_connections 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0f38055060 0x7f0f38057520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5c160530 0x7f0f5c172da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f5c063620 0x7f0f5c0d0630 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 --2- 172.21.15.181:0/2921696785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c142660 0x7f0f5c1728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 >> 172.21.15.181:0/2921696785 conn(0x7f0f5c0d2dc0 msgr2=0x7f0f5c13eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:20.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 shutdown_connections 2020-11-16T22:36:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.076+0000 7f0f61b68700 1 -- 172.21.15.181:0/2921696785 wait complete. 2020-11-16T22:36:20.266 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 123 2020-11-16T22:36:20.266 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:20.266 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":123,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:19.002434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:20.267 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'i', 'e'] 2020-11-16T22:36:20.267 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:20.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 Processor -- start 2020-11-16T22:36:20.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 -- start start 2020-11-16T22:36:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6b415f5b0 0x7fa6b40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6b40d1830 0x7fa6b40629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b4160e40 con 0x7fa6b40615f0 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b41609a0 con 0x7fa6b415f5b0 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6bb8e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b4160ba0 con 0x7fa6b40d1830 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6b9e84700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6b40d1830 0x7fa6b40629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6b9e84700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6b40d1830 0x7fa6b40629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37456/0 (socket says 172.21.15.181:37456) 2020-11-16T22:36:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6b8e82700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa6b40615f0 0x7fa6b40622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51756/0 (socket says 172.21.15.181:51756) 2020-11-16T22:36:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6b9e84700 1 -- 172.21.15.181:0/3688787761 learned_addr learned my addr 172.21.15.181:0/3688787761 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.404+0000 7fa6b9683700 1 --2- 172.21.15.181:0/3688787761 >> v2:172.21.15.175:3300/0 conn(0x7fa6b415f5b0 0x7fa6b40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6b9e84700 1 -- 172.21.15.181:0/3688787761 >> v2:172.21.15.175:3300/0 conn(0x7fa6b415f5b0 msgr2=0x7fa6b40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6b9e84700 1 --2- 172.21.15.181:0/3688787761 >> v2:172.21.15.175:3300/0 conn(0x7fa6b415f5b0 0x7fa6b40d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6abfff700 1 -- 172.21.15.181:0/3688787761 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520819492 0 0) 0x7fa6b4160e40 con 0x7fa6b40615f0 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6b9e84700 1 -- 172.21.15.181:0/3688787761 >> v1:172.21.15.181:6789/0 conn(0x7fa6b40615f0 legacy=0x7fa6b40622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6b9e84700 1 -- 172.21.15.181:0/3688787761 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4060970 con 0x7fa6b40d1830 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6b9e84700 1 --2- 172.21.15.181:0/3688787761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40d1830 0x7fa6b40629b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fa6b40d2ea0 tx=0x7fa6ac009290).ready entity=mon.1 client_cookie=d1f07472bb1bea2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6abfff700 1 -- 172.21.15.181:0/3688787761 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00bbc0 con 0x7fa6b40d1830 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6abfff700 1 -- 172.21.15.181:0/3688787761 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac00bd20 con 0x7fa6b40d1830 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6abfff700 1 -- 172.21.15.181:0/3688787761 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011610 con 0x7fa6b40d1830 2020-11-16T22:36:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/3688787761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40d1830 msgr2=0x7fa6b40629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/3688787761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40d1830 0x7fa6b40629b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fa6b40d2ea0 tx=0x7fa6ac009290).stop 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/3688787761 shutdown_connections 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/3688787761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40d1830 0x7fa6b40629b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/3688787761 >> v2:172.21.15.175:3300/0 conn(0x7fa6b415f5b0 0x7fa6b40d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/3688787761 >> 172.21.15.181:0/3688787761 conn(0x7fa6b40d69a0 msgr2=0x7fa6b40d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/3688787761 shutdown_connections 2020-11-16T22:36:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/3688787761 wait complete. 2020-11-16T22:36:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 Processor -- start 2020-11-16T22:36:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.408+0000 7fa6bb8e7700 1 -- start start 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 0x7fa6b4163840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b415f5b0 0x7fa6b40d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40d6230 con 0x7fa6b415f5b0 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b41727d0 con 0x7fa6b40615f0 2020-11-16T22:36:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4172a70 con 0x7fa6b40d1830 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b9683700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b9683700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37458/0 (socket says 172.21.15.181:37458) 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b9683700 1 -- 172.21.15.181:0/318449456 learned_addr learned my addr 172.21.15.181:0/318449456 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 0x7fa6b4163840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b9e84700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b415f5b0 0x7fa6b40d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 -- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 msgr2=0x7fa6b4081080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 -- 172.21.15.181:0/318449456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b415f5b0 msgr2=0x7fa6b40d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b415f5b0 0x7fa6b40d57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 -- 172.21.15.181:0/318449456 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4172d10 con 0x7fa6b40d1830 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b9683700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6b8e82700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 0x7fa6b4163840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa6b4061a50 tx=0x7fa6b0007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00072e0 con 0x7fa6b40d1830 2020-11-16T22:36:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0005530 con 0x7fa6b40d1830 2020-11-16T22:36:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b4172fb0 con 0x7fa6b40d1830 2020-11-16T22:36:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00058d0 con 0x7fa6b40d1830 2020-11-16T22:36:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b4173450 con 0x7fa6b40d1830 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.412+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b4173140 con 0x7fa6b40d1830 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.416+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa6b0010070 con 0x7fa6b40d1830 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.416+0000 7fa6a97fa700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 0x7fa68c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.416+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa6b006d500 con 0x7fa6b40d1830 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.416+0000 7fa6b9683700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 0x7fa68c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.416+0000 7fa6b9683700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 0x7fa68c057550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa6a4006fa0 tx=0x7fa6a4008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.420+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa6b0043650 con 0x7fa6b40d1830 2020-11-16T22:36:20.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa6b4069eb0 con 0x7fa6b40d1830 2020-11-16T22:36:20.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6a97fa700 1 -- 172.21.15.181:0/318449456 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2202 (secure 0 0 0) 0x7fa6b0046c70 con 0x7fa6b40d1830 2020-11-16T22:36:20.613 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:20.613 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":585,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":115880,"bytes_used":64602112,"bytes_avail":773029511168,"bytes_total":773094113280,"write_bytes_sec":2985,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":123,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"e","status":"up:active","gid":8040},{"filesystem_id":7,"rank":1,"name":"i","status":"up:active","gid":8043},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8101}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":36,"modified":"2020-11-16T22:36:18.853348+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2020-11-16T22:36:20.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 msgr2=0x7fa68c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 0x7fa68c057550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fa6a4006fa0 tx=0x7fa6a4008040).stop 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 msgr2=0x7fa6b4163840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 0x7fa6b4163840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7fa6b4061a50 tx=0x7fa6b0007a20).stop 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 shutdown_connections 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa68c055090 0x7fa68c057550 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b415f5b0 0x7fa6b40d57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b40615f0 0x7fa6b4081080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 --2- 172.21.15.181:0/318449456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40d1830 0x7fa6b4163840 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 >> 172.21.15.181:0/318449456 conn(0x7fa6b40d69a0 msgr2=0x7fa6b4061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 shutdown_connections 2020-11-16T22:36:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.624+0000 7fa6bb8e7700 1 -- 172.21.15.181:0/318449456 wait complete. 2020-11-16T22:36:20.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/d1/d2/d3]', 'e': 'paused paths [/d1/d2/d3]', 'i': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:36:20.628 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:20.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 Processor -- start 2020-11-16T22:36:20.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 -- start start 2020-11-16T22:36:20.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe71c061810 0x7fe71c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0cccd0 con 0x7fe71c15f1a0 2020-11-16T22:36:20.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0ccf70 con 0x7fe71c156a90 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe7211af700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c15e030 con 0x7fe71c061810 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe71c156a90 0x7fe71c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51770/0 (socket says 172.21.15.181:51770) 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71bfff700 1 -- 172.21.15.181:0/1526085384 learned_addr learned my addr 172.21.15.181:0/1526085384 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71b7fe700 1 --2- 172.21.15.181:0/1526085384 >> v2:172.21.15.181:3300/0 conn(0x7fe71c061810 0x7fe71c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214476420 0 0) 0x7fe71c0ccf70 con 0x7fe71c156a90 2020-11-16T22:36:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe708003240 con 0x7fe71c156a90 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 424139501 0 0) 0x7fe708003240 con 0x7fe71c156a90 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71b7fe700 1 -- 172.21.15.181:0/1526085384 >> v1:172.21.15.181:6789/0 conn(0x7fe71c156a90 legacy=0x7fe71c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71b7fe700 1 -- 172.21.15.181:0/1526085384 >> v1:172.21.15.175:6789/0 conn(0x7fe71c15f1a0 legacy=0x7fe71c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.760+0000 7fe71b7fe700 1 -- 172.21.15.181:0/1526085384 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c15e2d0 con 0x7fe71c061810 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe71b7fe700 1 --2- 172.21.15.181:0/1526085384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c061810 0x7fe71c15ecb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fe71000b9a0 tx=0x7fe710006df0).ready entity=mon.1 client_cookie=f5b23e763b807fea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c001860 con 0x7fe71c061810 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe710012460 con 0x7fe71c061810 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe71affd700 1 -- 172.21.15.181:0/1526085384 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7100127e0 con 0x7fe71c061810 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 -- 172.21.15.181:0/1526085384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c061810 msgr2=0x7fe71c15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 --2- 172.21.15.181:0/1526085384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c061810 0x7fe71c15ecb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fe71000b9a0 tx=0x7fe710006df0).stop 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 -- 172.21.15.181:0/1526085384 shutdown_connections 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 --2- 172.21.15.181:0/1526085384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c061810 0x7fe71c15ecb0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 -- 172.21.15.181:0/1526085384 >> 172.21.15.181:0/1526085384 conn(0x7fe71c0cf630 msgr2=0x7fe71c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 -- 172.21.15.181:0/1526085384 shutdown_connections 2020-11-16T22:36:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.764+0000 7fe7211af700 1 -- 172.21.15.181:0/1526085384 wait complete. 2020-11-16T22:36:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 Processor -- start 2020-11-16T22:36:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 -- start start 2020-11-16T22:36:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 0x7fe71c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 0x7fe71c146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 0x7fe71c0d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c170bf0 con 0x7fe71c061810 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c170e90 con 0x7fe71c15f1a0 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7211af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c171130 con 0x7fe71c156a90 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 0x7fe71c146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe7209ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 0x7fe71c0d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 0x7fe71c146a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45316/0 (socket says 172.21.15.181:45316) 2020-11-16T22:36:20.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe71b7fe700 1 -- 172.21.15.181:0/3786529856 learned_addr learned my addr 172.21.15.181:0/3786529856 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:20.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.768+0000 7fe71bfff700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 0x7fe71c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 -- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 msgr2=0x7fe71c146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 0x7fe71c146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 -- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 msgr2=0x7fe71c0d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 0x7fe71c0d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 -- 172.21.15.181:0/3786529856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c1713d0 con 0x7fe71c061810 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe7209ae700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 0x7fe71c0d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe71bfff700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 0x7fe71c1608c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fe70c000f40 tx=0x7fe70c001570).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00ce30 con 0x7fe71c061810 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c004a90 con 0x7fe71c061810 2020-11-16T22:36:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c1591a0 con 0x7fe71c061810 2020-11-16T22:36:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c004e50 con 0x7fe71c061810 2020-11-16T22:36:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c170720 con 0x7fe71c061810 2020-11-16T22:36:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c170a00 con 0x7fe71c061810 2020-11-16T22:36:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.772+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe70c01b060 con 0x7fe71c061810 2020-11-16T22:36:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.776+0000 7fe6fbfff700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 0x7fe6f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.776+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe70c06e030 con 0x7fe71c061810 2020-11-16T22:36:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.776+0000 7fe71b7fe700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 0x7fe6f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.776+0000 7fe71b7fe700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 0x7fe6f4057520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fe71c1488c0 tx=0x7fe710006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:20.772 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.784+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe70c044310 con 0x7fe71c061810 2020-11-16T22:36:20.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.988+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe71c044f60 con 0x7fe71c061810 2020-11-16T22:36:20.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe6fbfff700 1 -- 172.21.15.181:0/3786529856 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4672 (secure 0 0 0) 0x7fe70c035020 con 0x7fe71c061810 2020-11-16T22:36:20.979 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:20.980 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":124,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:20.821259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 msgr2=0x7fe6f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 0x7fe6f4057520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fe71c1488c0 tx=0x7fe710006df0).stop 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 msgr2=0x7fe71c1608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 0x7fe71c1608c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fe70c000f40 tx=0x7fe70c001570).stop 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 shutdown_connections 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe6f4055060 0x7fe6f4057520 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe71c15f1a0 0x7fe71c0d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c061810 0x7fe71c1608c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 --2- 172.21.15.181:0/3786529856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe71c156a90 0x7fe71c146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 >> 172.21.15.181:0/3786529856 conn(0x7fe71c0cf630 msgr2=0x7fe71c0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 shutdown_connections 2020-11-16T22:36:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:20.992+0000 7fe7211af700 1 -- 172.21.15.181:0/3786529856 wait complete. 2020-11-16T22:36:20.983 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 124 2020-11-16T22:36:20.994 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub abort 2020-11-16T22:36:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 Processor -- start 2020-11-16T22:36:21.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 -- start start 2020-11-16T22:36:21.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38f8145a30 0x7f38f8145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38f80814c0 0x7f38f80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f8061b50 con 0x7f38f8145a30 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80618b0 con 0x7f38f80814c0 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80cb930 con 0x7f38f81492c0 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38f80814c0 0x7f38f80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45326/0 (socket says 172.21.15.181:45326) 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38effff700 1 -- 172.21.15.181:0/2852927481 learned_addr learned my addr 172.21.15.181:0/2852927481 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f659c700 1 --2- 172.21.15.181:0/2852927481 >> v2:172.21.15.175:3300/0 conn(0x7f38f8145a30 0x7f38f8145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 -- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 msgr2=0x7f38f8080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 -- 172.21.15.181:0/2852927481 >> v2:172.21.15.175:3300/0 conn(0x7f38f8145a30 msgr2=0x7f38f8145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/2852927481 >> v2:172.21.15.175:3300/0 conn(0x7f38f8145a30 0x7f38f8145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38f6d9d700 1 -- 172.21.15.181:0/2852927481 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80cbbd0 con 0x7f38f80814c0 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.128+0000 7f38effff700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:21.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80814c0 0x7f38f80cc5b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f38f8082560 tx=0x7f38f00091d0).ready entity=mon.1 client_cookie=3c6ba3158b91e5fd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f5d9b700 1 -- 172.21.15.181:0/2852927481 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000b9c0 con 0x7f38f80814c0 2020-11-16T22:36:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f5d9b700 1 -- 172.21.15.181:0/2852927481 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000bb20 con 0x7f38f80814c0 2020-11-16T22:36:21.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f5d9b700 1 -- 172.21.15.181:0/2852927481 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000bea0 con 0x7f38f80814c0 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80814c0 msgr2=0x7f38f80cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80814c0 0x7f38f80cc5b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f38f8082560 tx=0x7f38f00091d0).stop 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- 172.21.15.181:0/2852927481 shutdown_connections 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80814c0 0x7f38f80cc5b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 --2- 172.21.15.181:0/2852927481 >> v2:172.21.15.175:3300/0 conn(0x7f38f8145a30 0x7f38f8145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 --2- 172.21.15.181:0/2852927481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f8080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- 172.21.15.181:0/2852927481 >> 172.21.15.181:0/2852927481 conn(0x7f38f80d68c0 msgr2=0x7f38f80d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- 172.21.15.181:0/2852927481 shutdown_connections 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- 172.21.15.181:0/2852927481 wait complete. 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 Processor -- start 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.132+0000 7f38f759e700 1 -- start start 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38f80814c0 0x7f38f80bb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f816f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f816fa20 con 0x7f38f80814c0 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f816fcc0 con 0x7f38f8145a30 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f81465f0 con 0x7f38f81492c0 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37486/0 (socket says 172.21.15.181:37486) 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 -- 172.21.15.181:0/4083644231 learned_addr learned my addr 172.21.15.181:0/4083644231 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f816f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f659c700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38f80814c0 0x7f38f80bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 -- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 msgr2=0x7f38f816f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f816f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 -- 172.21.15.181:0/4083644231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38f80814c0 msgr2=0x7f38f80bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38f80814c0 0x7f38f80bb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 -- 172.21.15.181:0/4083644231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8146890 con 0x7f38f8145a30 2020-11-16T22:36:21.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38effff700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f38e000ef20 tx=0x7f38e0006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 -- 172.21.15.181:0/4083644231 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000cbc0 con 0x7f38f8145a30 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 -- 172.21.15.181:0/4083644231 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e000cd20 con 0x7f38f8145a30 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f8146b30 con 0x7f38f8145a30 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 -- 172.21.15.181:0/4083644231 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0012600 con 0x7f38f8145a30 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80bbb10 con 0x7f38f8145a30 2020-11-16T22:36:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 -- 172.21.15.181:0/4083644231 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f38e0022070 con 0x7f38f8145a30 2020-11-16T22:36:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38ee7fc700 1 -- 172.21.15.181:0/4083644231 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38e006e3a0 con 0x7f38f8145a30 2020-11-16T22:36:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.136+0000 7f38f659c700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.140+0000 7f38f659c700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f38e8006fd0 tx=0x7f38e8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.140+0000 7f390b8a7740 1 Processor -- start 2020-11-16T22:36:21.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.140+0000 7f390b8a7740 1 -- start start 2020-11-16T22:36:21.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 0x1e69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 0x1e606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf24a0 con 0x1c487a0 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da6920 con 0x1e66fc0 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c666e0 con 0x1bdaba0 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 0x1e606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45338/0 (socket says 172.21.15.181:45338) 2020-11-16T22:36:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38effff700 1 -- 172.21.15.181:0/3455264911 learned_addr learned my addr 172.21.15.181:0/3455264911 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f659c700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 0x1e69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 -- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 msgr2=0x1e67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 -- 172.21.15.181:0/3455264911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 msgr2=0x1e69710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 0x1e69710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 -- 172.21.15.181:0/3455264911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c61590 con 0x1e66fc0 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38effff700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f659c700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 0x1e69710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 0x1e606f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f38f8082560 tx=0x7f38f0016000).ready entity=mon.1 client_cookie=59836c9aa0921839 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38cf7fe700 1 -- 172.21.15.181:0/3455264911 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00099e0 con 0x1e66fc0 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38cf7fe700 1 -- 172.21.15.181:0/3455264911 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000bea0 con 0x1e66fc0 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f38cf7fe700 1 -- 172.21.15.181:0/3455264911 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011820 con 0x1e66fc0 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 msgr2=0x1e606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 0x1e606f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f38f8082560 tx=0x7f38f0016000).stop 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- 172.21.15.181:0/3455264911 shutdown_connections 2020-11-16T22:36:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1e66fc0 0x1e606f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1c487a0 0x1e69710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 --2- 172.21.15.181:0/3455264911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1bdaba0 0x1e67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- 172.21.15.181:0/3455264911 >> 172.21.15.181:0/3455264911 conn(0x1e16e30 msgr2=0x1e63390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- 172.21.15.181:0/3455264911 shutdown_connections 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.144+0000 7f390b8a7740 1 -- 172.21.15.181:0/3455264911 wait complete. 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- start start 2020-11-16T22:36:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 Processor -- start 2020-11-16T22:36:21.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.141 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e66fc0 0x1e630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d936d0 con 0x1bdaba0 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d93810 con 0x1c487a0 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d94a80 con 0x1e66fc0 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 10 client.0 ll_register_callbacks cb 0x1d89ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37500/0 (socket says 172.21.15.181:37500) 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38f6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e66fc0 0x1e630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 -- 172.21.15.181:0/72347153 learned_addr learned my addr 172.21.15.181:0/72347153 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38f659c700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 -- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e66fc0 msgr2=0x1e630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e66fc0 0x1e630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 -- 172.21.15.181:0/72347153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 msgr2=0x1e66670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e66670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 -- 172.21.15.181:0/72347153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d84270 con 0x1c487a0 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38effff700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f38e0047fc0 tx=0x7f38e006e040).ready entity=mon.1 client_cookie=a5e8e69d2357b7c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0044ba0 con 0x1c487a0 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e476b0 con 0x1c487a0 2020-11-16T22:36:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38e00479c0 con 0x1c487a0 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e0030e80 con 0x1c487a0 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 10 client.8629 fetch_fsmap learned FSMap version 124 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x1bb9fc0 con 0x1c487a0 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e476b0 con 0x1c487a0 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 124) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f38e006cb70 con 0x1c487a0 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 10 client.8629 fetch_fsmap finished waiting for FSMap version 124 2020-11-16T22:36:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 10 client.8629 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 20 client.8629 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1ec7e50 0x1e68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 4 client.8629 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1c84c40 con 0x1ec7e50 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38f659c700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1ec7e50 0x1e68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38ccff9700 10 client.8629 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.148+0000 7f38f659c700 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1ec7e50 0x1e68a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=92717341e2c878d6 server_cookie=22c99384b4c218c8 in_seq=0 out_seq=0 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f38ccff9700 1 -- 172.21.15.181:0/72347153 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1c84c40 con 0x1ec7e50 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f38ccff9700 10 client.8629 handle_command_reply: tid=0 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 client.8629 shutdown 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 msgr2=0x1e89a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f38e0047fc0 tx=0x7f38e006e040).stop 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 shutdown_connections 2020-11-16T22:36:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1ec7e50 0x1e68a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e66fc0 0x1e630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e66670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f38ccff9700 0 client.8629 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 --2- 172.21.15.181:0/72347153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e89a50 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 >> 172.21.15.181:0/72347153 conn(0x1e16e30 msgr2=0x1e9c980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 shutdown_connections 2020-11-16T22:36:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 1 -- 172.21.15.181:0/72347153 wait complete. 2020-11-16T22:36:21.149 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.152+0000 7f390b8a7740 20 client.8629 trim_cache size 0 max 16384 2020-11-16T22:36:21.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 Processor -- start 2020-11-16T22:36:21.158 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 -- start start 2020-11-16T22:36:21.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e16e30 0x1e62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d6be30 con 0x1bdaba0 2020-11-16T22:36:21.159 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e62550 con 0x1c487a0 2020-11-16T22:36:21.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.168+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e5c470 con 0x1e16e30 2020-11-16T22:36:21.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e16e30 0x1e62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37510/0 (socket says 172.21.15.181:37510) 2020-11-16T22:36:21.160 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 -- 172.21.15.181:0/111691624 learned_addr learned my addr 172.21.15.181:0/111691624 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38effff700 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 -- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e16e30 msgr2=0x1e62bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e16e30 0x1e62bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 -- 172.21.15.181:0/111691624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 msgr2=0x1e630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.162 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 -- 172.21.15.181:0/111691624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00081d0 con 0x1c487a0 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f38f80d43d0 tx=0x7f38f001f040).ready entity=mon.1 client_cookie=622b9d9b32438cbf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38cffff700 1 -- 172.21.15.181:0/111691624 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011b90 con 0x1c487a0 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38cffff700 1 -- 172.21.15.181:0/111691624 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f002c040 con 0x1c487a0 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f38cffff700 1 -- 172.21.15.181:0/111691624 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011400 con 0x1c487a0 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 msgr2=0x1e5b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f38f80d43d0 tx=0x7f38f001f040).stop 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- 172.21.15.181:0/111691624 shutdown_connections 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1e16e30 0x1e62bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 --2- 172.21.15.181:0/111691624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1bdaba0 0x1e630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- 172.21.15.181:0/111691624 >> 172.21.15.181:0/111691624 conn(0x7f38c0002320 msgr2=0x1e99830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.164 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- 172.21.15.181:0/111691624 shutdown_connections 2020-11-16T22:36:21.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- 172.21.15.181:0/111691624 wait complete. 2020-11-16T22:36:21.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 -- start start 2020-11-16T22:36:21.165 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.172+0000 7f390b8a7740 1 Processor -- start 2020-11-16T22:36:21.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 0x1e5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f18730 con 0x1e16e30 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e9a2c0 con 0x1bdaba0 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e5c360 con 0x1c487a0 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 10 client.0 ll_register_callbacks cb 0x1e678b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37514/0 (socket says 172.21.15.181:37514) 2020-11-16T22:36:21.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45358/0 (socket says 172.21.15.181:45358) 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 -- 172.21.15.181:0/1682341233 learned_addr learned my addr 172.21.15.181:0/1682341233 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 0x1e5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 -- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 msgr2=0x1e5b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 -- 172.21.15.181:0/1682341233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 msgr2=0x1e5bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 0x1e5bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 -- 172.21.15.181:0/1682341233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1bf8710 con 0x1bdaba0 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38effff700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f6d9d700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 0x1e5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38f659c700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f38e8023a90 tx=0x7f38e8016040).ready entity=mon.1 client_cookie=8f4de09246704063 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8028070 con 0x1bdaba0 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38e8023e20 con 0x1bdaba0 2020-11-16T22:36:21.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38f000bea0 con 0x1bdaba0 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e8011800 con 0x1bdaba0 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 10 client.8635 fetch_fsmap learned FSMap version 124 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x1c03df0 con 0x1bdaba0 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000bea0 con 0x1bdaba0 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 124) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f38e8011960 con 0x1bdaba0 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 10 client.8635 fetch_fsmap finished waiting for FSMap version 124 2020-11-16T22:36:21.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 10 client.8635 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 20 client.8635 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1e66fc0 0x1ebe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 4 client.8635 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,a,b,o,r,t,",}] 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b a b o r t " }) v1 -- 0x1c8bf40 con 0x1e66fc0 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38effff700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1e66fc0 0x1ebe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38ccff9700 10 client.8635 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:21.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.176+0000 7f38effff700 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1e66fc0 0x1ebe060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bbdf00ccebe1f028 server_cookie=4ab357f5dddba255 in_seq=0 out_seq=0 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38ccff9700 1 -- 172.21.15.181:0/1682341233 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x1c8bf40 con 0x1e66fc0 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38ccff9700 10 client.8635 handle_command_reply: tid=0 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 client.8635 shutdown 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 msgr2=0x1f17ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f38e8023a90 tx=0x7f38e8016040).stop 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 shutdown_connections 2020-11-16T22:36:21.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1e66fc0 0x1ebe060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1e16e30 0x1e5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38ccff9700 0 client.8635 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1bdaba0 0x1f17ef0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 --2- 172.21.15.181:0/1682341233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1c487a0 0x1e5b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 >> 172.21.15.181:0/1682341233 conn(0x7f38c0002320 msgr2=0x1eb0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 shutdown_connections 2020-11-16T22:36:21.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 1 -- 172.21.15.181:0/1682341233 wait complete. 2020-11-16T22:36:21.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f390b8a7740 20 client.8635 trim_cache size 0 max 16384 2020-11-16T22:36:21.176 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:21.176 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:21.176 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 msgr2=0x7f38d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f38e8006fd0 tx=0x7f38e8006df0).stop 2020-11-16T22:36:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 msgr2=0x7f38f816ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f38e000ef20 tx=0x7f38e0006eb0).stop 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 shutdown_connections 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f81492c0 0x7f38f816f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38f80814c0 0x7f38f80bb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 --2- 172.21.15.181:0/4083644231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f8145a30 0x7f38f816ef20 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 >> 172.21.15.181:0/4083644231 conn(0x7f38f80d68c0 msgr2=0x7f38f80d6080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 shutdown_connections 2020-11-16T22:36:21.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.180+0000 7f38f759e700 1 -- 172.21.15.181:0/4083644231 wait complete. 2020-11-16T22:36:21.186 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:21.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.316+0000 7fe812fc6700 1 Processor -- start 2020-11-16T22:36:21.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.316+0000 7fe812fc6700 1 -- start start 2020-11-16T22:36:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80c0cc7d0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80c15eb10 con 0x7fe80c157a80 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c15f0b0 con 0x7fe80c061100 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe811fc4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe80c157a80 0x7fe80c0cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51828/0 (socket says 172.21.15.181:51828) 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45364/0 (socket says 172.21.15.181:45364) 2020-11-16T22:36:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe811fc4700 1 -- 172.21.15.181:0/2065844738 learned_addr learned my addr 172.21.15.181:0/2065844738 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 -- 172.21.15.181:0/2065844738 >> v1:172.21.15.181:6789/0 conn(0x7fe80c157a80 legacy=0x7fe80c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 -- 172.21.15.181:0/2065844738 >> v1:172.21.15.175:6789/0 conn(0x7fe80c15dfd0 legacy=0x7fe80c15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe810fc2700 1 -- 172.21.15.181:0/2065844738 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724283919 0 0) 0x7fe80c15eb10 con 0x7fe80c157a80 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 -- 172.21.15.181:0/2065844738 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c1611b0 con 0x7fe80c061100 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe8117c3700 1 --2- 172.21.15.181:0/2065844738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fe7fc00b9d0 tx=0x7fe7fc006df0).ready entity=mon.2 client_cookie=1755469d61c95b9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe810fc2700 1 -- 172.21.15.181:0/2065844738 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009770 con 0x7fe80c061100 2020-11-16T22:36:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe810fc2700 1 -- 172.21.15.181:0/2065844738 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0098d0 con 0x7fe80c061100 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe810fc2700 1 -- 172.21.15.181:0/2065844738 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009c50 con 0x7fe80c061100 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- 172.21.15.181:0/2065844738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 msgr2=0x7fe80c061580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2065844738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fe7fc00b9d0 tx=0x7fe7fc006df0).stop 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- 172.21.15.181:0/2065844738 shutdown_connections 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2065844738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c061100 0x7fe80c061580 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- 172.21.15.181:0/2065844738 >> 172.21.15.181:0/2065844738 conn(0x7fe80c0cf630 msgr2=0x7fe80c0cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- 172.21.15.181:0/2065844738 shutdown_connections 2020-11-16T22:36:21.313 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.320+0000 7fe812fc6700 1 -- 172.21.15.181:0/2065844738 wait complete. 2020-11-16T22:36:21.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 Processor -- start 2020-11-16T22:36:21.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- start start 2020-11-16T22:36:21.314 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 0x7fe80c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe80c157a80 0x7fe80c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0c7c00 con 0x7fe80c157a80 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c1701f0 con 0x7fe80c061100 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c16fcb0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe811fc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 0x7fe80c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.315 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe811fc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 0x7fe80c0b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37528/0 (socket says 172.21.15.181:37528) 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45374/0 (socket says 172.21.15.181:45374) 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe811fc4700 1 -- 172.21.15.181:0/2899518159 learned_addr learned my addr 172.21.15.181:0/2899518159 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8117c3700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe80c157a80 0x7fe80c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 -- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 msgr2=0x7fe80c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.316 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 0x7fe80c0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 -- 172.21.15.181:0/2899518159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe80c157a80 msgr2=0x7fe80c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe80c157a80 0x7fe80c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 -- 172.21.15.181:0/2899518159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c16ff50 con 0x7fe80c15dfd0 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe8127c5700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fe80400cae0 tx=0x7fe804007dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400a6d0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80400a830 con 0x7fe80c15dfd0 2020-11-16T22:36:21.317 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c0c7e60 con 0x7fe80c15dfd0 2020-11-16T22:36:21.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400abf0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.324+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c0c8cb0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c0c82d0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe804010070 con 0x7fe80c15dfd0 2020-11-16T22:36:21.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe7f9ffb700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 0x7fe7e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe80406dbf0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe811fc4700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 0x7fe7e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.319 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.328+0000 7fe811fc4700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 0x7fe7e4057520 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fe8080011a0 tx=0x7fe8080079e0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.324 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.336+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe804043d40 con 0x7fe80c15dfd0 2020-11-16T22:36:21.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.532+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe80c0c7ff0 con 0x7fe80c15dfd0 2020-11-16T22:36:21.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.532+0000 7fe7f9ffb700 1 -- 172.21.15.181:0/2899518159 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4672 (secure 0 0 0) 0x7fe804047360 con 0x7fe80c15dfd0 2020-11-16T22:36:21.523 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:21.523 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":124,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:20.821259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 msgr2=0x7fe7e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 0x7fe7e4057520 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fe8080011a0 tx=0x7fe8080079e0).stop 2020-11-16T22:36:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 msgr2=0x7fe80c16f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fe80400cae0 tx=0x7fe804007dc0).stop 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 shutdown_connections 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe7e4055060 0x7fe7e4057520 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80c15dfd0 0x7fe80c16f260 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c061100 0x7fe80c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 --2- 172.21.15.181:0/2899518159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe80c157a80 0x7fe80c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 >> 172.21.15.181:0/2899518159 conn(0x7fe80c0cf630 msgr2=0x7fe80c0614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 shutdown_connections 2020-11-16T22:36:21.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.536+0000 7fe812fc6700 1 -- 172.21.15.181:0/2899518159 wait complete. 2020-11-16T22:36:21.526 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 124 2020-11-16T22:36:21.537 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:36:21.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 Processor -- start 2020-11-16T22:36:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 -- start start 2020-11-16T22:36:21.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 0x7f67e40644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40c8db0 con 0x7f67e4161480 2020-11-16T22:36:21.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e4165030 con 0x7f67e4161860 2020-11-16T22:36:21.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e8c76700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40c8330 con 0x7f67e4062960 2020-11-16T22:36:21.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.672+0000 7f67e2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f67e4161860 0x7f67e4062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51842/0 (socket says 172.21.15.181:51842) 2020-11-16T22:36:21.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e2ffd700 1 -- 172.21.15.181:0/1727076037 learned_addr learned my addr 172.21.15.181:0/1727076037 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:21.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e3fff700 1 --2- 172.21.15.181:0/1727076037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 0x7f67e40644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750846944 0 0) 0x7f67e4165030 con 0x7f67e4161860 2020-11-16T22:36:21.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c8003240 con 0x7f67e4161860 2020-11-16T22:36:21.665 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573473058 0 0) 0x7f67e40c8db0 con 0x7f67e4161480 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e4165030 con 0x7f67e4161480 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e3fff700 1 -- 172.21.15.181:0/1727076037 >> v1:172.21.15.181:6789/0 conn(0x7f67e4161860 legacy=0x7f67e4062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e3fff700 1 -- 172.21.15.181:0/1727076037 >> v1:172.21.15.175:6789/0 conn(0x7f67e4161480 legacy=0x7f67e4164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e3fff700 1 -- 172.21.15.181:0/1727076037 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40c85d0 con 0x7f67e4062960 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e3fff700 1 --2- 172.21.15.181:0/1727076037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 0x7f67e40644e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f67dc00b9d0 tx=0x7f67dc006df0).ready entity=mon.2 client_cookie=68e33038935a8daa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc009680 con 0x7f67e4062960 2020-11-16T22:36:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc0097e0 con 0x7f67e4062960 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e27fc700 1 -- 172.21.15.181:0/1727076037 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc009b60 con 0x7f67e4062960 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 -- 172.21.15.181:0/1727076037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 msgr2=0x7f67e40644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1727076037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 0x7f67e40644e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f67dc00b9d0 tx=0x7f67dc006df0).stop 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 -- 172.21.15.181:0/1727076037 shutdown_connections 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1727076037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4062960 0x7f67e40644e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 -- 172.21.15.181:0/1727076037 >> 172.21.15.181:0/1727076037 conn(0x7f67e40d2dc0 msgr2=0x7f67e40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 -- 172.21.15.181:0/1727076037 shutdown_connections 2020-11-16T22:36:21.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.676+0000 7f67e8c76700 1 -- 172.21.15.181:0/1727076037 wait complete. 2020-11-16T22:36:21.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.680+0000 7f67e8c76700 1 Processor -- start 2020-11-16T22:36:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.680+0000 7f67e8c76700 1 -- start start 2020-11-16T22:36:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161480 0x7f67e40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 0x7f67e40b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4166e20 con 0x7f67e4161480 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4174410 con 0x7f67e4062960 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4177870 con 0x7f67e4161860 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37542/0 (socket says 172.21.15.181:37542) 2020-11-16T22:36:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 -- 172.21.15.181:0/1796698445 learned_addr learned my addr 172.21.15.181:0/1796698445 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e3fff700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 0x7f67e40b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161480 0x7f67e40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 -- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 msgr2=0x7f67e40b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 0x7f67e40b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 -- 172.21.15.181:0/1796698445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161480 msgr2=0x7f67e40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161480 0x7f67e40b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.674 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 -- 172.21.15.181:0/1796698445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4173a30 con 0x7f67e4062960 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e3fff700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 0x7f67e40b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e37fe700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f67cc00c710 tx=0x7f67cc007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67db7fe700 1 -- 172.21.15.181:0/1796698445 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00e490 con 0x7f67e4062960 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67db7fe700 1 -- 172.21.15.181:0/1796698445 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc00e5f0 con 0x7f67e4062960 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e4173cd0 con 0x7f67e4062960 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67db7fe700 1 -- 172.21.15.181:0/1796698445 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00e990 con 0x7f67e4062960 2020-11-16T22:36:21.675 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e4173500 con 0x7f67e4062960 2020-11-16T22:36:21.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.684+0000 7f67db7fe700 1 -- 172.21.15.181:0/1796698445 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f67cc019030 con 0x7f67e4062960 2020-11-16T22:36:21.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.688+0000 7f67db7fe700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 0x7f67b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.676 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.688+0000 7f67db7fe700 1 -- 172.21.15.181:0/1796698445 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67cc06d180 con 0x7f67e4062960 2020-11-16T22:36:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.688+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 0x7f67b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.688+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 0x7f67b4057520 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f67e40d21e0 tx=0x7f67d4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.688+0000 7f67f8969740 1 Processor -- start 2020-11-16T22:36:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- start start 2020-11-16T22:36:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f36720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f33fd0 0x1f2d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dbf4b0 con 0x1d157b0 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e73930 con 0x1ca7bb0 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d336f0 con 0x1f33fd0 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37550/0 (socket says 172.21.15.181:37550) 2020-11-16T22:36:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2922741776 learned_addr learned my addr 172.21.15.181:0/2922741776 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e3fff700 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f33fd0 0x1f2d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e37fe700 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f36720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f33fd0 msgr2=0x1f2d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f33fd0 0x1f2d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2922741776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 msgr2=0x1f36720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f36720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2922741776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d2e5a0 con 0x1ca7bb0 2020-11-16T22:36:21.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f67d4015ce0 tx=0x7f67d4011000).ready entity=mon.1 client_cookie=79bc94cb30a8ea71 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67d8ff9700 1 -- 172.21.15.181:0/2922741776 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d401a070 con 0x1ca7bb0 2020-11-16T22:36:21.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67d8ff9700 1 -- 172.21.15.181:0/2922741776 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d40129c0 con 0x1ca7bb0 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67d8ff9700 1 -- 172.21.15.181:0/2922741776 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4012d80 con 0x1ca7bb0 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 msgr2=0x1f34670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f67d4015ce0 tx=0x7f67d4011000).stop 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- 172.21.15.181:0/2922741776 shutdown_connections 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1f33fd0 0x1f2d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f36720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 --2- 172.21.15.181:0/2922741776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- 172.21.15.181:0/2922741776 >> 172.21.15.181:0/2922741776 conn(0x1ee3e40 msgr2=0x1f303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- 172.21.15.181:0/2922741776 shutdown_connections 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.692+0000 7f67f8969740 1 -- 172.21.15.181:0/2922741776 wait complete. 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- start start 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 Processor -- start 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 0x1f2e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e606e0 con 0x1f33fd0 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e60820 con 0x1ca7bb0 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e61a90 con 0x1d157b0 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 10 client.0 ll_register_callbacks cb 0x1e56ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45400/0 (socket says 172.21.15.181:45400) 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2176010393 learned_addr learned my addr 172.21.15.181:0/2176010393 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e3fff700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 0x1f2e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 msgr2=0x1f932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 msgr2=0x1f2e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 0x1f2e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 -- 172.21.15.181:0/2176010393 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e51280 con 0x1d157b0 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e3fff700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 0x1f2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f67d4006930 tx=0x7f67d4014000).ready entity=mon.2 client_cookie=51877264e0b21a6c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d401fd90 con 0x1d157b0 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67d4026040 con 0x1d157b0 2020-11-16T22:36:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f146c0 con 0x1d157b0 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d402be30 con 0x1d157b0 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 10 client.7802 fetch_fsmap learned FSMap version 124 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x1c86fd0 con 0x1d157b0 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f146c0 con 0x1d157b0 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 124) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f67d4028030 con 0x1d157b0 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 10 client.7802 fetch_fsmap finished waiting for FSMap version 124 2020-11-16T22:36:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 10 client.7802 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 20 client.7802 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f84370 0x1f32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 4 client.7802 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1d51c50 con 0x1f84370 2020-11-16T22:36:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.696+0000 7f67e37fe700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f84370 0x1f32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67d1ffb700 10 client.7802 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67e37fe700 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f84370 0x1f32db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=5ddcdabb25c69394 server_cookie=2548cce907e71fe5 in_seq=0 out_seq=0 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67d1ffb700 1 -- 172.21.15.181:0/2176010393 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d51c50 con 0x1f84370 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67d1ffb700 10 client.7802 handle_command_reply: tid=0 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 client.7802 shutdown 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 msgr2=0x1f516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f67d4006930 tx=0x7f67d4014000).stop 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 shutdown_connections 2020-11-16T22:36:21.695 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f84370 0x1f32db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1f33fd0 0x1f2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ca7bb0 0x1f932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67d1ffb700 0 client.7802 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 --2- 172.21.15.181:0/2176010393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1d157b0 0x1f516f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 >> 172.21.15.181:0/2176010393 conn(0x1ee3e40 msgr2=0x1f69a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 shutdown_connections 2020-11-16T22:36:21.696 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 1 -- 172.21.15.181:0/2176010393 wait complete. 2020-11-16T22:36:21.697 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.700+0000 7f67f8969740 20 client.7802 trim_cache size 0 max 16384 2020-11-16T22:36:21.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.716+0000 7f67f8969740 1 Processor -- start 2020-11-16T22:36:21.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.716+0000 7f67f8969740 1 -- start start 2020-11-16T22:36:21.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ee3e40 0x1f2e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 0x1f2fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e38e40 con 0x1ee3e40 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f823c0 con 0x1d157b0 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f7ed60 con 0x1ca7bb0 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 0x1f2fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45408/0 (socket says 172.21.15.181:45408) 2020-11-16T22:36:21.708 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e2ffd700 1 -- 172.21.15.181:0/3942114072 learned_addr learned my addr 172.21.15.181:0/3942114072 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.709 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e37fe700 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ee3e40 0x1f2e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 -- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 msgr2=0x1f34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 -- 172.21.15.181:0/3942114072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ee3e40 msgr2=0x1f2e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ee3e40 0x1f2e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 -- 172.21.15.181:0/3942114072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0081d0 con 0x1d157b0 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67e3fff700 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 0x1f2fbc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f67e4066500 tx=0x7f67dc015000).ready entity=mon.1 client_cookie=1bcbc9fbd144b502 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67d97fa700 1 -- 172.21.15.181:0/3942114072 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc024070 con 0x1d157b0 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67d97fa700 1 -- 172.21.15.181:0/3942114072 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc011860 con 0x1d157b0 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67d97fa700 1 -- 172.21.15.181:0/3942114072 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc009b60 con 0x1d157b0 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 msgr2=0x1f2fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 0x1f2fbc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f67e4066500 tx=0x7f67dc015000).stop 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- 172.21.15.181:0/3942114072 shutdown_connections 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d157b0 0x1f2fbc0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1ee3e40 0x1f2e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 --2- 172.21.15.181:0/3942114072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- 172.21.15.181:0/3942114072 >> 172.21.15.181:0/3942114072 conn(0x7f67ac002320 msgr2=0x1f7db10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- 172.21.15.181:0/3942114072 shutdown_connections 2020-11-16T22:36:21.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- 172.21.15.181:0/3942114072 wait complete. 2020-11-16T22:36:21.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 -- start start 2020-11-16T22:36:21.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.720+0000 7f67f8969740 1 Processor -- start 2020-11-16T22:36:21.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ee3e40 0x1f2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f85600 con 0x1d157b0 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fe9040 con 0x1ee3e40 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fe92e0 con 0x1ca7bb0 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45412/0 (socket says 172.21.15.181:45412) 2020-11-16T22:36:21.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 -- 172.21.15.181:0/936024881 learned_addr learned my addr 172.21.15.181:0/936024881 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 10 client.0 ll_register_callbacks cb 0x1f93830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:21.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e3fff700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ee3e40 0x1f2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 -- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ee3e40 msgr2=0x1f2fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ee3e40 0x1f2fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 -- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 msgr2=0x1f85110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f85110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 -- 172.21.15.181:0/936024881 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1cc5720 con 0x1ca7bb0 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e37fe700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f67cc06d120 tx=0x7f67cc014040).ready entity=mon.2 client_cookie=d5ce99a8065cce5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0280b0 con 0x1ca7bb0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67cc0467a0 con 0x1ca7bb0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d40129c0 con 0x1ca7bb0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67cc0430a0 con 0x1ca7bb0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 10 client.7808 fetch_fsmap learned FSMap version 124 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x1cd0e00 con 0x1ca7bb0 2020-11-16T22:36:21.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d40129c0 con 0x1ca7bb0 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 124) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f67cc046000 con 0x1ca7bb0 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 10 client.7808 fetch_fsmap finished waiting for FSMap version 124 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 10 client.7808 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 20 client.7808 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f33fd0 0x1f35480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 4 client.7808 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1d58f50 con 0x1f33fd0 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.724+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f33fd0 0x1f35480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67d1ffb700 10 client.7808 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e2ffd700 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f33fd0 0x1f35480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=5d9ac00d3f03d304 server_cookie=1bd504770cdc8d99 in_seq=0 out_seq=0 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67d1ffb700 1 -- 172.21.15.181:0/936024881 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1d58f50 con 0x1f33fd0 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67d1ffb700 10 client.7808 handle_command_reply: tid=0 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 client.7808 shutdown 2020-11-16T22:36:21.721 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 msgr2=0x1f7e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f67cc06d120 tx=0x7f67cc014040).stop 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 shutdown_connections 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1f33fd0 0x1f35480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ee3e40 0x1f2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67d1ffb700 0 client.7808 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1ca7bb0 0x1f7e7f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 --2- 172.21.15.181:0/936024881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1d157b0 0x1f85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 >> 172.21.15.181:0/936024881 conn(0x7f67ac002320 msgr2=0x1f2d700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 shutdown_connections 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 1 -- 172.21.15.181:0/936024881 wait complete. 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67f8969740 20 client.7808 trim_cache size 0 max 16384 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 msgr2=0x7f67b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 0x7f67b4057520 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f67e40d21e0 tx=0x7f67d4009040).stop 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 msgr2=0x7f67e40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f67cc00c710 tx=0x7f67cc007a60).stop 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 shutdown_connections 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67b4055060 0x7f67b4057520 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67e4161860 0x7f67e40b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67e4062960 0x7f67e40ce3f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 --2- 172.21.15.181:0/1796698445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e4161480 0x7f67e40b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 >> 172.21.15.181:0/1796698445 conn(0x7f67e40d2dc0 msgr2=0x7f67e4062f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 shutdown_connections 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.728+0000 7f67e8c76700 1 -- 172.21.15.181:0/1796698445 wait complete. 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:21.725 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:21.726 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:21.726 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:21.726 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:21.726 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:21.726 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:21.733 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 Processor -- start 2020-11-16T22:36:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 -- start start 2020-11-16T22:36:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09fc15c6c0 0x7f09fc15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc15cb50 con 0x7f09fc0612a0 2020-11-16T22:36:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc15cda0 con 0x7f09fc061680 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a024fa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc15d040 con 0x7f09fc15c6c0 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09fb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f09fc061680 0x7f09fc0cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51884/0 (socket says 172.21.15.181:51884) 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09fb7fe700 1 -- 172.21.15.181:0/2469664058 learned_addr learned my addr 172.21.15.181:0/2469664058 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f0a00a97700 1 --2- 172.21.15.181:0/2469664058 >> v2:172.21.15.175:3300/0 conn(0x7f09fc15c6c0 0x7f09fc15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349334387 0 0) 0x7f09fc15cda0 con 0x7f09fc061680 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e8003240 con 0x7f09fc061680 2020-11-16T22:36:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2341579502 0 0) 0x7f09e8003240 con 0x7f09fc061680 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 >> v2:172.21.15.175:3300/0 conn(0x7f09fc15c6c0 msgr2=0x7f09fc15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 --2- 172.21.15.181:0/2469664058 >> v2:172.21.15.175:3300/0 conn(0x7f09fc15c6c0 0x7f09fc15fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 >> v1:172.21.15.175:6789/0 conn(0x7f09fc0612a0 legacy=0x7f09fc0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc15d2e0 con 0x7f09fc061680 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 244376255 0 0) 0x7f09f00014e0 con 0x7f09fc061680 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 >> v1:172.21.15.181:6789/0 conn(0x7f09fc061680 legacy=0x7f09fc0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e8003730 0x7f09e8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 0x7f09e8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004e50 0x7f09e800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc15d040 con 0x7f09e8004e50 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8004260 con 0x7f09e80044e0 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.860+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800bc20 con 0x7f09e8003730 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 0x7f09e8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a00a97700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004e50 0x7f09e800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fbfff700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e8003730 0x7f09e8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 -- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e8003730 msgr2=0x7f09e8003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e8003730 0x7f09e8003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 -- 172.21.15.181:0/2469664058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004e50 msgr2=0x7f09e800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004e50 0x7f09e800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a00a97700 1 -- 172.21.15.181:0/2469664058 reap_dead start 2020-11-16T22:36:21.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 -- 172.21.15.181:0/2469664058 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8003240 con 0x7f09e80044e0 2020-11-16T22:36:21.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 0x7f09e8004960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f09f000c470 tx=0x7f09fc063440).ready entity=mon.1 client_cookie=505640245bc2910e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f00070d0 con 0x7f09e80044e0 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f0007230 con 0x7f09e80044e0 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f09faffd700 1 -- 172.21.15.181:0/2469664058 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f00055f0 con 0x7f09e80044e0 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 msgr2=0x7f09e8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 0x7f09e8004960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f09f000c470 tx=0x7f09fc063440).stop 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- 172.21.15.181:0/2469664058 shutdown_connections 2020-11-16T22:36:21.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 --2- 172.21.15.181:0/2469664058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80044e0 0x7f09e8004960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- 172.21.15.181:0/2469664058 >> 172.21.15.181:0/2469664058 conn(0x7f09fc0cf890 msgr2=0x7f09fc0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- 172.21.15.181:0/2469664058 shutdown_connections 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- 172.21.15.181:0/2469664058 wait complete. 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 Processor -- start 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.864+0000 7f0a024fa700 1 -- start start 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 0x7f09fc16df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 0x7f09fc170a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc14c5d0 con 0x7f09fc0cc9c0 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc063a50 con 0x7f09fc0635d0 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc171270 con 0x7f09e80044e0 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45432/0 (socket says 172.21.15.181:45432) 2020-11-16T22:36:21.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 -- 172.21.15.181:0/1556821050 learned_addr learned my addr 172.21.15.181:0/1556821050 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a00a97700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 0x7f09fc170a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 0x7f09fc16df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 -- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 msgr2=0x7f09fc170a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 0x7f09fc170a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 -- 172.21.15.181:0/1556821050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 msgr2=0x7f09fc16df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 0x7f09fc16df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:21.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 -- 172.21.15.181:0/1556821050 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc171510 con 0x7f09e80044e0 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 0x7f09fc16df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a00a97700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 0x7f09fc170a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09fbfff700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f09ec00e700 tx=0x7f09ec009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec010480 con 0x7f09e80044e0 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec010040 con 0x7f09e80044e0 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0107f0 con 0x7f09e80044e0 2020-11-16T22:36:21.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc1717b0 con 0x7f09e80044e0 2020-11-16T22:36:21.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.868+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc16da80 con 0x7f09e80044e0 2020-11-16T22:36:21.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc069eb0 con 0x7f09e80044e0 2020-11-16T22:36:21.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f09ec010b60 con 0x7f09e80044e0 2020-11-16T22:36:21.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f09dbfff700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 0x7f09d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:21.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09ec06dca0 con 0x7f09e80044e0 2020-11-16T22:36:21.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 0x7f09d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:21.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.872+0000 7f09fb7fe700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 0x7f09d4057520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f09fc171a70 tx=0x7f09f0013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:21.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:21.876+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f09ec072070 con 0x7f09e80044e0 2020-11-16T22:36:22.065 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.076+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09fc0cf350 con 0x7f09e80044e0 2020-11-16T22:36:22.066 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.076+0000 7f09dbfff700 1 -- 172.21.15.181:0/1556821050 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f09ec043f00 con 0x7f09e80044e0 2020-11-16T22:36:22.066 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:22.066 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 msgr2=0x7f09d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 0x7f09d4057520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f09fc171a70 tx=0x7f09f0013000).stop 2020-11-16T22:36:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 msgr2=0x7f09fc063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f09ec00e700 tx=0x7f09ec009040).stop 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 shutdown_connections 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f09d4055060 0x7f09d4057520 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09fc0635d0 0x7f09fc170a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80044e0 0x7f09fc063c60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 --2- 172.21.15.181:0/1556821050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc0cc9c0 0x7f09fc16df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 >> 172.21.15.181:0/1556821050 conn(0x7f09fc0cf890 msgr2=0x7f09fc15d910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 shutdown_connections 2020-11-16T22:36:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.080+0000 7f0a024fa700 1 -- 172.21.15.181:0/1556821050 wait complete. 2020-11-16T22:36:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:22.084 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:36:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 Processor -- start 2020-11-16T22:36:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 -- start start 2020-11-16T22:36:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbcd4145e10 0x7fbcd41492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd4061b50 con 0x7fbcd40816e0 2020-11-16T22:36:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40618b0 con 0x7fbcd4145a30 2020-11-16T22:36:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd9109700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40cb930 con 0x7fbcd4145e10 2020-11-16T22:36:22.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbcd4145a30 0x7fbcd4080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51902/0 (socket says 172.21.15.181:51902) 2020-11-16T22:36:22.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.220+0000 7fbcd3fff700 1 -- 172.21.15.181:0/2798360731 learned_addr learned my addr 172.21.15.181:0/2798360731 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:22.212 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd8908700 1 --2- 172.21.15.181:0/2798360731 >> v2:172.21.15.175:3300/0 conn(0x7fbcd4145e10 0x7fbcd41492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946245324 0 0) 0x7fbcd40618b0 con 0x7fbcd4145a30 2020-11-16T22:36:22.213 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc0003240 con 0x7fbcd4145a30 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134596315 0 0) 0x7fbcd4061b50 con 0x7fbcd40816e0 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd40618b0 con 0x7fbcd40816e0 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd8908700 1 -- 172.21.15.181:0/2798360731 >> v1:172.21.15.181:6789/0 conn(0x7fbcd4145a30 legacy=0x7fbcd4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd8908700 1 -- 172.21.15.181:0/2798360731 >> v1:172.21.15.175:6789/0 conn(0x7fbcd40816e0 legacy=0x7fbcd40cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd8908700 1 -- 172.21.15.181:0/2798360731 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40cbbd0 con 0x7fbcd4145e10 2020-11-16T22:36:22.214 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd8908700 1 --2- 172.21.15.181:0/2798360731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd4145e10 0x7fbcd41492c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fbccc009c90 tx=0x7fbccc0091d0).ready entity=mon.0 client_cookie=77473cb7eeab77c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:22.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00b9c0 con 0x7fbcd4145e10 2020-11-16T22:36:22.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00bb20 con 0x7fbcd4145e10 2020-11-16T22:36:22.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd2ffd700 1 -- 172.21.15.181:0/2798360731 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00bea0 con 0x7fbcd4145e10 2020-11-16T22:36:22.215 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 -- 172.21.15.181:0/2798360731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd4145e10 msgr2=0x7fbcd41492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 --2- 172.21.15.181:0/2798360731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd4145e10 0x7fbcd41492c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fbccc009c90 tx=0x7fbccc0091d0).stop 2020-11-16T22:36:22.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 -- 172.21.15.181:0/2798360731 shutdown_connections 2020-11-16T22:36:22.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 --2- 172.21.15.181:0/2798360731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd4145e10 0x7fbcd41492c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 -- 172.21.15.181:0/2798360731 >> 172.21.15.181:0/2798360731 conn(0x7fbcd40d68c0 msgr2=0x7fbcd40d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 -- 172.21.15.181:0/2798360731 shutdown_connections 2020-11-16T22:36:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.224+0000 7fbcd9109700 1 -- 172.21.15.181:0/2798360731 wait complete. 2020-11-16T22:36:22.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 Processor -- start 2020-11-16T22:36:22.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- start start 2020-11-16T22:36:22.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd4145a30 0x7fbcd4143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 0x7fbcd4143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40ba220 con 0x7fbcd40816e0 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40ba4c0 con 0x7fbcd4145a30 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40ba760 con 0x7fbcd4145e10 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd8908700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 0x7fbcd4143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd4145a30 0x7fbcd4143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd8908700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 0x7fbcd4143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45450/0 (socket says 172.21.15.181:45450) 2020-11-16T22:36:22.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:51606/0 (socket says 172.21.15.181:51606) 2020-11-16T22:36:22.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd8908700 1 -- 172.21.15.181:0/925788519 learned_addr learned my addr 172.21.15.181:0/925788519 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 -- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 msgr2=0x7fbcd4143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 0x7fbcd4143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 -- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd4145a30 msgr2=0x7fbcd4143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd4145a30 0x7fbcd4143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 -- 172.21.15.181:0/925788519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40baa00 con 0x7fbcd40816e0 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fbcc800c6e0 tx=0x7fbcc8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:22.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcb3fff700 1 -- 172.21.15.181:0/925788519 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80072e0 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcb3fff700 1 -- 172.21.15.181:0/925788519 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8007440 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd40baca0 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcb3fff700 1 -- 172.21.15.181:0/925788519 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8005800 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.228+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd40bb3e0 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.232+0000 7fbcb3fff700 1 -- 172.21.15.181:0/925788519 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbcc8010070 con 0x7fbcd40816e0 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.232+0000 7fbcb3fff700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 0x7fbcac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.232+0000 7fbcb3fff700 1 -- 172.21.15.181:0/925788519 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcc806d470 con 0x7fbcd40816e0 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.232+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 0x7fbcac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.232+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 0x7fbcac057520 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fbcd4147eb0 tx=0x7fbcc4007a20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 Processor -- start 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- start start 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12e27b0 0x1503720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1500fd0 0x14fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:22.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x138c4b0 con 0x1500fd0 2020-11-16T22:36:22.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1440930 con 0x12e27b0 2020-11-16T22:36:22.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13006f0 con 0x1274bb0 2020-11-16T22:36:22.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12e27b0 0x1503720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45456/0 (socket says 172.21.15.181:45456) 2020-11-16T22:36:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 -- 172.21.15.181:0/2278578872 learned_addr learned my addr 172.21.15.181:0/2278578872 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd8908700 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1500fd0 0x14fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 -- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12e27b0 msgr2=0x1503720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12e27b0 0x1503720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 -- 172.21.15.181:0/2278578872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1500fd0 msgr2=0x14fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1500fd0 0x14fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 -- 172.21.15.181:0/2278578872 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12fb5a0 con 0x1274bb0 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fbcc4017ae0 tx=0x7fbcc400a000).ready entity=mon.2 client_cookie=653ae9e568b97b46 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:22.843 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcb17fa700 1 -- 172.21.15.181:0/2278578872 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4017240 con 0x1274bb0 2020-11-16T22:36:22.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcb17fa700 1 -- 172.21.15.181:0/2278578872 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc40173a0 con 0x1274bb0 2020-11-16T22:36:22.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbcb17fa700 1 -- 172.21.15.181:0/2278578872 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4015a10 con 0x1274bb0 2020-11-16T22:36:22.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 msgr2=0x1501670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fbcc4017ae0 tx=0x7fbcc400a000).stop 2020-11-16T22:36:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- 172.21.15.181:0/2278578872 shutdown_connections 2020-11-16T22:36:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1500fd0 0x14fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x12e27b0 0x1503720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/2278578872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x1501670 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- 172.21.15.181:0/2278578872 >> 172.21.15.181:0/2278578872 conn(0x14b0e40 msgr2=0x14fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- 172.21.15.181:0/2278578872 shutdown_connections 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- 172.21.15.181:0/2278578872 wait complete. 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 -- start start 2020-11-16T22:36:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.236+0000 7fbce8dfc740 1 Processor -- start 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x1560370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 0x1551750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x142d6e0 con 0x12e27b0 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x142d820 con 0x1500fd0 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x142ea90 con 0x1274bb0 2020-11-16T22:36:23.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 10 client.0 ll_register_callbacks cb 0x1423ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:23.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45460/0 (socket says 172.21.15.181:45460) 2020-11-16T22:36:23.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 -- 172.21.15.181:0/886259697 learned_addr learned my addr 172.21.15.181:0/886259697 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:23.250 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd8908700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 0x1551750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x1560370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 -- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 msgr2=0x1551750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 0x1551750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 -- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 msgr2=0x1560370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x1560370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 -- 172.21.15.181:0/886259697 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x141e280 con 0x1274bb0 2020-11-16T22:36:23.251 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd8908700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 0x1551750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x1560370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fbcc80458f0 tx=0x7fbcc8009000).ready entity=mon.2 client_cookie=fc6685616be7cb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8043be0 con 0x1274bb0 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcc806bb80 con 0x1274bb0 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x131ec50 con 0x1274bb0 2020-11-16T22:36:23.252 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcc806d2e0 con 0x1274bb0 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 10 client.7820 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x1253fd0 con 0x1274bb0 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x131ec50 con 0x1274bb0 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fbcc8051070 con 0x1274bb0 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 10 client.7820 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 10 client.7820 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:23.253 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 20 client.7820 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1560860 0x154ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 4 client.7820 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1325f50 con 0x1560860 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1560860 0x154ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcaa7fc700 10 client.7820 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.240+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1560860 0x154ed50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=f511c18f6d322f7f server_cookie=cd0546c076be57ef in_seq=0 out_seq=0 2020-11-16T22:36:23.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/886259697 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1325f50 con 0x1560860 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbcaa7fc700 10 client.7820 handle_command_reply: tid=0 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 client.7820 shutdown 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 msgr2=0x14fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fbcc80458f0 tx=0x7fbcc8009000).stop 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 shutdown_connections 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1560860 0x154ed50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1500fd0 0x1551750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x14fd530 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/886259697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x1560370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbcaa7fc700 0 client.7820 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 >> 172.21.15.181:0/886259697 conn(0x14b0e40 msgr2=0x14fb600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 shutdown_connections 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 1 -- 172.21.15.181:0/886259697 wait complete. 2020-11-16T22:36:23.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.244+0000 7fbce8dfc740 20 client.7820 trim_cache size 0 max 16384 2020-11-16T22:36:23.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 Processor -- start 2020-11-16T22:36:23.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 -- start start 2020-11-16T22:36:23.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x1536b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x15aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1405e40 con 0x12e27b0 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x15a67a0 con 0x14b0e40 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1502230 con 0x1274bb0 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45472/0 (socket says 172.21.15.181:45472) 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x1536b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 -- 172.21.15.181:0/719195893 learned_addr learned my addr 172.21.15.181:0/719195893 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd8908700 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x15aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 -- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 msgr2=0x1536b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x1536b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 -- 172.21.15.181:0/719195893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 msgr2=0x15aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x15aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 -- 172.21.15.181:0/719195893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca00081d0 con 0x1274bb0 2020-11-16T22:36:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd8908700 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x15aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fbcc4007e40 tx=0x7fbcc4025040).ready entity=mon.2 client_cookie=e613c1c61de2135f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcb1ffb700 1 -- 172.21.15.181:0/719195893 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4028040 con 0x1274bb0 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcb1ffb700 1 -- 172.21.15.181:0/719195893 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4017830 con 0x1274bb0 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.264+0000 7fbcb1ffb700 1 -- 172.21.15.181:0/719195893 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc402c280 con 0x1274bb0 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 msgr2=0x15adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fbcc4007e40 tx=0x7fbcc4025040).stop 2020-11-16T22:36:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- 172.21.15.181:0/719195893 shutdown_connections 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x12e27b0 0x15aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x1536b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/719195893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1274bb0 0x15adee0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- 172.21.15.181:0/719195893 >> 172.21.15.181:0/719195893 conn(0x7fbca0002320 msgr2=0x14fb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- 172.21.15.181:0/719195893 shutdown_connections 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- 172.21.15.181:0/719195893 wait complete. 2020-11-16T22:36:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 -- start start 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.268+0000 7fbce8dfc740 1 Processor -- start 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1274bb0 0x1590720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b74d0 con 0x1274bb0 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x15b7230 con 0x14b0e40 2020-11-16T22:36:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x15b3180 con 0x12e27b0 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 10 client.0 ll_register_callbacks cb 0x1560d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45476/0 (socket says 172.21.15.181:45476) 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37636/0 (socket says 172.21.15.181:37636) 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd37fe700 1 -- 172.21.15.181:0/4118825768 learned_addr learned my addr 172.21.15.181:0/4118825768 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1274bb0 0x1590720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 -- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 msgr2=0x15a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 -- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1274bb0 msgr2=0x1590720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1274bb0 0x1590720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 -- 172.21.15.181:0/4118825768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1292720 con 0x14b0e40 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd37fe700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd8908700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fbccc00b6c0 tx=0x7fbccc016000).ready entity=mon.1 client_cookie=caa4d305eba47216 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc01d6d0 con 0x14b0e40 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbccc01d830 con 0x14b0e40 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x14e16c0 con 0x14b0e40 2020-11-16T22:36:23.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbccc01dc00 con 0x14b0e40 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 10 client.8683 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x129de00 con 0x14b0e40 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x14e16c0 con 0x14b0e40 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fbccc01c3e0 con 0x14b0e40 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 10 client.8683 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 10 client.8683 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:23.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 20 client.8683 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1500fd0 0x15b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 4 client.8683 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x13031d0 con 0x1500fd0 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1500fd0 0x15b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcaa7fc700 10 client.8683 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.272+0000 7fbcd3fff700 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1500fd0 0x15b66f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=71e7f6b7a04ef4a0 server_cookie=571e56e1b53c234a in_seq=0 out_seq=0 2020-11-16T22:36:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcaa7fc700 1 -- 172.21.15.181:0/4118825768 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x13031d0 con 0x1500fd0 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcaa7fc700 10 client.8683 handle_command_reply: tid=0 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 client.8683 shutdown 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 msgr2=0x15a9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fbccc00b6c0 tx=0x7fbccc016000).stop 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 shutdown_connections 2020-11-16T22:36:23.670 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1500fd0 0x15b66f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x14b0e40 0x15a9150 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1274bb0 0x1590720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 --2- 172.21.15.181:0/4118825768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12e27b0 0x15a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 >> 172.21.15.181:0/4118825768 conn(0x7fbca0002320 msgr2=0x14fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcaa7fc700 0 client.8683 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 shutdown_connections 2020-11-16T22:36:23.671 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 1 -- 172.21.15.181:0/4118825768 wait complete. 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbce8dfc740 20 client.8683 trim_cache size 0 max 16384 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 msgr2=0x7fbcac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 0x7fbcac057520 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fbcd4147eb0 tx=0x7fbcc4007a20).stop 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 msgr2=0x7fbcd40b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fbcc800c6e0 tx=0x7fbcc8007a40).stop 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 shutdown_connections 2020-11-16T22:36:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fbcac055060 0x7fbcac057520 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd4145e10 0x7fbcd4143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd40816e0 0x7fbcd40b9850 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 --2- 172.21.15.181:0/925788519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd4145a30 0x7fbcd4143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 >> 172.21.15.181:0/925788519 conn(0x7fbcd40d68c0 msgr2=0x7fbcd4146530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 shutdown_connections 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:22.276+0000 7fbcd9109700 1 -- 172.21.15.181:0/925788519 wait complete. 2020-11-16T22:36:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:23.675 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:23.675 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:23.675 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:23.675 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:23.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 Processor -- start 2020-11-16T22:36:23.805 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 -- start start 2020-11-16T22:36:23.806 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd5c140550 0x7fcd5c140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5c0cf390 con 0x7fcd5c0d07e0 2020-11-16T22:36:23.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5c15ebb0 con 0x7fcd5c140550 2020-11-16T22:36:23.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0cf5d0 con 0x7fcd5c0cef10 2020-11-16T22:36:23.807 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45482/0 (socket says 172.21.15.181:45482) 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/367820809 learned_addr learned my addr 172.21.15.181:0/367820809 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5affd700 1 --2- 172.21.15.181:0/367820809 >> v2:172.21.15.181:3300/0 conn(0x7fcd5c140550 0x7fcd5c140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/367820809 >> v2:172.21.15.181:3300/0 conn(0x7fcd5c140550 msgr2=0x7fcd5c140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 --2- 172.21.15.181:0/367820809 >> v2:172.21.15.181:3300/0 conn(0x7fcd5c140550 0x7fcd5c140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/367820809 >> v1:172.21.15.175:6789/0 conn(0x7fcd5c0d07e0 legacy=0x7fcd5c063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.808 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/367820809 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0cf870 con 0x7fcd5c0cef10 2020-11-16T22:36:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.816+0000 7fcd5a7fc700 1 --2- 172.21.15.181:0/367820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fcd50006fd0 tx=0x7fcd50006eb0).ready entity=mon.2 client_cookie=ee0cccbf5c7cf42c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:23.809 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd59ffb700 1 -- 172.21.15.181:0/367820809 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50009810 con 0x7fcd5c0cef10 2020-11-16T22:36:23.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd59ffb700 1 -- 172.21.15.181:0/367820809 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50009970 con 0x7fcd5c0cef10 2020-11-16T22:36:23.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd59ffb700 1 -- 172.21.15.181:0/367820809 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50009cf0 con 0x7fcd5c0cef10 2020-11-16T22:36:23.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- 172.21.15.181:0/367820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 msgr2=0x7fcd5c0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:23.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/367820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fcd50006fd0 tx=0x7fcd50006eb0).stop 2020-11-16T22:36:23.810 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- 172.21.15.181:0/367820809 shutdown_connections 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/367820809 >> v2:172.21.15.181:3300/0 conn(0x7fcd5c140550 0x7fcd5c140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/367820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0cef10 0x7fcd5c0d02f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- 172.21.15.181:0/367820809 >> 172.21.15.181:0/367820809 conn(0x7fcd5c0d4570 msgr2=0x7fcd5c0d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- 172.21.15.181:0/367820809 shutdown_connections 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- 172.21.15.181:0/367820809 wait complete. 2020-11-16T22:36:23.811 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 Processor -- start 2020-11-16T22:36:23.812 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 -- start start 2020-11-16T22:36:23.813 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.820+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 0x7fcd5c170e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c140550 0x7fcd5c16e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0b79d0 con 0x7fcd5c0cef10 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0b7c70 con 0x7fcd5c0d07e0 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0b7f10 con 0x7fcd5c140550 2020-11-16T22:36:23.814 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37648/0 (socket says 172.21.15.181:37648) 2020-11-16T22:36:23.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/774921290 learned_addr learned my addr 172.21.15.181:0/774921290 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:23.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5b7fe700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c140550 0x7fcd5c16e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5affd700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 0x7fcd5c170e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:23.815 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c140550 msgr2=0x7fcd5c16e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c140550 0x7fcd5c16e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/774921290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 msgr2=0x7fcd5c170e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.018 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 0x7fcd5c170e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 -- 172.21.15.181:0/774921290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0b81b0 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5affd700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 0x7fcd5c170e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:24.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5a7fc700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7fcd5c15ecf0 tx=0x7fcd50015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50011d50 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.019 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5001e460 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5001e7e0 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c0b8450 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c171320 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c044f60 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcd50025030 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.021 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 0x7fcd38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd5006fc00 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5affd700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 0x7fcd38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.824+0000 7fcd5affd700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 0x7fcd38057520 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fcd4c006fd0 tx=0x7fcd4c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:23.832+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcd50045f90 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd5c0cae90 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd42ffd700 1 -- 172.21.15.181:0/774921290 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7fcd5002d070 con 0x7fcd5c0d07e0 2020-11-16T22:36:24.024 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:24.024 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 msgr2=0x7fcd38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 0x7fcd38057520 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fcd4c006fd0 tx=0x7fcd4c008040).stop 2020-11-16T22:36:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 msgr2=0x7fcd5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7fcd5c15ecf0 tx=0x7fcd50015000).stop 2020-11-16T22:36:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 shutdown_connections 2020-11-16T22:36:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd38055060 0x7fcd38057520 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c140550 0x7fcd5c16e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5c0cef10 0x7fcd5c170e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 --2- 172.21.15.181:0/774921290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c0d07e0 0x7fcd5c0b8b90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 >> 172.21.15.181:0/774921290 conn(0x7fcd5c0d4570 msgr2=0x7fcd5c0b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 shutdown_connections 2020-11-16T22:36:24.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.036+0000 7fcd5bfff700 1 -- 172.21.15.181:0/774921290 wait complete. 2020-11-16T22:36:24.232 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:24.233 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:36:24.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.380+0000 7f20f58e2700 1 Processor -- start 2020-11-16T22:36:24.371 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f58e2700 1 -- start start 2020-11-16T22:36:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f58e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f58e2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00c8db0 con 0x7f20f0164e10 2020-11-16T22:36:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f58e2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00c8330 con 0x7f20f0161860 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f58e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00c85d0 con 0x7f20f0161480 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f50e1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f20f0161860 0x7f20f00644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51960/0 (socket says 172.21.15.181:51960) 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f50e1700 1 -- 172.21.15.181:0/1281188179 learned_addr learned my addr 172.21.15.181:0/1281188179 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45496/0 (socket says 172.21.15.181:45496) 2020-11-16T22:36:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20ef7fe700 1 -- 172.21.15.181:0/1281188179 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020799178 0 0) 0x7f20f00c8330 con 0x7f20f0161860 2020-11-16T22:36:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 -- 172.21.15.181:0/1281188179 >> v1:172.21.15.181:6789/0 conn(0x7f20f0161860 legacy=0x7f20f00644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 -- 172.21.15.181:0/1281188179 >> v1:172.21.15.175:6789/0 conn(0x7f20f0164e10 legacy=0x7f20f0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 -- 172.21.15.181:0/1281188179 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00c8870 con 0x7f20f0161480 2020-11-16T22:36:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20f48e0700 1 --2- 172.21.15.181:0/1281188179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f20d800b9d0 tx=0x7f20d8006df0).ready entity=mon.2 client_cookie=b0b62a8d4c7177db server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20ef7fe700 1 -- 172.21.15.181:0/1281188179 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8009870 con 0x7f20f0161480 2020-11-16T22:36:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20ef7fe700 1 -- 172.21.15.181:0/1281188179 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d80099d0 con 0x7f20f0161480 2020-11-16T22:36:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.384+0000 7f20ef7fe700 1 -- 172.21.15.181:0/1281188179 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8009d50 con 0x7f20f0161480 2020-11-16T22:36:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- 172.21.15.181:0/1281188179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 msgr2=0x7f20f0164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 --2- 172.21.15.181:0/1281188179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f20d800b9d0 tx=0x7f20d8006df0).stop 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- 172.21.15.181:0/1281188179 shutdown_connections 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 --2- 172.21.15.181:0/1281188179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f0164920 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- 172.21.15.181:0/1281188179 >> 172.21.15.181:0/1281188179 conn(0x7f20f00d2dc0 msgr2=0x7f20f00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- 172.21.15.181:0/1281188179 shutdown_connections 2020-11-16T22:36:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- 172.21.15.181:0/1281188179 wait complete. 2020-11-16T22:36:24.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 Processor -- start 2020-11-16T22:36:24.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.388+0000 7f20f58e2700 1 -- start start 2020-11-16T22:36:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f0161860 0x7f20f00b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 0x7f20f00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0176e60 con 0x7f20f00b5b60 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f01770c0 con 0x7f20f0161860 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0177360 con 0x7f20f0161480 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f0161860 0x7f20f00b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45502/0 (socket says 172.21.15.181:45502) 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 -- 172.21.15.181:0/445148438 learned_addr learned my addr 172.21.15.181:0/445148438 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f50e1700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 0x7f20f00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 -- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f0161860 msgr2=0x7f20f00b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f0161860 0x7f20f00b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 -- 172.21.15.181:0/445148438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 msgr2=0x7f20f00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 0x7f20f00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 -- 172.21.15.181:0/445148438 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0177600 con 0x7f20f0161480 2020-11-16T22:36:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f50e1700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 0x7f20f00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f48e0700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f20d8000dd0 tx=0x7f20d8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20ecff9700 1 -- 172.21.15.181:0/445148438 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d801e460 con 0x7f20f0161480 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20ecff9700 1 -- 172.21.15.181:0/445148438 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d801e5c0 con 0x7f20f0161480 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f01778a0 con 0x7f20f0161480 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20ecff9700 1 -- 172.21.15.181:0/445148438 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d801e940 con 0x7f20f0161480 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.392+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f00b5fc0 con 0x7f20f0161480 2020-11-16T22:36:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.396+0000 7f20ecff9700 1 -- 172.21.15.181:0/445148438 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f20d8025030 con 0x7f20f0161480 2020-11-16T22:36:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.396+0000 7f20ecff9700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 0x7f20cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.396+0000 7f20ecff9700 1 -- 172.21.15.181:0/445148438 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20d806f7b0 con 0x7f20f0161480 2020-11-16T22:36:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.396+0000 7f20effff700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 0x7f20cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.396+0000 7f20effff700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 0x7f20cc057520 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f20f00cf930 tx=0x7f20e0007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 Processor -- start 2020-11-16T22:36:24.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 -- start start 2020-11-16T22:36:24.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29c27b0 0x2be3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2be1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a6c4b0 con 0x29c27b0 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b20930 con 0x2be0fd0 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x29e06f0 con 0x2954bb0 2020-11-16T22:36:24.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2be1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37670/0 (socket says 172.21.15.181:37670) 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 -- 172.21.15.181:0/154807731 learned_addr learned my addr 172.21.15.181:0/154807731 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f48e0700 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29c27b0 0x2be3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 -- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 msgr2=0x2be1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2be1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 -- 172.21.15.181:0/154807731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29c27b0 msgr2=0x2be3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29c27b0 0x2be3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 -- 172.21.15.181:0/154807731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29db5a0 con 0x2be0fd0 2020-11-16T22:36:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20effff700 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2be1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.400+0000 7f20f50e1700 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f20e8001fe0 tx=0x7f20e8009040).ready entity=mon.1 client_cookie=7ef3aa1de27e5123 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20e5ffb700 1 -- 172.21.15.181:0/154807731 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8005b80 con 0x2be0fd0 2020-11-16T22:36:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20e5ffb700 1 -- 172.21.15.181:0/154807731 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8005ce0 con 0x2be0fd0 2020-11-16T22:36:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20e5ffb700 1 -- 172.21.15.181:0/154807731 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e800ea90 con 0x2be0fd0 2020-11-16T22:36:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 msgr2=0x2bda700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f20e8001fe0 tx=0x7f20e8009040).stop 2020-11-16T22:36:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- 172.21.15.181:0/154807731 shutdown_connections 2020-11-16T22:36:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2be0fd0 0x2bda700 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x29c27b0 0x2be3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- 172.21.15.181:0/154807731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2be1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- 172.21.15.181:0/154807731 >> 172.21.15.181:0/154807731 conn(0x2b90e40 msgr2=0x2bdd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- 172.21.15.181:0/154807731 shutdown_connections 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- 172.21.15.181:0/154807731 wait complete. 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- start start 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 Processor -- start 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2954bb0 0x2bdd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c2fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 0x2c2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b0d6e0 con 0x2954bb0 2020-11-16T22:36:24.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b0d820 con 0x29c27b0 2020-11-16T22:36:24.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b0ea90 con 0x2be0fd0 2020-11-16T22:36:24.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f21055d5740 10 client.0 ll_register_callbacks cb 0x2b03ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:24.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c2fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 0x2c2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c2fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37676/0 (socket says 172.21.15.181:37676) 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20effff700 1 -- 172.21.15.181:0/3245019694 learned_addr learned my addr 172.21.15.181:0/3245019694 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f48e0700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2954bb0 0x2bdd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 -- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 msgr2=0x2c2fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c2fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 -- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2954bb0 msgr2=0x2bdd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2954bb0 0x2bdd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.404+0000 7f20f50e1700 1 -- 172.21.15.181:0/3245019694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2afe280 con 0x2be0fd0 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20f50e1700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 0x2c2ff40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f20e8014000 tx=0x7f20e8009040).ready entity=mon.2 client_cookie=93cbaffff3f34325 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8005e50 con 0x2be0fd0 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2bc16c0 con 0x2be0fd0 2020-11-16T22:36:24.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20e8001bb0 con 0x2be0fd0 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e800ea90 con 0x2be0fd0 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 10 client.7844 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x2933fd0 con 0x2be0fd0 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2bc16c0 con 0x2be0fd0 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f20e801f3f0 con 0x2be0fd0 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 10 client.7844 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:24.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 10 client.7844 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:24.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 20 client.7844 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:24.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2c40a10 0x2c2f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 4 client.7844 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:24.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x29fec50 con 0x2c40a10 2020-11-16T22:36:24.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20f48e0700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2c40a10 0x2c2f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 10 client.7844 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:24.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20f48e0700 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2c40a10 0x2c2f070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=9bb9ae0287b9ff80 server_cookie=30bfb96e334f9919 in_seq=0 out_seq=0 2020-11-16T22:36:24.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 1 -- 172.21.15.181:0/3245019694 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x29fec50 con 0x2c40a10 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 10 client.7844 handle_command_reply: tid=0 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 client.7844 shutdown 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 msgr2=0x2c2ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 0x2c2ff40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f20e8014000 tx=0x7f20e8009040).stop 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 shutdown_connections 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2c40a10 0x2c2f070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2be0fd0 0x2c2ff40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2954bb0 0x2bdd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 --2- 172.21.15.181:0/3245019694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c2fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 >> 172.21.15.181:0/3245019694 conn(0x2b90e40 msgr2=0x2c16a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f20beffd700 0 client.7844 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 shutdown_connections 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 1 -- 172.21.15.181:0/3245019694 wait complete. 2020-11-16T22:36:24.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.408+0000 7f21055d5740 20 client.7844 trim_cache size 0 max 16384 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 Processor -- start 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 -- start start 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2bd56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2bdc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae5e40 con 0x2b90e40 2020-11-16T22:36:24.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bd61f0 con 0x29c27b0 2020-11-16T22:36:24.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2c2d2c0 con 0x2954bb0 2020-11-16T22:36:24.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45524/0 (socket says 172.21.15.181:45524) 2020-11-16T22:36:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 -- 172.21.15.181:0/2319321636 learned_addr learned my addr 172.21.15.181:0/2319321636 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20f50e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2bdc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20f48e0700 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2bd56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 -- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 msgr2=0x2bdc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2bdc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 -- 172.21.15.181:0/2319321636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 msgr2=0x2bd56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2bd56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 -- 172.21.15.181:0/2319321636 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40081d0 con 0x2954bb0 2020-11-16T22:36:24.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20effff700 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f20f0061220 tx=0x7f20e0019040).ready entity=mon.2 client_cookie=63130ee86424b4ee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:24.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20e67fc700 1 -- 172.21.15.181:0/2319321636 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e001c070 con 0x2954bb0 2020-11-16T22:36:24.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20e67fc700 1 -- 172.21.15.181:0/2319321636 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e001d040 con 0x2954bb0 2020-11-16T22:36:24.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f20e67fc700 1 -- 172.21.15.181:0/2319321636 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0015d30 con 0x2954bb0 2020-11-16T22:36:24.820 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 -- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 msgr2=0x2bda7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.432+0000 7f21055d5740 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f20f0061220 tx=0x7f20e0019040).stop 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- 172.21.15.181:0/2319321636 shutdown_connections 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2bdc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2bd56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- 172.21.15.181:0/2319321636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2bda7e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- 172.21.15.181:0/2319321636 >> 172.21.15.181:0/2319321636 conn(0x7f20c4002320 msgr2=0x2bdfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:24.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- 172.21.15.181:0/2319321636 shutdown_connections 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- 172.21.15.181:0/2319321636 wait complete. 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- start start 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 Processor -- start 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2c32440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:24.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c330e0 con 0x2b90e40 2020-11-16T22:36:24.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c32e40 con 0x29c27b0 2020-11-16T22:36:24.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2c95a00 con 0x2954bb0 2020-11-16T22:36:24.823 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f21055d5740 10 client.0 ll_register_callbacks cb 0x2be1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:25.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f20f48e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45530/0 (socket says 172.21.15.181:45530) 2020-11-16T22:36:25.026 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f20f48e0700 1 -- 172.21.15.181:0/1143494098 learned_addr learned my addr 172.21.15.181:0/1143494098 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.436+0000 7f20effff700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f50e1700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2c32440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 -- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 msgr2=0x2c84b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c84b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 -- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 msgr2=0x2c32440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2c32440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 -- 172.21.15.181:0/1143494098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2972720 con 0x2954bb0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f50e1700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2c32440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20f48e0700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f20d8049600 tx=0x7f20d8020040).ready entity=mon.2 client_cookie=257e05a0534f2a01 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8048630 con 0x2954bb0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20d8048790 con 0x2954bb0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20e8005ce0 con 0x2954bb0 2020-11-16T22:36:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d8048950 con 0x2954bb0 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 10 client.7850 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x297de00 con 0x2954bb0 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8005ce0 con 0x2954bb0 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f20d801ed30 con 0x2954bb0 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 10 client.7850 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 10 client.7850 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:25.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 20 client.7850 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2be0fd0 0x2c922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 4 client.7850 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x2a05f50 con 0x2be0fd0 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20effff700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2be0fd0 0x2c922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 10 client.7850 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:25.030 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20effff700 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2be0fd0 0x2c922d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=36c7d1b216116ef1 server_cookie=764982164ed2df95 in_seq=0 out_seq=0 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 1 -- 172.21.15.181:0/1143494098 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x2a05f50 con 0x2be0fd0 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 10 client.7850 handle_command_reply: tid=0 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 client.7850 shutdown 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 msgr2=0x2c3edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f20d8049600 tx=0x7f20d8020040).stop 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 shutdown_connections 2020-11-16T22:36:25.031 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x2be0fd0 0x2c922d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2b90e40 0x2c32440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f20beffd700 0 client.7850 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2954bb0 0x2c3edc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 --2- 172.21.15.181:0/1143494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x29c27b0 0x2c84b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 >> 172.21.15.181:0/1143494098 conn(0x7f20c4002320 msgr2=0x2bdfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 shutdown_connections 2020-11-16T22:36:25.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.440+0000 7f21055d5740 1 -- 172.21.15.181:0/1143494098 wait complete. 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f21055d5740 20 client.7850 trim_cache size 0 max 16384 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 msgr2=0x7f20cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 0x7f20cc057520 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f20f00cf930 tx=0x7f20e0007b00).stop 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 msgr2=0x7f20f01603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f20d8000dd0 tx=0x7f20d8015000).stop 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 shutdown_connections 2020-11-16T22:36:25.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f20cc055060 0x7f20cc057520 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20f00b5b60 0x7f20f00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20f0161480 0x7f20f01603c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 --2- 172.21.15.181:0/445148438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f0161860 0x7f20f00b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 >> 172.21.15.181:0/445148438 conn(0x7f20f00d2dc0 msgr2=0x7f20f0147660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 shutdown_connections 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:24.444+0000 7f20f58e2700 1 -- 172.21.15.181:0/445148438 wait complete. 2020-11-16T22:36:25.034 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:25.035 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:25.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:25.036 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:25.036 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:25.037 INFO:tasks.ceph.mon.b.smithi175.stderr:2020-11-16T22:36:24.815+0000 7fc89c0c1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:36:25.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 Processor -- start 2020-11-16T22:36:25.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 -- start start 2020-11-16T22:36:25.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff078145a30 0x7ff078145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff078061b50 con 0x7ff0781492c0 2020-11-16T22:36:25.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780618b0 con 0x7ff0780816e0 2020-11-16T22:36:25.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0780cb930 con 0x7ff078145a30 2020-11-16T22:36:25.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0777fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff0780816e0 0x7ff0780cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52000/0 (socket says 172.21.15.181:52000) 2020-11-16T22:36:25.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff076ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff078145a30 0x7ff078145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0777fe700 1 -- 172.21.15.181:0/423463271 learned_addr learned my addr 172.21.15.181:0/423463271 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:25.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0767fc700 1 -- 172.21.15.181:0/423463271 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899298121 0 0) 0x7ff0780618b0 con 0x7ff0780816e0 2020-11-16T22:36:25.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0767fc700 1 -- 172.21.15.181:0/423463271 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff058003240 con 0x7ff0780816e0 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff076ffd700 1 -- 172.21.15.181:0/423463271 >> v1:172.21.15.181:6789/0 conn(0x7ff0780816e0 legacy=0x7ff0780cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff076ffd700 1 -- 172.21.15.181:0/423463271 >> v1:172.21.15.175:6789/0 conn(0x7ff0781492c0 legacy=0x7ff078080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff076ffd700 1 -- 172.21.15.181:0/423463271 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780cbbd0 con 0x7ff078145a30 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff076ffd700 1 --2- 172.21.15.181:0/423463271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff078145a30 0x7ff078145e10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ff06800b9a0 tx=0x7ff068006df0).ready entity=mon.1 client_cookie=4fb29bf46f708a86 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0767fc700 1 -- 172.21.15.181:0/423463271 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0680097b0 con 0x7ff078145a30 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0767fc700 1 -- 172.21.15.181:0/423463271 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068009910 con 0x7ff078145a30 2020-11-16T22:36:25.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff0767fc700 1 -- 172.21.15.181:0/423463271 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068009c90 con 0x7ff078145a30 2020-11-16T22:36:25.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 -- 172.21.15.181:0/423463271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff078145a30 msgr2=0x7ff078145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.184+0000 7ff077fff700 1 --2- 172.21.15.181:0/423463271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff078145a30 0x7ff078145e10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ff06800b9a0 tx=0x7ff068006df0).stop 2020-11-16T22:36:25.176 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 -- 172.21.15.181:0/423463271 shutdown_connections 2020-11-16T22:36:25.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 --2- 172.21.15.181:0/423463271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff078145a30 0x7ff078145e10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 -- 172.21.15.181:0/423463271 >> 172.21.15.181:0/423463271 conn(0x7ff0780d68c0 msgr2=0x7ff0780d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 -- 172.21.15.181:0/423463271 shutdown_connections 2020-11-16T22:36:25.177 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 -- 172.21.15.181:0/423463271 wait complete. 2020-11-16T22:36:25.178 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 Processor -- start 2020-11-16T22:36:25.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.188+0000 7ff077fff700 1 -- start start 2020-11-16T22:36:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0780816e0 0x7ff0780cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 0x7ff0780cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07816f650 con 0x7ff0780816e0 2020-11-16T22:36:25.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07816f840 con 0x7ff0781492c0 2020-11-16T22:36:25.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07816fae0 con 0x7ff078145a30 2020-11-16T22:36:25.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff06e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 0x7ff0780cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37706/0 (socket says 172.21.15.181:37706) 2020-11-16T22:36:25.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff06e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 0x7ff0780cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45546/0 (socket says 172.21.15.181:45546) 2020-11-16T22:36:25.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 -- 172.21.15.181:0/3839884891 learned_addr learned my addr 172.21.15.181:0/3839884891 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff076ffd700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0780816e0 0x7ff0780cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 -- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 msgr2=0x7ff0780cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 0x7ff0780cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 -- 172.21.15.181:0/3839884891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0780816e0 msgr2=0x7ff0780cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0780816e0 0x7ff0780cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 -- 172.21.15.181:0/3839884891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07816fd80 con 0x7ff0781492c0 2020-11-16T22:36:25.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff0777fe700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ff07000c9e0 tx=0x7ff070007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070005b80 con 0x7ff0781492c0 2020-11-16T22:36:25.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070005ce0 con 0x7ff0781492c0 2020-11-16T22:36:25.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff078170020 con 0x7ff0781492c0 2020-11-16T22:36:25.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070007320 con 0x7ff0781492c0 2020-11-16T22:36:25.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.192+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0780d18f0 con 0x7ff0781492c0 2020-11-16T22:36:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff070010030 con 0x7ff0781492c0 2020-11-16T22:36:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0780625b0 con 0x7ff0781492c0 2020-11-16T22:36:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff06f7fe700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 0x7ff054057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff07006db90 con 0x7ff0781492c0 2020-11-16T22:36:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff076ffd700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 0x7ff054057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.196+0000 7ff076ffd700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 0x7ff054057520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7ff068006b80 tx=0x7ff068015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.200+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff0780625b0 con 0x7ff0781492c0 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0780625b0 con 0x7ff0781492c0 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff06f7fe700 1 -- 172.21.15.181:0/3839884891 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7ff0780625b0 con 0x7ff0781492c0 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 msgr2=0x7ff054057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.391 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 0x7ff054057520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7ff068006b80 tx=0x7ff068015000).stop 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 msgr2=0x7ff0780d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ff07000c9e0 tx=0x7ff070007d40).stop 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 shutdown_connections 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff054055060 0x7ff054057520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0781492c0 0x7ff0780d1400 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0780816e0 0x7ff0780cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.392 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 --2- 172.21.15.181:0/3839884891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff078145a30 0x7ff0780cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 >> 172.21.15.181:0/3839884891 conn(0x7ff0780d68c0 msgr2=0x7ff078144c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 shutdown_connections 2020-11-16T22:36:25.393 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.400+0000 7ff077fff700 1 -- 172.21.15.181:0/3839884891 wait complete. 2020-11-16T22:36:25.393 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:25.393 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:25.394 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:25.403 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:36:25.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 Processor -- start 2020-11-16T22:36:25.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- start start 2020-11-16T22:36:25.533 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 0x7f2614142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2614063620 0x7f2614063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f2614061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2614062700 con 0x7f2614142660 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140629a0 con 0x7f2614063620 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2614062c40 con 0x7f2614160530 2020-11-16T22:36:25.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2613fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f2614061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2614063620 0x7f2614063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2613fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f2614061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45552/0 (socket says 172.21.15.181:45552) 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2614063620 0x7f2614063ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37710/0 (socket says 172.21.15.181:37710) 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2613fff700 1 -- 172.21.15.181:0/2959121115 learned_addr learned my addr 172.21.15.181:0/2959121115 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f26137fe700 1 --2- 172.21.15.181:0/2959121115 >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 0x7f2614142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 -- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 msgr2=0x7f2614061810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f2614061810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 -- 172.21.15.181:0/2959121115 >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 msgr2=0x7f2614142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2959121115 >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 0x7f2614142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 -- 172.21.15.181:0/2959121115 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614062ee0 con 0x7f2614063620 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f26137fe700 1 --2- 172.21.15.181:0/2959121115 >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 0x7f2614142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614063620 0x7f2614063ec0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f261405ff50 tx=0x7f2608006df0).ready entity=mon.1 client_cookie=4686880d3a8349b2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f26127fc700 1 -- 172.21.15.181:0/2959121115 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608009840 con 0x7f2614063620 2020-11-16T22:36:25.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f26127fc700 1 -- 172.21.15.181:0/2959121115 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26080099a0 con 0x7f2614063620 2020-11-16T22:36:25.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f26127fc700 1 -- 172.21.15.181:0/2959121115 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608009d20 con 0x7f2614063620 2020-11-16T22:36:25.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614063620 msgr2=0x7f2614063ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614063620 0x7f2614063ec0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f261405ff50 tx=0x7f2608006df0).stop 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- 172.21.15.181:0/2959121115 shutdown_connections 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f2614061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- 172.21.15.181:0/2959121115 >> v2:172.21.15.175:3300/0 conn(0x7f2614142660 0x7f2614142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 --2- 172.21.15.181:0/2959121115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614063620 0x7f2614063ec0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- 172.21.15.181:0/2959121115 >> 172.21.15.181:0/2959121115 conn(0x7f26140d2dc0 msgr2=0x7f26140ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- 172.21.15.181:0/2959121115 shutdown_connections 2020-11-16T22:36:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.544+0000 7f2618d79700 1 -- 172.21.15.181:0/2959121115 wait complete. 2020-11-16T22:36:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 Processor -- start 2020-11-16T22:36:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 -- start start 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2614063620 0x7f26140ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f261413f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614139f10 con 0x7f2614063620 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614172710 con 0x7f2614142660 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2618d79700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26141729b0 con 0x7f2614160530 2020-11-16T22:36:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37718/0 (socket says 172.21.15.181:37718) 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 -- 172.21.15.181:0/3106200040 learned_addr learned my addr 172.21.15.181:0/3106200040 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2613fff700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f261413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f26137fe700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2614063620 0x7f26140ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 -- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 msgr2=0x7f261413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f261413f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 -- 172.21.15.181:0/3106200040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2614063620 msgr2=0x7f26140ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2614063620 0x7f26140ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 -- 172.21.15.181:0/3106200040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614172c50 con 0x7f2614142660 2020-11-16T22:36:25.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f2614062ae0 tx=0x7f2608015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.548+0000 7f25f37fe700 1 -- 172.21.15.181:0/3106200040 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608011e50 con 0x7f2614142660 2020-11-16T22:36:25.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2614172ef0 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f25f37fe700 1 -- 172.21.15.181:0/3106200040 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260801e460 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f25f37fe700 1 -- 172.21.15.181:0/3106200040 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801e7e0 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614065660 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f25f37fe700 1 -- 172.21.15.181:0/3106200040 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2608025030 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f25f37fe700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 0x7f25ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f25f37fe700 1 -- 172.21.15.181:0/3106200040 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f260806f430 con 0x7f2614142660 2020-11-16T22:36:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f26137fe700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 0x7f25ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.552+0000 7f26137fe700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 0x7f25ec057520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f2604006bd0 tx=0x7f2604008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 Processor -- start 2020-11-16T22:36:25.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 -- start start 2020-11-16T22:36:25.958 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xef8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xeef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xd814b0 con 0xcd77b0 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe35930 con 0xef5fd0 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xcf56f0 con 0xc69bb0 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.959 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45568/0 (socket says 172.21.15.181:45568) 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2612ffd700 1 -- 172.21.15.181:0/2246434568 learned_addr learned my addr 172.21.15.181:0/2246434568 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.556+0000 7f2613fff700 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xeef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f26137fe700 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xef8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 -- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 msgr2=0xeef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xeef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 -- 172.21.15.181:0/2246434568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 msgr2=0xef8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xef8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 -- 172.21.15.181:0/2246434568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcf05a0 con 0xc69bb0 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f2608049220 tx=0x7f2608011000).ready entity=mon.2 client_cookie=8855bdc3c8e63c66 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f25f0ff9700 1 -- 172.21.15.181:0/2246434568 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260806f030 con 0xc69bb0 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f25f0ff9700 1 -- 172.21.15.181:0/2246434568 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260806f190 con 0xc69bb0 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f25f0ff9700 1 -- 172.21.15.181:0/2246434568 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801ede0 con 0xc69bb0 2020-11-16T22:36:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 msgr2=0xef6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f2608049220 tx=0x7f2608011000).stop 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- 172.21.15.181:0/2246434568 shutdown_connections 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xeef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xef8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2246434568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef6670 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- 172.21.15.181:0/2246434568 >> 172.21.15.181:0/2246434568 conn(0xea5e40 msgr2=0xef23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:25.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- 172.21.15.181:0/2246434568 shutdown_connections 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- 172.21.15.181:0/2246434568 wait complete. 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- start start 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 Processor -- start 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf47730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xf55470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xe226e0 con 0xc69bb0 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xe22820 con 0xef5fd0 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xe23a90 con 0xcd77b0 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2628a6c740 10 client.0 ll_register_callbacks cb 0xe18ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45574/0 (socket says 172.21.15.181:45574) 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2612ffd700 1 -- 172.21.15.181:0/2597640131 learned_addr learned my addr 172.21.15.181:0/2597640131 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f2613fff700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xf55470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.560+0000 7f26137fe700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf47730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 -- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 msgr2=0xf55470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xf55470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 -- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 msgr2=0xf47730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf47730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 -- 172.21.15.181:0/2597640131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe13280 con 0xcd77b0 2020-11-16T22:36:25.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f26137fe700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf47730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2612ffd700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f2608049220 tx=0x7f2608009f90).ready entity=mon.2 client_cookie=2cab99b9b9450d3d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608045400 con 0xcd77b0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2608048360 con 0xcd77b0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xed66c0 con 0xcd77b0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2608048730 con 0xcd77b0 2020-11-16T22:36:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 10 client.7865 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0xc48fd0 con 0xcd77b0 2020-11-16T22:36:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xed66c0 con 0xcd77b0 2020-11-16T22:36:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2608047a60 con 0xcd77b0 2020-11-16T22:36:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 10 client.7865 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 10 client.7865 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 20 client.7865 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xf11bd0 0xef1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 4 client.7865 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xd13c50 con 0xf11bd0 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f26137fe700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xf11bd0 0xef1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 10 client.7865 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f26137fe700 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xf11bd0 0xef1e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=f8782201cc24ab95 server_cookie=a0f7f192e84c7e26 in_seq=0 out_seq=0 2020-11-16T22:36:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 1 -- 172.21.15.181:0/2597640131 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xd13c50 con 0xf11bd0 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 10 client.7865 handle_command_reply: tid=0 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 client.7865 shutdown 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 msgr2=0xef04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f2608049220 tx=0x7f2608009f90).stop 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 shutdown_connections 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xf11bd0 0xef1e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xef5fd0 0xf55470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f25e9ffb700 0 client.7865 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf47730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 --2- 172.21.15.181:0/2597640131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xcd77b0 0xef04c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.564+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 >> 172.21.15.181:0/2597640131 conn(0xea5e40 msgr2=0xf29270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.568+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 shutdown_connections 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.568+0000 7f2628a6c740 1 -- 172.21.15.181:0/2597640131 wait complete. 2020-11-16T22:36:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.568+0000 7f2628a6c740 20 client.7865 trim_cache size 0 max 16384 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 Processor -- start 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- start start 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xf43970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xdfae40 con 0xcd77b0 2020-11-16T22:36:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xf403a0 con 0xea5e40 2020-11-16T22:36:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf45850 con 0xc69bb0 2020-11-16T22:36:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.175 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37740/0 (socket says 172.21.15.181:37740) 2020-11-16T22:36:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 -- 172.21.15.181:0/4003311859 learned_addr learned my addr 172.21.15.181:0/4003311859 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2613fff700 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xf43970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 -- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 msgr2=0xef52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 -- 172.21.15.181:0/4003311859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 msgr2=0xf43970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xf43970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 -- 172.21.15.181:0/4003311859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00081d0 con 0xea5e40 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2613fff700 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xf43970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f26137fe700 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f2604012c40 tx=0x7f2604014040).ready entity=mon.1 client_cookie=7f698e9cd87f5b52 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f25f17fa700 1 -- 172.21.15.181:0/4003311859 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604021a20 con 0xea5e40 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f25f17fa700 1 -- 172.21.15.181:0/4003311859 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2604021b80 con 0xea5e40 2020-11-16T22:36:26.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f25f17fa700 1 -- 172.21.15.181:0/4003311859 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26040293a0 con 0xea5e40 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 msgr2=0xf411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f2604012c40 tx=0x7f2604014040).stop 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- 172.21.15.181:0/4003311859 shutdown_connections 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xcd77b0 0xf43970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xea5e40 0xf411e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 --2- 172.21.15.181:0/4003311859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xc69bb0 0xef52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- 172.21.15.181:0/4003311859 >> 172.21.15.181:0/4003311859 conn(0x7f25e0002320 msgr2=0xeef700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- 172.21.15.181:0/4003311859 shutdown_connections 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.588+0000 7f2628a6c740 1 -- 172.21.15.181:0/4003311859 wait complete. 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 -- start start 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 Processor -- start 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf24040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xea5e40 0xf440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0xf25e40 con 0xc69bb0 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0xf445c0 con 0xcd77b0 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf245f0 con 0xea5e40 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2628a6c740 10 client.0 ll_register_callbacks cb 0xef6540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2613fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xea5e40 0xf440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2612ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37744/0 (socket says 172.21.15.181:37744) 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f2612ffd700 1 -- 172.21.15.181:0/3865634771 learned_addr learned my addr 172.21.15.181:0/3865634771 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.592+0000 7f26137fe700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf24040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 -- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xea5e40 msgr2=0xf440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xea5e40 0xf440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 -- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 msgr2=0xf24040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf24040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 -- 172.21.15.181:0/3865634771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc87720 con 0xcd77b0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2612ffd700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f2608048310 tx=0x7f2608034040).ready entity=mon.1 client_cookie=a38222d85b754f58 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608047f60 con 0xcd77b0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2608048360 con 0xcd77b0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf41030 con 0xcd77b0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2608048730 con 0xcd77b0 2020-11-16T22:36:26.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 10 client.8734 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0xc92e00 con 0xcd77b0 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf41030 con 0xcd77b0 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f2608047a60 con 0xcd77b0 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 10 client.8734 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 10 client.8734 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 20 client.8734 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:26.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xef5fd0 0xf47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 4 client.8734 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x7f260806f190 con 0xef5fd0 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f26137fe700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xef5fd0 0xf47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 10 client.8734 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f26137fe700 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xef5fd0 0xf47660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=538e2985e658ef0a server_cookie=e4b35796e1ad11ba in_seq=0 out_seq=0 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 1 -- 172.21.15.181:0/3865634771 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7f260806f190 con 0xef5fd0 2020-11-16T22:36:26.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 10 client.8734 handle_command_reply: tid=0 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 client.8734 shutdown 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 msgr2=0xf4a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f2608048310 tx=0x7f2608034040).stop 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 shutdown_connections 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0xef5fd0 0xf47660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xea5e40 0xf440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f25e9ffb700 0 client.8734 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xc69bb0 0xf24040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 --2- 172.21.15.181:0/3865634771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xcd77b0 0xf4a090 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 >> 172.21.15.181:0/3865634771 conn(0x7f25e0002320 msgr2=0xeeaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 shutdown_connections 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 1 -- 172.21.15.181:0/3865634771 wait complete. 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.596+0000 7f2628a6c740 20 client.8734 trim_cache size 0 max 16384 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 msgr2=0x7f25ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 0x7f25ec057520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f2604006bd0 tx=0x7f2604008040).stop 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 msgr2=0x7f261413ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f2614062ae0 tx=0x7f2608015000).stop 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 shutdown_connections 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f25ec055060 0x7f25ec057520 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2614160530 0x7f261413f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2614063620 0x7f26140ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 --2- 172.21.15.181:0/3106200040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2614142660 0x7f261413ed50 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 >> 172.21.15.181:0/3106200040 conn(0x7f26140d2dc0 msgr2=0x7f26140d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 shutdown_connections 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:25.600+0000 7f2618d79700 1 -- 172.21.15.181:0/3106200040 wait complete. 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:26.391 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:26.392 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:26.525 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.536+0000 7f2443f06700 1 Processor -- start 2020-11-16T22:36:26.526 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.536+0000 7f2443f06700 1 -- start start 2020-11-16T22:36:26.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f243c063b80 0x7f243c066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c064050 con 0x7f243c15e270 2020-11-16T22:36:26.527 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0642f0 con 0x7f243c0b6c20 2020-11-16T22:36:26.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c064590 con 0x7f243c063b80 2020-11-16T22:36:26.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24414a1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f243c063b80 0x7f243c066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24424a3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f243c0b6c20 0x7f243c15f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52058/0 (socket says 172.21.15.181:52058) 2020-11-16T22:36:26.528 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24424a3700 1 -- 172.21.15.181:0/138301927 learned_addr learned my addr 172.21.15.181:0/138301927 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:26.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24414a1700 1 -- 172.21.15.181:0/138301927 >> v1:172.21.15.181:6789/0 conn(0x7f243c0b6c20 legacy=0x7f243c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.529 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24414a1700 1 -- 172.21.15.181:0/138301927 >> v1:172.21.15.175:6789/0 conn(0x7f243c15e270 legacy=0x7f243c0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2440ca0700 1 -- 172.21.15.181:0/138301927 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693547437 0 0) 0x7f243c0642f0 con 0x7f243c0b6c20 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24414a1700 1 -- 172.21.15.181:0/138301927 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c064830 con 0x7f243c063b80 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f24414a1700 1 --2- 172.21.15.181:0/138301927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c063b80 0x7f243c066ec0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f242c00b9a0 tx=0x7f242c006df0).ready entity=mon.1 client_cookie=742e859de3d43917 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2440ca0700 1 -- 172.21.15.181:0/138301927 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0097b0 con 0x7f243c063b80 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2440ca0700 1 -- 172.21.15.181:0/138301927 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c009910 con 0x7f243c063b80 2020-11-16T22:36:26.530 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2440ca0700 1 -- 172.21.15.181:0/138301927 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c009c90 con 0x7f243c063b80 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- 172.21.15.181:0/138301927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c063b80 msgr2=0x7f243c066ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 --2- 172.21.15.181:0/138301927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c063b80 0x7f243c066ec0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f242c00b9a0 tx=0x7f242c006df0).stop 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- 172.21.15.181:0/138301927 shutdown_connections 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 --2- 172.21.15.181:0/138301927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c063b80 0x7f243c066ec0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- 172.21.15.181:0/138301927 >> 172.21.15.181:0/138301927 conn(0x7f243c0d4790 msgr2=0x7f243c0d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- 172.21.15.181:0/138301927 shutdown_connections 2020-11-16T22:36:26.531 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.540+0000 7f2443f06700 1 -- 172.21.15.181:0/138301927 wait complete. 2020-11-16T22:36:26.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 Processor -- start 2020-11-16T22:36:26.532 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 -- start start 2020-11-16T22:36:26.534 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c15e270 0x7f243c159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c1622f0 0x7f243c162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c162c20 con 0x7f243c15e270 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2441ca2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c162e80 con 0x7f243c0b6c20 2020-11-16T22:36:26.535 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2441ca2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37756/0 (socket says 172.21.15.181:37756) 2020-11-16T22:36:26.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2443f06700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c163120 con 0x7f243c1622f0 2020-11-16T22:36:26.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f2441ca2700 1 -- 172.21.15.181:0/2862600764 learned_addr learned my addr 172.21.15.181:0/2862600764 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f24424a3700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c1622f0 0x7f243c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.536 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.544+0000 7f24414a1700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c15e270 0x7f243c159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 -- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c1622f0 msgr2=0x7f243c162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c1622f0 0x7f243c162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 -- 172.21.15.181:0/2862600764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c15e270 msgr2=0x7f243c159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.537 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c15e270 0x7f243c159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 -- 172.21.15.181:0/2862600764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c1633c0 con 0x7f243c0b6c20 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2441ca2700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f243800c6e0 tx=0x7f2438007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438007020 con 0x7f243c0b6c20 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438007180 con 0x7f243c0b6c20 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c163660 con 0x7f243c0b6c20 2020-11-16T22:36:26.538 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380064f0 con 0x7f243c0b6c20 2020-11-16T22:36:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c154bf0 con 0x7f243c0b6c20 2020-11-16T22:36:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c069eb0 con 0x7f243c0b6c20 2020-11-16T22:36:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2438010070 con 0x7f243c0b6c20 2020-11-16T22:36:26.539 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 0x7f2414057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f243806d050 con 0x7f243c0b6c20 2020-11-16T22:36:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f24414a1700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 0x7f2414057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.548+0000 7f24414a1700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 0x7f2414057520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f243c063f80 tx=0x7f242c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.552+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f24380432b0 con 0x7f243c0b6c20 2020-11-16T22:36:26.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.756+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f243c063410 con 0x7f243c0b6c20 2020-11-16T22:36:26.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.756+0000 7f2429ffb700 1 -- 172.21.15.181:0/2862600764 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f24380468d0 con 0x7f243c0b6c20 2020-11-16T22:36:26.745 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:26.746 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 msgr2=0x7f2414057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 0x7f2414057520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f243c063f80 tx=0x7f242c015000).stop 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 msgr2=0x7f243c158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f243800c6e0 tx=0x7f2438007c50).stop 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 shutdown_connections 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2414055060 0x7f2414057520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c1622f0 0x7f243c162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c0b6c20 0x7f243c158d40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 --2- 172.21.15.181:0/2862600764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c15e270 0x7f243c159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 >> 172.21.15.181:0/2862600764 conn(0x7f243c0d4790 msgr2=0x7f243c15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 shutdown_connections 2020-11-16T22:36:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.760+0000 7f2443f06700 1 -- 172.21.15.181:0/2862600764 wait complete. 2020-11-16T22:36:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:26.760 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:36:26.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.896+0000 7f4c57fff700 1 Processor -- start 2020-11-16T22:36:26.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.896+0000 7f4c57fff700 1 -- start start 2020-11-16T22:36:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c5813e870 0x7f4c5813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5815d180 con 0x7f4c5815fdc0 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c58160730 con 0x7f4c5813e870 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5813f140 con 0x7f4c58061810 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45610/0 (socket says 172.21.15.181:45610) 2020-11-16T22:36:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 -- 172.21.15.181:0/2885819305 learned_addr learned my addr 172.21.15.181:0/2885819305 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2885819305 >> v2:172.21.15.175:3300/0 conn(0x7f4c5813e870 0x7f4c5813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 -- 172.21.15.181:0/2885819305 >> v2:172.21.15.175:3300/0 conn(0x7f4c5813e870 msgr2=0x7f4c5813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2885819305 >> v2:172.21.15.175:3300/0 conn(0x7f4c5813e870 0x7f4c5813ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 -- 172.21.15.181:0/2885819305 >> v1:172.21.15.175:6789/0 conn(0x7f4c5815fdc0 legacy=0x7f4c58061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 -- 172.21.15.181:0/2885819305 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5815d810 con 0x7f4c58061810 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2885819305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f4c50009d50 tx=0x7f4c50009290).ready entity=mon.2 client_cookie=1ae636939479e007 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c55ffb700 1 -- 172.21.15.181:0/2885819305 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000bbc0 con 0x7f4c58061810 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c55ffb700 1 -- 172.21.15.181:0/2885819305 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000bd20 con 0x7f4c58061810 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c55ffb700 1 -- 172.21.15.181:0/2885819305 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000f680 con 0x7f4c58061810 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- 172.21.15.181:0/2885819305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 msgr2=0x7f4c58160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2885819305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f4c50009d50 tx=0x7f4c50009290).stop 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- 172.21.15.181:0/2885819305 shutdown_connections 2020-11-16T22:36:26.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2885819305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c58061810 0x7f4c58160240 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2885819305 >> v2:172.21.15.175:3300/0 conn(0x7f4c5813e870 0x7f4c5813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- 172.21.15.181:0/2885819305 >> 172.21.15.181:0/2885819305 conn(0x7f4c580cf630 msgr2=0x7f4c580cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- 172.21.15.181:0/2885819305 shutdown_connections 2020-11-16T22:36:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.900+0000 7f4c57fff700 1 -- 172.21.15.181:0/2885819305 wait complete. 2020-11-16T22:36:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 Processor -- start 2020-11-16T22:36:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 -- start start 2020-11-16T22:36:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c58061810 0x7f4c5816b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5813e870 0x7f4c5816dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5813a840 con 0x7f4c58061810 2020-11-16T22:36:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5816e8b0 con 0x7f4c5813e870 2020-11-16T22:36:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c57fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5816eb50 con 0x7f4c5815fdc0 2020-11-16T22:36:26.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45616/0 (socket says 172.21.15.181:45616) 2020-11-16T22:36:26.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c577fe700 1 -- 172.21.15.181:0/2662796445 learned_addr learned my addr 172.21.15.181:0/2662796445 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5813e870 0x7f4c5816dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.904+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c58061810 0x7f4c5816b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 -- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5813e870 msgr2=0x7f4c5816dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5813e870 0x7f4c5816dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 -- 172.21.15.181:0/2662796445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c58061810 msgr2=0x7f4c5816b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c58061810 0x7f4c5816b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 -- 172.21.15.181:0/2662796445 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5816edf0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f4c58160870 tx=0x7f4c50018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 -- 172.21.15.181:0/2662796445 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000bbc0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 -- 172.21.15.181:0/2662796445 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000b750 con 0x7f4c5815fdc0 2020-11-16T22:36:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 -- 172.21.15.181:0/2662796445 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000fea0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5816abb0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5816b0f0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 -- 172.21.15.181:0/2662796445 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4c50029070 con 0x7f4c5815fdc0 2020-11-16T22:36:26.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 0x7f4c34057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 0x7f4c34057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c3effd700 1 -- 172.21.15.181:0/2662796445 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c50070ad0 con 0x7f4c5815fdc0 2020-11-16T22:36:26.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.908+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 0x7f4c34057520 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f4c48006fd0 tx=0x7f4c48006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.912+0000 7f4c6c2b3740 1 Processor -- start 2020-11-16T22:36:26.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.912+0000 7f4c6c2b3740 1 -- start start 2020-11-16T22:36:26.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x25d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x25d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24634b0 con 0x234bbb0 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2517930 con 0x23b97b0 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x23d76f0 con 0x25d7fd0 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c56ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x25d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c56ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25da720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37778/0 (socket says 172.21.15.181:37778) 2020-11-16T22:36:26.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c56ffd700 1 -- 172.21.15.181:0/2411085974 learned_addr learned my addr 172.21.15.181:0/2411085974 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x25d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 -- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 msgr2=0x25da720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25da720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 -- 172.21.15.181:0/2411085974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 msgr2=0x25d8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x25d8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 -- 172.21.15.181:0/2411085974 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23d25a0 con 0x25d7fd0 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x25d1700 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f4c5004a880 tx=0x7f4c50014000).ready entity=mon.2 client_cookie=5280d400c5bb2684 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c2ffff700 1 -- 172.21.15.181:0/2411085974 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000be90 con 0x25d7fd0 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c2ffff700 1 -- 172.21.15.181:0/2411085974 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50048d90 con 0x25d7fd0 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c2ffff700 1 -- 172.21.15.181:0/2411085974 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50047100 con 0x25d7fd0 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 msgr2=0x25d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.916+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x25d1700 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f4c5004a880 tx=0x7f4c50014000).stop 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2411085974 shutdown_connections 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x25d1700 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2411085974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x25d8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2411085974 >> 172.21.15.181:0/2411085974 conn(0x2587e40 msgr2=0x25d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2411085974 shutdown_connections 2020-11-16T22:36:26.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2411085974 wait complete. 2020-11-16T22:36:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- start start 2020-11-16T22:36:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 Processor -- start 2020-11-16T22:36:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x2608380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x262cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25046e0 con 0x234bbb0 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2504820 con 0x23b97b0 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2505a90 con 0x25d7fd0 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c6c2b3740 10 client.0 ll_register_callbacks cb 0x24faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37786/0 (socket says 172.21.15.181:37786) 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 -- 172.21.15.181:0/4203440560 learned_addr learned my addr 172.21.15.181:0/4203440560 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x262cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x2608380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 -- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 msgr2=0x262cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x262cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 -- 172.21.15.181:0/4203440560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 msgr2=0x2608380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x2608380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 -- 172.21.15.181:0/4203440560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x24f5280 con 0x23b97b0 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x2608380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c577fe700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x262cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c567fc700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f4c580b75b0 tx=0x7f4c4c007b90).ready entity=mon.1 client_cookie=695b5a16338300ac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.920+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c014740 con 0x23b97b0 2020-11-16T22:36:26.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25b86c0 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c4c0148a0 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c4c014c70 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 10 client.8752 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x232afd0 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25b86c0 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f4c4c01c020 con 0x23b97b0 2020-11-16T22:36:26.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 10 client.8752 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 10 client.8752 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 20 client.8752 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2637b50 0x25f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 4 client.8752 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x23f5c50 con 0x2637b50 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2637b50 0x25f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.918 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 10 client.8752 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:26.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2637b50 0x25f3290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=88614abeb5616e00 server_cookie=2bd9e478ace6649c in_seq=0 out_seq=0 2020-11-16T22:36:26.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/4203440560 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x23f5c50 con 0x2637b50 2020-11-16T22:36:26.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 10 client.8752 handle_command_reply: tid=0 2020-11-16T22:36:26.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 client.8752 shutdown 2020-11-16T22:36:26.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 msgr2=0x262c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f4c580b75b0 tx=0x7f4c4c007b90).stop 2020-11-16T22:36:26.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 shutdown_connections 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2637b50 0x25f3290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x25d7fd0 0x262cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x234bbb0 0x2608380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c2d7fa700 0 client.8752 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/4203440560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x262c620 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 >> 172.21.15.181:0/4203440560 conn(0x2587e40 msgr2=0x25d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 shutdown_connections 2020-11-16T22:36:26.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/4203440560 wait complete. 2020-11-16T22:36:26.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.924+0000 7f4c6c2b3740 20 client.8752 trim_cache size 0 max 16384 2020-11-16T22:36:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 Processor -- start 2020-11-16T22:36:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- start start 2020-11-16T22:36:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2587e40 0x262b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 0x25d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x24dce40 con 0x23b97b0 2020-11-16T22:36:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d2530 con 0x2587e40 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26086c0 con 0x234bbb0 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45636/0 (socket says 172.21.15.181:45636) 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 -- 172.21.15.181:0/2957586727 learned_addr learned my addr 172.21.15.181:0/2957586727 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2587e40 0x262b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 0x25d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 -- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2587e40 msgr2=0x262b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2587e40 0x262b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 -- 172.21.15.181:0/2957586727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 msgr2=0x25d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 0x25d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 -- 172.21.15.181:0/2957586727 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24008160 con 0x234bbb0 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 0x25d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f4c4c0039f0 tx=0x7f4c4c01f000).ready entity=mon.2 client_cookie=3bbbe034680a6d3f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c3cff9700 1 -- 172.21.15.181:0/2957586727 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c02b070 con 0x234bbb0 2020-11-16T22:36:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c3cff9700 1 -- 172.21.15.181:0/2957586727 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c014740 con 0x234bbb0 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c3cff9700 1 -- 172.21.15.181:0/2957586727 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c000db0 con 0x234bbb0 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 msgr2=0x26081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f4c4c0039f0 tx=0x7f4c4c01f000).stop 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2957586727 shutdown_connections 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23b97b0 0x25d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2587e40 0x262b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2957586727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x26081d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2957586727 >> 172.21.15.181:0/2957586727 conn(0x7f4c24002320 msgr2=0x263a900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2957586727 shutdown_connections 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2957586727 wait complete. 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 -- start start 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.948+0000 7f4c6c2b3740 1 Processor -- start 2020-11-16T22:36:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 0x2637170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d4d90 con 0x2587e40 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2690ea0 con 0x23b97b0 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x2691140 con 0x234bbb0 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 10 client.0 ll_register_callbacks cb 0x25d8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45640/0 (socket says 172.21.15.181:45640) 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 -- 172.21.15.181:0/2713083889 learned_addr learned my addr 172.21.15.181:0/2713083889 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 0x2637170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 -- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 msgr2=0x25d4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25d4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 -- 172.21.15.181:0/2713083889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 msgr2=0x2637170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 0x2637170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 -- 172.21.15.181:0/2713083889 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2369720 con 0x234bbb0 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c577fe700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 0x2637170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c56ffd700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x262a190 tx=0x7f4c48026040).ready entity=mon.2 client_cookie=8cfaa80b8474d1fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48029040 con 0x234bbb0 2020-11-16T22:36:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c48014030 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c50048d90 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c48024e70 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 10 client.7892 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x2374e00 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50048d90 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f4c4802e430 con 0x234bbb0 2020-11-16T22:36:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 10 client.7892 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 10 client.7892 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 20 client.7892 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x25d7fd0 0x2637660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 4 client.7892 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x23fcf50 con 0x25d7fd0 2020-11-16T22:36:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x25d7fd0 0x2637660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:26.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c2d7fa700 10 client.7892 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.952+0000 7f4c567fc700 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x25d7fd0 0x2637660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=900a04bac3536e59 server_cookie=15312513c62e4b2c in_seq=0 out_seq=0 2020-11-16T22:36:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c2d7fa700 1 -- 172.21.15.181:0/2713083889 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x23fcf50 con 0x25d7fd0 2020-11-16T22:36:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c2d7fa700 10 client.7892 handle_command_reply: tid=0 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 client.7892 shutdown 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 msgr2=0x25cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x262a190 tx=0x7f4c48026040).stop 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 shutdown_connections 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x25d7fd0 0x2637660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x2587e40 0x2637170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x234bbb0 0x25cd000 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 --2- 172.21.15.181:0/2713083889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23b97b0 0x25d4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 >> 172.21.15.181:0/2713083889 conn(0x7f4c24002320 msgr2=0x263a890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c2d7fa700 0 client.7892 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 shutdown_connections 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 1 -- 172.21.15.181:0/2713083889 wait complete. 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c6c2b3740 20 client.7892 trim_cache size 0 max 16384 2020-11-16T22:36:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 msgr2=0x7f4c34057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 0x7f4c34057520 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f4c48006fd0 tx=0x7f4c48006df0).stop 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 msgr2=0x7f4c580b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f4c58160870 tx=0x7f4c50018000).stop 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 shutdown_connections 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4c34055060 0x7f4c34057520 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c5815fdc0 0x7f4c580b5b20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c58061810 0x7f4c5816b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 --2- 172.21.15.181:0/2662796445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5813e870 0x7f4c5816dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 >> 172.21.15.181:0/2662796445 conn(0x7f4c580cf630 msgr2=0x7f4c580cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 shutdown_connections 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:26.956+0000 7f4c57fff700 1 -- 172.21.15.181:0/2662796445 wait complete. 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "03d12be3-3211-4866-bb7d-6e192b97eebc": { 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/d1/d2/d3", 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "03d12be3-3211-4866-bb7d-6e192b97eebc", 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive,force" 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:26.956 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:26.961 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.092+0000 7f25a759e700 1 Processor -- start 2020-11-16T22:36:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.092+0000 7f25a759e700 1 -- start start 2020-11-16T22:36:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 0x7f25a8145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25a8160450 0x7f25a81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a8081f20 con 0x7f25a80d2320 2020-11-16T22:36:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a8060470 con 0x7f25a8160450 2020-11-16T22:36:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a8060710 con 0x7f25a8145770 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 0x7f25a8145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25a8160450 0x7f25a81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25a8160450 0x7f25a81608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:51810/0 (socket says 172.21.15.181:51810) 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 -- 172.21.15.181:0/3561000731 learned_addr learned my addr 172.21.15.181:0/3561000731 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 0x7f25a8145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37806/0 (socket says 172.21.15.181:37806) 2020-11-16T22:36:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a6d9d700 1 --1- 172.21.15.181:0/3561000731 >> v1:172.21.15.175:6789/0 conn(0x7f25a80d2320 0x7f25a80d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:45226/0 (socket says 172.21.15.181:45226) 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 -- 172.21.15.181:0/3561000731 >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 msgr2=0x7f25a8145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 --2- 172.21.15.181:0/3561000731 >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 0x7f25a8145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 -- 172.21.15.181:0/3561000731 >> v1:172.21.15.175:6789/0 conn(0x7f25a80d2320 legacy=0x7f25a80d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 -- 172.21.15.181:0/3561000731 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80609b0 con 0x7f25a8160450 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f259dd9b700 1 --2- 172.21.15.181:0/3561000731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8160450 0x7f25a81608b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f25a8069250 tx=0x7f2590006eb0).ready entity=mon.0 client_cookie=87d8a382204d2077 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a5d9b700 1 -- 172.21.15.181:0/3561000731 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590009810 con 0x7f25a8160450 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a5d9b700 1 -- 172.21.15.181:0/3561000731 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590009970 con 0x7f25a8160450 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a5d9b700 1 -- 172.21.15.181:0/3561000731 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590009cf0 con 0x7f25a8160450 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- 172.21.15.181:0/3561000731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8160450 msgr2=0x7f25a81608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 --2- 172.21.15.181:0/3561000731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8160450 0x7f25a81608b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f25a8069250 tx=0x7f2590006eb0).stop 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- 172.21.15.181:0/3561000731 shutdown_connections 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 --2- 172.21.15.181:0/3561000731 >> v2:172.21.15.181:3300/0 conn(0x7f25a8145770 0x7f25a8145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 --2- 172.21.15.181:0/3561000731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8160450 0x7f25a81608b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- 172.21.15.181:0/3561000731 >> 172.21.15.181:0/3561000731 conn(0x7f25a80d6760 msgr2=0x7f25a80d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- 172.21.15.181:0/3561000731 shutdown_connections 2020-11-16T22:36:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.096+0000 7f25a759e700 1 -- 172.21.15.181:0/3561000731 wait complete. 2020-11-16T22:36:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 Processor -- start 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- start start 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 0x7f25a8144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a8160450 0x7f25a8161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8145070 con 0x7f25a8145770 2020-11-16T22:36:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8171d10 con 0x7f25a80d2320 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8171fb0 con 0x7f25a8160450 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37810/0 (socket says 172.21.15.181:37810) 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 -- 172.21.15.181:0/751456110 learned_addr learned my addr 172.21.15.181:0/751456110 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a6d9d700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a8160450 0x7f25a8161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f259dd9b700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 0x7f25a8144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 -- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a8160450 msgr2=0x7f25a8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a8160450 0x7f25a8161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 -- 172.21.15.181:0/751456110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 msgr2=0x7f25a8144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 0x7f25a8144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 -- 172.21.15.181:0/751456110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8172250 con 0x7f25a80d2320 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f259dd9b700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 0x7f25a8144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a659c700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f259800ebd0 tx=0x7f2598006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800b3a0 con 0x7f25a80d2320 2020-11-16T22:36:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800ca60 con 0x7f25a80d2320 2020-11-16T22:36:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a81724f0 con 0x7f25a80d2320 2020-11-16T22:36:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800ce20 con 0x7f25a80d2320 2020-11-16T22:36:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.100+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a80cea00 con 0x7f25a80d2320 2020-11-16T22:36:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80cebf0 con 0x7f25a80d2320 2020-11-16T22:36:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2598012600 con 0x7f25a80d2320 2020-11-16T22:36:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f259effd700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 0x7f2578057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f259dd9b700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 0x7f2578057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f259806e1c0 con 0x7f25a80d2320 2020-11-16T22:36:27.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.104+0000 7f259dd9b700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 0x7f2578057550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f25a80d6100 tx=0x7f2590015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.112+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f25980443c0 con 0x7f25a80d2320 2020-11-16T22:36:27.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.316+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25a8044f60 con 0x7f25a80d2320 2020-11-16T22:36:27.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.316+0000 7f259effd700 1 -- 172.21.15.181:0/751456110 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f25980479e0 con 0x7f25a80d2320 2020-11-16T22:36:27.306 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:27.306 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:27.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 msgr2=0x7f2578057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 0x7f2578057550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f25a80d6100 tx=0x7f2590015000).stop 2020-11-16T22:36:27.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 msgr2=0x7f25a8144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f259800ebd0 tx=0x7f2598006df0).stop 2020-11-16T22:36:27.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 shutdown_connections 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2578055090 0x7f2578057550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25a8160450 0x7f25a8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25a80d2320 0x7f25a8144690 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 --2- 172.21.15.181:0/751456110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25a8145770 0x7f25a8144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 >> 172.21.15.181:0/751456110 conn(0x7f25a80d6760 msgr2=0x7f25a80d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 shutdown_connections 2020-11-16T22:36:27.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.320+0000 7f25a759e700 1 -- 172.21.15.181:0/751456110 wait complete. 2020-11-16T22:36:27.310 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:27.321 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:36:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.456+0000 7f6ec091c700 1 Processor -- start 2020-11-16T22:36:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.456+0000 7f6ec091c700 1 -- start start 2020-11-16T22:36:27.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ebc0629b0 0x7f6ebc080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc160e40 con 0x7f6ebc15f5b0 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc160ba0 con 0x7f6ebc0615f0 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc1609a0 con 0x7f6ebc0629b0 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6ebc0615f0 0x7f6ebc0622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52124/0 (socket says 172.21.15.181:52124) 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/4178886579 learned_addr learned my addr 172.21.15.181:0/4178886579 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/4178886579 >> v2:172.21.15.175:3300/0 conn(0x7f6ebc0629b0 0x7f6ebc080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713171913 0 0) 0x7f6ebc160ba0 con 0x7f6ebc0615f0 2020-11-16T22:36:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea0003240 con 0x7f6ebc0615f0 2020-11-16T22:36:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3105942450 0 0) 0x7f6ea0003240 con 0x7f6ebc0615f0 2020-11-16T22:36:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ebad9d700 1 -- 172.21.15.181:0/4178886579 >> v1:172.21.15.181:6789/0 conn(0x7f6ebc0615f0 legacy=0x7f6ebc0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ebad9d700 1 -- 172.21.15.181:0/4178886579 >> v1:172.21.15.175:6789/0 conn(0x7f6ebc15f5b0 legacy=0x7f6ebc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ebad9d700 1 -- 172.21.15.181:0/4178886579 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc060970 con 0x7f6ebc0629b0 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/4178886579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc0629b0 0x7f6ebc080fd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6eb4009c10 tx=0x7f6eb40091d0).ready entity=mon.0 client_cookie=afcf9f9410af9e4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0014e0 con 0x7f6ebc0629b0 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb400b790 con 0x7f6ebc0629b0 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6eb959a700 1 -- 172.21.15.181:0/4178886579 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400bb10 con 0x7f6ebc0629b0 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- 172.21.15.181:0/4178886579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc0629b0 msgr2=0x7f6ebc080fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 --2- 172.21.15.181:0/4178886579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc0629b0 0x7f6ebc080fd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6eb4009c10 tx=0x7f6eb40091d0).stop 2020-11-16T22:36:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- 172.21.15.181:0/4178886579 shutdown_connections 2020-11-16T22:36:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 --2- 172.21.15.181:0/4178886579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc0629b0 0x7f6ebc080fd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- 172.21.15.181:0/4178886579 >> 172.21.15.181:0/4178886579 conn(0x7f6ebc0d69a0 msgr2=0x7f6ebc0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- 172.21.15.181:0/4178886579 shutdown_connections 2020-11-16T22:36:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.460+0000 7f6ec091c700 1 -- 172.21.15.181:0/4178886579 wait complete. 2020-11-16T22:36:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 Processor -- start 2020-11-16T22:36:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- start start 2020-11-16T22:36:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ebc0615f0 0x7f6ebc15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc15f5b0 0x7f6ebc0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc172ac0 con 0x7f6ebc15f5b0 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc172d60 con 0x7f6ebc0615f0 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc173000 con 0x7f6ebc0629b0 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45666/0 (socket says 172.21.15.181:45666) 2020-11-16T22:36:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/1305058766 learned_addr learned my addr 172.21.15.181:0/1305058766 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ebc0615f0 0x7f6ebc15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc15f5b0 0x7f6ebc0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ebc0615f0 msgr2=0x7f6ebc15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ebc0615f0 0x7f6ebc15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/1305058766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc15f5b0 msgr2=0x7f6ebc0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc15f5b0 0x7f6ebc0d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/1305058766 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc1732a0 con 0x7f6ebc0629b0 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f6eac00cab0 tx=0x7f6eac001e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eaa7fc700 1 -- 172.21.15.181:0/1305058766 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac007320 con 0x7f6ebc0629b0 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eaa7fc700 1 -- 172.21.15.181:0/1305058766 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac0074a0 con 0x7f6ebc0629b0 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc173540 con 0x7f6ebc0629b0 2020-11-16T22:36:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6eaa7fc700 1 -- 172.21.15.181:0/1305058766 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac005840 con 0x7f6ebc0629b0 2020-11-16T22:36:27.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.464+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc15ad90 con 0x7f6ebc0629b0 2020-11-16T22:36:27.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.468+0000 7f6eaa7fc700 1 -- 172.21.15.181:0/1305058766 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6eac019020 con 0x7f6ebc0629b0 2020-11-16T22:36:27.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.468+0000 7f6eaa7fc700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 0x7f6e90057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.468+0000 7f6eaa7fc700 1 -- 172.21.15.181:0/1305058766 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eac06d920 con 0x7f6ebc0629b0 2020-11-16T22:36:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.468+0000 7f6eba59c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 0x7f6e90057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.468+0000 7f6eba59c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 0x7f6e90057520 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f6eb0003350 tx=0x7f6eb0007af0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 Processor -- start 2020-11-16T22:36:27.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- start start 2020-11-16T22:36:27.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11e57b0 0x13fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x143b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x128f4b0 con 0x11e57b0 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1343930 con 0x1177bb0 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x12036f0 con 0x13fe470 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x143b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45678/0 (socket says 172.21.15.181:45678) 2020-11-16T22:36:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x143b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37832/0 (socket says 172.21.15.181:37832) 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2534367236 learned_addr learned my addr 172.21.15.181:0/2534367236 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6eba59c700 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11e57b0 0x13fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 msgr2=0x143b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x143b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2534367236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11e57b0 msgr2=0x13fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11e57b0 0x13fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2534367236 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11fe5a0 con 0x13fe470 2020-11-16T22:36:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f6ebc083ad0 tx=0x7f6eb4016000).ready entity=mon.2 client_cookie=bb13bd98adbce650 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6e977fe700 1 -- 172.21.15.181:0/2534367236 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400bad0 con 0x13fe470 2020-11-16T22:36:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6e977fe700 1 -- 172.21.15.181:0/2534367236 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb401d660 con 0x13fe470 2020-11-16T22:36:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6e977fe700 1 -- 172.21.15.181:0/2534367236 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb401d9e0 con 0x13fe470 2020-11-16T22:36:27.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 msgr2=0x1458c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f6ebc083ad0 tx=0x7f6eb4016000).stop 2020-11-16T22:36:27.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2534367236 shutdown_connections 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13fe470 0x1458c90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11e57b0 0x13fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2534367236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x143b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.472+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2534367236 >> 172.21.15.181:0/2534367236 conn(0x13b3e40 msgr2=0x1458580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2534367236 shutdown_connections 2020-11-16T22:36:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2534367236 wait complete. 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- start start 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 Processor -- start 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x1426f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 0x14646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x13306e0 con 0x13fe470 2020-11-16T22:36:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1330820 con 0x1177bb0 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1331a90 con 0x11e57b0 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 10 client.0 ll_register_callbacks cb 0x1326ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x1426f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45682/0 (socket says 172.21.15.181:45682) 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x1426f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37838/0 (socket says 172.21.15.181:37838) 2020-11-16T22:36:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/874251701 learned_addr learned my addr 172.21.15.181:0/874251701 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 0x14646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 msgr2=0x1426f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x1426f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/874251701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 msgr2=0x14646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 0x14646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/874251701 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1321280 con 0x11e57b0 2020-11-16T22:36:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 0x14646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f6eac047540 tx=0x7f6eac006000).ready entity=mon.2 client_cookie=33dbf5688c2d77b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac04d070 con 0x11e57b0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13f8e10 con 0x11e57b0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eac005bb0 con 0x11e57b0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac043630 con 0x11e57b0 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 10 client.7904 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x1156fd0 con 0x11e57b0 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13f8e10 con 0x11e57b0 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f6eac043790 con 0x11e57b0 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 10 client.7904 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 10 client.7904 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 20 client.7904 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1454520 0x1415600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 4 client.7904 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1221c50 con 0x1454520 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eba59c700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1454520 0x1415600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6e94ff9700 10 client.7904 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.476+0000 7f6eba59c700 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1454520 0x1415600 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=23229309593cc2f0 server_cookie=bd3cf90b6713af20 in_seq=0 out_seq=0 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6e94ff9700 1 -- 172.21.15.181:0/874251701 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1221c50 con 0x1454520 2020-11-16T22:36:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6e94ff9700 10 client.7904 handle_command_reply: tid=0 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 client.7904 shutdown 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 msgr2=0x14641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f6eac047540 tx=0x7f6eac006000).stop 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 shutdown_connections 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x1454520 0x1415600 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13fe470 0x14646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6e94ff9700 0 client.7904 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x1426f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/874251701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x14641c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 >> 172.21.15.181:0/874251701 conn(0x13b3e40 msgr2=0x1429320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 shutdown_connections 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/874251701 wait complete. 2020-11-16T22:36:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.480+0000 7f6ecf3ad740 20 client.7904 trim_cache size 0 max 16384 2020-11-16T22:36:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 Processor -- start 2020-11-16T22:36:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 -- start start 2020-11-16T22:36:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 0x13ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x13fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1308e40 con 0x13b3e40 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x143aa90 con 0x1177bb0 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x143ac80 con 0x11e57b0 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6eb9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37846/0 (socket says 172.21.15.181:37846) 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/887368069 learned_addr learned my addr 172.21.15.181:0/887368069 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.496+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x13fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eba59c700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 0x13ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 msgr2=0x13fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x13fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/887368069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 msgr2=0x13ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 0x13ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 -- 172.21.15.181:0/887368069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840081d0 con 0x1177bb0 2020-11-16T22:36:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eba59c700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 0x13ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f6eac001e90 tx=0x7f6eac01d000).ready entity=mon.1 client_cookie=1ded40410e7677c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6e97fff700 1 -- 172.21.15.181:0/887368069 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac04d070 con 0x1177bb0 2020-11-16T22:36:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6e97fff700 1 -- 172.21.15.181:0/887368069 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac04e040 con 0x1177bb0 2020-11-16T22:36:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6e97fff700 1 -- 172.21.15.181:0/887368069 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac005bb0 con 0x1177bb0 2020-11-16T22:36:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 msgr2=0x13ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f6eac001e90 tx=0x7f6eac01d000).stop 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/887368069 shutdown_connections 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11e57b0 0x13fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x13b3e40 0x13ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/887368069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1177bb0 0x13ff5d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/887368069 >> 172.21.15.181:0/887368069 conn(0x7f6e84002320 msgr2=0x1453ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/887368069 shutdown_connections 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/887368069 wait complete. 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- start start 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 Processor -- start 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 0x13ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 0x14ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x143de30 con 0x1177bb0 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x13ff730 con 0x11e57b0 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x14b88f0 con 0x13b3e40 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ecf3ad740 10 client.0 ll_register_callbacks cb 0x143d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45696/0 (socket says 172.21.15.181:45696) 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2923216557 learned_addr learned my addr 172.21.15.181:0/2923216557 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 0x14ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eba59c700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 0x13ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 msgr2=0x14ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 0x14ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2923216557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 msgr2=0x13ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 0x13ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6ebad9d700 1 -- 172.21.15.181:0/2923216557 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1195720 con 0x13b3e40 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eb9d9b700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 0x14ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.500+0000 7f6eba59c700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 0x13ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ebad9d700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f6eb4009c10 tx=0x7f6eb4020040).ready entity=mon.2 client_cookie=f19e1c4ec03465d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb402e070 con 0x13b3e40 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eb4028620 con 0x13b3e40 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb401d660 con 0x13b3e40 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb40289f0 con 0x13b3e40 2020-11-16T22:36:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 10 client.7910 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x11a0e00 con 0x13b3e40 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb401d660 con 0x13b3e40 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f6eb4025020 con 0x13b3e40 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 10 client.7910 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 10 client.7910 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 20 client.7910 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x13fe470 0x14b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 4 client.7910 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1228f50 con 0x13fe470 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6eba59c700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x13fe470 0x14b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 10 client.7910 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6eba59c700 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x13fe470 0x14b51e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee4cd4c5b31a5829 server_cookie=5dc9f351f3e4b281 in_seq=0 out_seq=0 2020-11-16T22:36:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 1 -- 172.21.15.181:0/2923216557 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x1228f50 con 0x13fe470 2020-11-16T22:36:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 10 client.7910 handle_command_reply: tid=0 2020-11-16T22:36:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 client.7910 shutdown 2020-11-16T22:36:27.502 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 msgr2=0x143a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f6eb4009c10 tx=0x7f6eb4020040).stop 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 shutdown_connections 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x13fe470 0x14b51e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x13b3e40 0x143a170 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1177bb0 0x13ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 --2- 172.21.15.181:0/2923216557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11e57b0 0x14ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6e94ff9700 0 client.7910 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 >> 172.21.15.181:0/2923216557 conn(0x7f6e84002320 msgr2=0x1463690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 shutdown_connections 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 1 -- 172.21.15.181:0/2923216557 wait complete. 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ecf3ad740 20 client.7910 trim_cache size 0 max 16384 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 msgr2=0x7f6e90057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.504 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 0x7f6e90057520 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f6eb0003350 tx=0x7f6eb0007af0).stop 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 msgr2=0x7f6ebc0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f6eac00cab0 tx=0x7f6eac001e20).stop 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 shutdown_connections 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f6e90055060 0x7f6e90057520 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ebc15f5b0 0x7f6ebc0d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.505 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ebc0615f0 0x7f6ebc15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 --2- 172.21.15.181:0/1305058766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ebc0629b0 0x7f6ebc0d4970 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 >> 172.21.15.181:0/1305058766 conn(0x7f6ebc0d69a0 msgr2=0x7f6ebc061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 shutdown_connections 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.504+0000 7f6ec091c700 1 -- 172.21.15.181:0/1305058766 wait complete. 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:36:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:36:27.507 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:27.508 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:36:27.508 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:27.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 Processor -- start 2020-11-16T22:36:27.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- start start 2020-11-16T22:36:27.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb1d0164e10 0x7fb1d0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00c8db0 con 0x7fb1d0161860 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d00c8330 con 0x7fb1d0164e10 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00c85d0 con 0x7fb1d0161480 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45702/0 (socket says 172.21.15.181:45702) 2020-11-16T22:36:27.629 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 -- 172.21.15.181:0/2082566075 learned_addr learned my addr 172.21.15.181:0/2082566075 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.630 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cf7fe700 1 --2- 172.21.15.181:0/2082566075 >> v2:172.21.15.175:3300/0 conn(0x7fb1d0164e10 0x7fb1d0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 -- 172.21.15.181:0/2082566075 >> v2:172.21.15.175:3300/0 conn(0x7fb1d0164e10 msgr2=0x7fb1d0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 --2- 172.21.15.181:0/2082566075 >> v2:172.21.15.175:3300/0 conn(0x7fb1d0164e10 0x7fb1d0062250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 -- 172.21.15.181:0/2082566075 >> v1:172.21.15.175:6789/0 conn(0x7fb1d0161860 legacy=0x7fb1d0062740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 -- 172.21.15.181:0/2082566075 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00c8870 con 0x7fb1d0161480 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1cffff700 1 --2- 172.21.15.181:0/2082566075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fb1c0009d50 tx=0x7fb1c0009290).ready entity=mon.2 client_cookie=804b3b839535cda7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1ceffd700 1 -- 172.21.15.181:0/2082566075 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000bc80 con 0x7fb1d0161480 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1ceffd700 1 -- 172.21.15.181:0/2082566075 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c000bde0 con 0x7fb1d0161480 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1ceffd700 1 -- 172.21.15.181:0/2082566075 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0011610 con 0x7fb1d0161480 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- 172.21.15.181:0/2082566075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 msgr2=0x7fb1d0164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 --2- 172.21.15.181:0/2082566075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fb1c0009d50 tx=0x7fb1c0009290).stop 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- 172.21.15.181:0/2082566075 shutdown_connections 2020-11-16T22:36:27.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 --2- 172.21.15.181:0/2082566075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0164920 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 --2- 172.21.15.181:0/2082566075 >> v2:172.21.15.175:3300/0 conn(0x7fb1d0164e10 0x7fb1d0062250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- 172.21.15.181:0/2082566075 >> 172.21.15.181:0/2082566075 conn(0x7fb1d00d2dc0 msgr2=0x7fb1d00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- 172.21.15.181:0/2082566075 shutdown_connections 2020-11-16T22:36:27.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.640+0000 7fb1d5768700 1 -- 172.21.15.181:0/2082566075 wait complete. 2020-11-16T22:36:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 Processor -- start 2020-11-16T22:36:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 -- start start 2020-11-16T22:36:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 0x7fb1d01603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00cb1a0 con 0x7fb1d00cad40 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00cc490 con 0x7fb1d0161860 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0174e00 con 0x7fb1d0161480 2020-11-16T22:36:27.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45708/0 (socket says 172.21.15.181:45708) 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37868/0 (socket says 172.21.15.181:37868) 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cffff700 1 -- 172.21.15.181:0/1248998159 learned_addr learned my addr 172.21.15.181:0/1248998159 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d4f67700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 0x7fb1d01603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 -- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 msgr2=0x7fb1d0146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 -- 172.21.15.181:0/1248998159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 msgr2=0x7fb1d01603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 0x7fb1d01603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 -- 172.21.15.181:0/1248998159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d01750a0 con 0x7fb1d0161860 2020-11-16T22:36:27.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d4f67700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 0x7fb1d01603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:27.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1cf7fe700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fb1d0061220 tx=0x7fb1b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800f5e0 con 0x7fb1d0161860 2020-11-16T22:36:27.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800f740 con 0x7fb1d0161860 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.644+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d0175340 con 0x7fb1d0161860 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800fac0 con 0x7fb1d0161860 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d0175880 con 0x7fb1d0161860 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb1b801c460 con 0x7fb1d0161860 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1c7fff700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 0x7fb1ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0174c20 con 0x7fb1d0161860 2020-11-16T22:36:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb1b806dfe0 con 0x7fb1d0161860 2020-11-16T22:36:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1cffff700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 0x7fb1ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.648+0000 7fb1cffff700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 0x7fb1ac057520 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb1c000c010 tx=0x7fb1c0016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.656+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb1b80441e0 con 0x7fb1d0161860 2020-11-16T22:36:27.844 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d0175a10 con 0x7fb1d0161860 2020-11-16T22:36:27.845 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1c7fff700 1 -- 172.21.15.181:0/1248998159 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7fb1b8047800 con 0x7fb1d0161860 2020-11-16T22:36:27.845 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:27.846 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:27.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 msgr2=0x7fb1ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 0x7fb1ac057520 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb1c000c010 tx=0x7fb1c0016000).stop 2020-11-16T22:36:27.847 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 msgr2=0x7fb1d00ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.856+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fb1d0061220 tx=0x7fb1b8008040).stop 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 shutdown_connections 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb1ac055060 0x7fb1ac057520 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d00cad40 0x7fb1d01603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1d0161480 0x7fb1d0146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.848 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 --2- 172.21.15.181:0/1248998159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1d0161860 0x7fb1d00ca850 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 >> 172.21.15.181:0/1248998159 conn(0x7fb1d00d2dc0 msgr2=0x7fb1d00ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 shutdown_connections 2020-11-16T22:36:27.849 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.860+0000 7fb1d5768700 1 -- 172.21.15.181:0/1248998159 wait complete. 2020-11-16T22:36:27.849 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:27.859 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'i', 'e'] 2020-11-16T22:36:27.860 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 Processor -- start 2020-11-16T22:36:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 -- start start 2020-11-16T22:36:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36f014a180 0x7f36f014ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f00cef30 con 0x7f36f01657b0 2020-11-16T22:36:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f00cf1d0 con 0x7f36f0149da0 2020-11-16T22:36:27.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f5176700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f014a610 con 0x7f36f014a180 2020-11-16T22:36:27.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36effff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f36f0149da0 0x7f36f0061b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52178/0 (socket says 172.21.15.181:52178) 2020-11-16T22:36:27.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36effff700 1 -- 172.21.15.181:0/83289760 learned_addr learned my addr 172.21.15.181:0/83289760 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:27.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f4975700 1 --2- 172.21.15.181:0/83289760 >> v2:172.21.15.181:3300/0 conn(0x7f36f014a180 0x7f36f014ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941275884 0 0) 0x7f36f00cf1d0 con 0x7f36f0149da0 2020-11-16T22:36:27.979 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d0003240 con 0x7f36f0149da0 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563511717 0 0) 0x7f36f00cef30 con 0x7f36f01657b0 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f4975700 1 -- 172.21.15.181:0/83289760 >> v1:172.21.15.181:6789/0 conn(0x7f36f0149da0 legacy=0x7f36f0061b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f4975700 1 -- 172.21.15.181:0/83289760 >> v1:172.21.15.175:6789/0 conn(0x7f36f01657b0 legacy=0x7f36f00d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f4975700 1 -- 172.21.15.181:0/83289760 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00cf470 con 0x7f36f014a180 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36f4975700 1 --2- 172.21.15.181:0/83289760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f014ce60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f36e800b9a0 tx=0x7f36e8006df0).ready entity=mon.1 client_cookie=d99ada91a49f53e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80097b0 con 0x7f36f014a180 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8009910 con 0x7f36f014a180 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.988+0000 7f36eeffd700 1 -- 172.21.15.181:0/83289760 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8009c90 con 0x7f36f014a180 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- 172.21.15.181:0/83289760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 msgr2=0x7f36f014ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 --2- 172.21.15.181:0/83289760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f014ce60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f36e800b9a0 tx=0x7f36e8006df0).stop 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- 172.21.15.181:0/83289760 shutdown_connections 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 --2- 172.21.15.181:0/83289760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f014ce60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- 172.21.15.181:0/83289760 >> 172.21.15.181:0/83289760 conn(0x7f36f00d4590 msgr2=0x7f36f00d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- 172.21.15.181:0/83289760 shutdown_connections 2020-11-16T22:36:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- 172.21.15.181:0/83289760 wait complete. 2020-11-16T22:36:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 Processor -- start 2020-11-16T22:36:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.992+0000 7f36f5176700 1 -- start start 2020-11-16T22:36:27.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f0149da0 0x7f36f0173ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.985 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 0x7f36f00ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00d4400 con 0x7f36f0149da0 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00d3ec0 con 0x7f36f014a180 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00d4160 con 0x7f36f01657b0 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.986 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37882/0 (socket says 172.21.15.181:37882) 2020-11-16T22:36:27.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36effff700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f0149da0 0x7f36f0173ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 -- 172.21.15.181:0/1892322385 learned_addr learned my addr 172.21.15.181:0/1892322385 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:27.987 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f4975700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 0x7f36f00ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 -- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 msgr2=0x7f36f00ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 0x7f36f00ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 -- 172.21.15.181:0/1892322385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f0149da0 msgr2=0x7f36f0173ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:27.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f0149da0 0x7f36f0173ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:27.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 -- 172.21.15.181:0/1892322385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00c9600 con 0x7f36f014a180 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f4975700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 0x7f36f00ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36ef7fe700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f36d8007fd0 tx=0x7f36d8007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800e490 con 0x7f36f014a180 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800e5f0 con 0x7f36f014a180 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800e9b0 con 0x7f36f014a180 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f00c98a0 con 0x7f36f014a180 2020-11-16T22:36:27.989 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:27.996+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f01743e0 con 0x7f36f014a180 2020-11-16T22:36:27.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f0174660 con 0x7f36f014a180 2020-11-16T22:36:27.990 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f36d8019030 con 0x7f36f014a180 2020-11-16T22:36:27.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36e7fff700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 0x7f36cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:27.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36d806d400 con 0x7f36f014a180 2020-11-16T22:36:27.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36effff700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 0x7f36cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:27.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.000+0000 7f36effff700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 0x7f36cc057520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f36e0005920 tx=0x7f36e0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:27.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.008+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f36d8043550 con 0x7f36f014a180 2020-11-16T22:36:28.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36f0044f60 con 0x7f36f014a180 2020-11-16T22:36:28.204 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36e7fff700 1 -- 172.21.15.181:0/1892322385 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1814 (secure 0 0 0) 0x7f36d8046b70 con 0x7f36f014a180 2020-11-16T22:36:28.205 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:28.205 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":593,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":115880,"bytes_used":64602112,"bytes_avail":773029511168,"bytes_total":773094113280,"write_bytes_sec":426,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":125,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"e","status":"up:active","gid":8040},{"filesystem_id":7,"rank":1,"name":"i","status":"up:active","gid":8043},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8101}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":37,"modified":"2020-11-16T22:36:22.854229+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2020-11-16T22:36:28.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 msgr2=0x7f36cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 0x7f36cc057520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f36e0005920 tx=0x7f36e0009040).stop 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 msgr2=0x7f36f00cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f36d8007fd0 tx=0x7f36d8007ac0).stop 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 shutdown_connections 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f36cc055060 0x7f36cc057520 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36f01657b0 0x7f36f00ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f0149da0 0x7f36f0173ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 --2- 172.21.15.181:0/1892322385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36f014a180 0x7f36f00cddc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 >> 172.21.15.181:0/1892322385 conn(0x7f36f00d4590 msgr2=0x7f36f0061870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 shutdown_connections 2020-11-16T22:36:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.216+0000 7f36f5176700 1 -- 172.21.15.181:0/1892322385 wait complete. 2020-11-16T22:36:28.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/d1/d2/d3]', 'e': 'paused paths [/d1/d2/d3]', 'i': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:36:28.219 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:28.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 Processor -- start 2020-11-16T22:36:28.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 -- start start 2020-11-16T22:36:28.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfa0156a90 0x7fcfa0061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcfa015f3c0 0x7fcfa00cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00cccd0 con 0x7fcfa00615f0 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa00ccf70 con 0x7fcfa015f3c0 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcfa4a02700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa015e030 con 0x7fcfa0156a90 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfa0156a90 0x7fcfa0061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfa0156a90 0x7fcfa0061100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37888/0 (socket says 172.21.15.181:37888) 2020-11-16T22:36:28.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 -- 172.21.15.181:0/2802511942 learned_addr learned my addr 172.21.15.181:0/2802511942 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9ffff700 1 --2- 172.21.15.181:0/2802511942 >> v2:172.21.15.175:3300/0 conn(0x7fcfa015f3c0 0x7fcfa00cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 -- 172.21.15.181:0/2802511942 >> v2:172.21.15.175:3300/0 conn(0x7fcfa015f3c0 msgr2=0x7fcfa00cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 --2- 172.21.15.181:0/2802511942 >> v2:172.21.15.175:3300/0 conn(0x7fcfa015f3c0 0x7fcfa00cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 -- 172.21.15.181:0/2802511942 >> v1:172.21.15.175:6789/0 conn(0x7fcfa00615f0 legacy=0x7fcfa015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 -- 172.21.15.181:0/2802511942 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa015e2d0 con 0x7fcfa0156a90 2020-11-16T22:36:28.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9f7fe700 1 --2- 172.21.15.181:0/2802511942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa0061100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fcfa00675c0 tx=0x7fcf90006eb0).ready entity=mon.1 client_cookie=a855a53cfc52b47e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.356+0000 7fcf9e7fc700 1 -- 172.21.15.181:0/2802511942 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90009890 con 0x7fcfa0156a90 2020-11-16T22:36:28.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcf9e7fc700 1 -- 172.21.15.181:0/2802511942 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf900099f0 con 0x7fcfa0156a90 2020-11-16T22:36:28.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcf9e7fc700 1 -- 172.21.15.181:0/2802511942 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90009d70 con 0x7fcfa0156a90 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2802511942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 msgr2=0x7fcfa0061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2802511942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa0061100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fcfa00675c0 tx=0x7fcf90006eb0).stop 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2802511942 shutdown_connections 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2802511942 >> v2:172.21.15.175:3300/0 conn(0x7fcfa015f3c0 0x7fcfa00cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2802511942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa0061100 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2802511942 >> 172.21.15.181:0/2802511942 conn(0x7fcfa00cf630 msgr2=0x7fcfa00cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2802511942 shutdown_connections 2020-11-16T22:36:28.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2802511942 wait complete. 2020-11-16T22:36:28.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 Processor -- start 2020-11-16T22:36:28.351 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- start start 2020-11-16T22:36:28.352 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa00615f0 0x7fcfa00d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 0x7fcfa015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00ce3a0 con 0x7fcfa00615f0 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0158680 con 0x7fcfa0156a90 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcfa4a02700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0158920 con 0x7fcfa015f3c0 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcf9effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.360+0000 7fcf9effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37896/0 (socket says 172.21.15.181:37896) 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9effd700 1 -- 172.21.15.181:0/2865355337 learned_addr learned my addr 172.21.15.181:0/2865355337 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 0x7fcfa015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9f7fe700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa00615f0 0x7fcfa00d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 -- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 msgr2=0x7fcfa015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 -- 172.21.15.181:0/2865355337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa00615f0 msgr2=0x7fcfa00d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa00615f0 0x7fcfa00d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 -- 172.21.15.181:0/2865355337 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa0158bc0 con 0x7fcfa015f3c0 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9effd700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf9ffff700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 0x7fcfa015d1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fcf9800eb40 tx=0x7fcf98008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98017070 con 0x7fcfa015f3c0 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf980095b0 con 0x7fcfa015f3c0 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98009930 con 0x7fcfa015f3c0 2020-11-16T22:36:28.355 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa0158e60 con 0x7fcfa015f3c0 2020-11-16T22:36:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa00cad40 con 0x7fcfa015f3c0 2020-11-16T22:36:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa0158ff0 con 0x7fcfa015f3c0 2020-11-16T22:36:28.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.364+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcf98009d60 con 0x7fcfa015f3c0 2020-11-16T22:36:28.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.368+0000 7fcf7f7fe700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 0x7fcf78057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.368+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf9806d970 con 0x7fcfa015f3c0 2020-11-16T22:36:28.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.368+0000 7fcf9f7fe700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 0x7fcf78057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.357 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.368+0000 7fcf9f7fe700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 0x7fcf78057550 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fcf90006ef0 tx=0x7fcf90015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.376+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcf98043bd0 con 0x7fcfa015f3c0 2020-11-16T22:36:28.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfa0044f60 con 0x7fcfa015f3c0 2020-11-16T22:36:28.565 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcf7f7fe700 1 -- 172.21.15.181:0/2865355337 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7fcf98006e00 con 0x7fcfa015f3c0 2020-11-16T22:36:28.565 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:28.565 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 msgr2=0x7fcf78057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 0x7fcf78057550 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fcf90006ef0 tx=0x7fcf90015000).stop 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 msgr2=0x7fcfa015d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 0x7fcfa015d1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fcf9800eb40 tx=0x7fcf98008040).stop 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 shutdown_connections 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcf78055090 0x7fcf78057550 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.567 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f3c0 0x7fcfa015d1c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfa00615f0 0x7fcfa00d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 --2- 172.21.15.181:0/2865355337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa0156a90 0x7fcfa015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 >> 172.21.15.181:0/2865355337 conn(0x7fcfa00cf630 msgr2=0x7fcfa015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 shutdown_connections 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.576+0000 7fcfa4a02700 1 -- 172.21.15.181:0/2865355337 wait complete. 2020-11-16T22:36:28.568 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:28.580 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub resume 2020-11-16T22:36:28.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.716+0000 7f10dae96700 1 Processor -- start 2020-11-16T22:36:28.707 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.716+0000 7f10dae96700 1 -- start start 2020-11-16T22:36:28.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10d413e870 0x7f10d413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d4160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10d4061100 0x7f10d4061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.710 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d4160730 con 0x7f10d4061100 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d415d180 con 0x7f10d413e870 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10dae96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d413f140 con 0x7f10d415fdc0 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10d413e870 0x7f10d413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d4160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10d413e870 0x7f10d413ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37902/0 (socket says 172.21.15.181:37902) 2020-11-16T22:36:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 -- 172.21.15.181:0/4161271875 learned_addr learned my addr 172.21.15.181:0/4161271875 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10da695700 1 --2- 172.21.15.181:0/4161271875 >> v2:172.21.15.175:3300/0 conn(0x7f10d4061100 0x7f10d4061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 -- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 msgr2=0x7f10d4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 --2- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d4160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 -- 172.21.15.181:0/4161271875 >> v2:172.21.15.175:3300/0 conn(0x7f10d4061100 msgr2=0x7f10d4061590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 --2- 172.21.15.181:0/4161271875 >> v2:172.21.15.175:3300/0 conn(0x7f10d4061100 0x7f10d4061590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 -- 172.21.15.181:0/4161271875 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d415d810 con 0x7f10d413e870 2020-11-16T22:36:28.712 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d9e94700 1 --2- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d413ec50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f10d40675c0 tx=0x7f10d0009290).ready entity=mon.1 client_cookie=bd6dcef7f2a98b2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.713 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d8e92700 1 -- 172.21.15.181:0/4161271875 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d000bb80 con 0x7f10d413e870 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d8e92700 1 -- 172.21.15.181:0/4161271875 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d000bce0 con 0x7f10d413e870 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.720+0000 7f10d8e92700 1 -- 172.21.15.181:0/4161271875 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0011610 con 0x7f10d413e870 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 msgr2=0x7f10d413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 --2- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d413ec50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f10d40675c0 tx=0x7f10d0009290).stop 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- 172.21.15.181:0/4161271875 shutdown_connections 2020-11-16T22:36:28.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 --2- 172.21.15.181:0/4161271875 >> v2:172.21.15.175:3300/0 conn(0x7f10d4061100 0x7f10d4061590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 --2- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d413ec50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 --2- 172.21.15.181:0/4161271875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d4160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- 172.21.15.181:0/4161271875 >> 172.21.15.181:0/4161271875 conn(0x7f10d40cf630 msgr2=0x7f10d40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- 172.21.15.181:0/4161271875 shutdown_connections 2020-11-16T22:36:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- 172.21.15.181:0/4161271875 wait complete. 2020-11-16T22:36:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 Processor -- start 2020-11-16T22:36:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.724+0000 7f10dae96700 1 -- start start 2020-11-16T22:36:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 0x7f10d40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40cbdf0 con 0x7f10d4061100 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40cc090 con 0x7f10d413e870 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40cc330 con 0x7f10d415fdc0 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37908/0 (socket says 172.21.15.181:37908) 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 -- 172.21.15.181:0/1253462731 learned_addr learned my addr 172.21.15.181:0/1253462731 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10da695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 0x7f10d40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 -- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 msgr2=0x7f10d40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d40cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 -- 172.21.15.181:0/1253462731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 msgr2=0x7f10d40d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 0x7f10d40d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 -- 172.21.15.181:0/1253462731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40cc5d0 con 0x7f10d413e870 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 0x7f10d40d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9693700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f10c400eba0 tx=0x7f10c4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1253462731 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c400c9b0 con 0x7f10d413e870 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1253462731 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c400cb10 con 0x7f10d413e870 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d40b71f0 con 0x7f10d413e870 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1253462731 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c400ce90 con 0x7f10d413e870 2020-11-16T22:36:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d40cb860 con 0x7f10d413e870 2020-11-16T22:36:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1253462731 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f10c4012600 con 0x7f10d413e870 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 0x7f10ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10c1ffb700 1 -- 172.21.15.181:0/1253462731 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10c406e150 con 0x7f10d413e870 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.728+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 0x7f10ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 0x7f10ac057520 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f10d0003340 tx=0x7f10d0016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 Processor -- start 2020-11-16T22:36:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- start start 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x2683720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 0x267a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x250c4b0 con 0x24627b0 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c0930 con 0x23f4bb0 2020-11-16T22:36:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x24806f0 con 0x2680fd0 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37918/0 (socket says 172.21.15.181:37918) 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 -- 172.21.15.181:0/3677866266 learned_addr learned my addr 172.21.15.181:0/3677866266 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10da695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 0x267a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9e94700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x2683720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 -- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 msgr2=0x267a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 0x267a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 -- 172.21.15.181:0/3677866266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 msgr2=0x2683720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x2683720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 -- 172.21.15.181:0/3677866266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x247b5a0 con 0x23f4bb0 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9e94700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x2683720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10da695700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 0x267a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10d9693700 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f10c4047c80 tx=0x7f10c4014000).ready entity=mon.1 client_cookie=db103af020dd8c37 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10b2ffd700 1 -- 172.21.15.181:0/3677866266 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c406c3c0 con 0x23f4bb0 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10b2ffd700 1 -- 172.21.15.181:0/3677866266 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c406c520 con 0x23f4bb0 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10b2ffd700 1 -- 172.21.15.181:0/3677866266 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4047500 con 0x23f4bb0 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 msgr2=0x2681670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f10c4047c80 tx=0x7f10c4014000).stop 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- 172.21.15.181:0/3677866266 shutdown_connections 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2680fd0 0x267a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x2683720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 --2- 172.21.15.181:0/3677866266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x23f4bb0 0x2681670 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- 172.21.15.181:0/3677866266 >> 172.21.15.181:0/3677866266 conn(0x2630e40 msgr2=0x267d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- 172.21.15.181:0/3677866266 shutdown_connections 2020-11-16T22:36:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.732+0000 7f10eab89740 1 -- 172.21.15.181:0/3677866266 wait complete. 2020-11-16T22:36:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- start start 2020-11-16T22:36:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 Processor -- start 2020-11-16T22:36:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x267d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 0x26ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ad6e0 con 0x24627b0 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ad820 con 0x2680fd0 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x25aea90 con 0x23f4bb0 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 10 client.0 ll_register_callbacks cb 0x25a3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9e94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x267d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 0x26ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9e94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x267d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45764/0 (socket says 172.21.15.181:45764) 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9e94700 1 -- 172.21.15.181:0/3434932826 learned_addr learned my addr 172.21.15.181:0/3434932826 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9693700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 -- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 msgr2=0x267d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x267d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 -- 172.21.15.181:0/3434932826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 msgr2=0x26d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 -- 172.21.15.181:0/3434932826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x259e280 con 0x2680fd0 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9693700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10da695700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 0x26ce9b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f10cc013a10 tx=0x7f10cc008040).ready entity=mon.1 client_cookie=9a7b76ee558ed672 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc011990 con 0x2680fd0 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10cc011af0 con 0x2680fd0 2020-11-16T22:36:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26616c0 con 0x2680fd0 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10cc011ec0 con 0x2680fd0 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 10 client.8812 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x23d3fd0 con 0x2680fd0 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26616c0 con 0x2680fd0 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f10cc025020 con 0x2680fd0 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 10 client.8812 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 10 client.8812 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 20 client.8812 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x26d1c70 0x267fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 4 client.8812 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x249ec50 con 0x26d1c70 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9e94700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x26d1c70 0x267fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10a7fff700 10 client.8812 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.736+0000 7f10d9e94700 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x26d1c70 0x267fb10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=936e770fa4ddbbc0 server_cookie=2795be12918d14c0 in_seq=0 out_seq=0 2020-11-16T22:36:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10a7fff700 1 -- 172.21.15.181:0/3434932826 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x249ec50 con 0x26d1c70 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10a7fff700 10 client.8812 handle_command_reply: tid=0 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 client.8812 shutdown 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 msgr2=0x26ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 0x26ce9b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f10cc013a10 tx=0x7f10cc008040).stop 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 shutdown_connections 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x26d1c70 0x267fb10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2680fd0 0x26ce9b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x267d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10a7fff700 0 client.8812 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 --2- 172.21.15.181:0/3434932826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 >> 172.21.15.181:0/3434932826 conn(0x2630e40 msgr2=0x2682080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 shutdown_connections 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 1 -- 172.21.15.181:0/3434932826 wait complete. 2020-11-16T22:36:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.740+0000 7f10eab89740 20 client.8812 trim_cache size 0 max 16384 2020-11-16T22:36:28.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 Processor -- start 2020-11-16T22:36:28.744 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 -- start start 2020-11-16T22:36:28.745 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x26cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2585e40 con 0x24627b0 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x2661540 con 0x2630e40 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26b4520 con 0x23f4bb0 2020-11-16T22:36:28.746 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x26cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x26cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45776/0 (socket says 172.21.15.181:45776) 2020-11-16T22:36:28.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9693700 1 -- 172.21.15.181:0/1327876872 learned_addr learned my addr 172.21.15.181:0/1327876872 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37932/0 (socket says 172.21.15.181:37932) 2020-11-16T22:36:28.747 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10da695700 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.748 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 -- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 msgr2=0x26cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x26cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 -- 172.21.15.181:0/1327876872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 msgr2=0x26d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 -- 172.21.15.181:0/1327876872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00081d0 con 0x2630e40 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10da695700 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10d9e94700 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x267db60 tx=0x7f10d0014000).ready entity=mon.1 client_cookie=f90fb9e10aadafa6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.749 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10b37fe700 1 -- 172.21.15.181:0/1327876872 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d002e030 con 0x2630e40 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10b37fe700 1 -- 172.21.15.181:0/1327876872 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d002a0b0 con 0x2630e40 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.756+0000 7f10b37fe700 1 -- 172.21.15.181:0/1327876872 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d002a680 con 0x2630e40 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 msgr2=0x26cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x267db60 tx=0x7f10d0014000).stop 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- 172.21.15.181:0/1327876872 shutdown_connections 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x24627b0 0x26d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x26cbe40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- 172.21.15.181:0/1327876872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x23f4bb0 0x26cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- 172.21.15.181:0/1327876872 >> 172.21.15.181:0/1327876872 conn(0x7f10a0002320 msgr2=0x26cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- 172.21.15.181:0/1327876872 shutdown_connections 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- 172.21.15.181:0/1327876872 wait complete. 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- start start 2020-11-16T22:36:28.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 Processor -- start 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f4bb0 0x2733bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x27324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b1a90 con 0x23f4bb0 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b1d30 con 0x2630e40 2020-11-16T22:36:28.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x26e0d70 con 0x24627b0 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10eab89740 10 client.0 ll_register_callbacks cb 0x2681b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10da695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x27324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45780/0 (socket says 172.21.15.181:45780) 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 -- 172.21.15.181:0/363485159 learned_addr learned my addr 172.21.15.181:0/363485159 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9e94700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f4bb0 0x2733bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 -- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 msgr2=0x27324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x27324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 -- 172.21.15.181:0/363485159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f4bb0 msgr2=0x2733bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f4bb0 0x2733bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.760+0000 7f10d9693700 1 -- 172.21.15.181:0/363485159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2412720 con 0x24627b0 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10d9693700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x26b63b0 tx=0x7f10c406e000).ready entity=mon.2 client_cookie=5740a8454ecd94b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c406c3c0 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10c406c520 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10c4047500 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c406c920 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 10 client.7940 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x241de00 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c406c520 con 0x24627b0 2020-11-16T22:36:28.758 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f10c406ca80 con 0x24627b0 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 10 client.7940 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 10 client.7940 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 20 client.7940 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2680fd0 0x2732990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 4 client.7940 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,r,e,s,u,m,e,",}] 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b r e s u m e " }) v1 -- 0x24a5f50 con 0x2680fd0 2020-11-16T22:36:28.759 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10d9e94700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2680fd0 0x2732990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 10 client.7940 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10d9e94700 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2680fd0 0x2732990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8e9847aaa4b35498 server_cookie=777cd49ceae38f31 in_seq=0 out_seq=0 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 1 -- 172.21.15.181:0/363485159 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x24a5f50 con 0x2680fd0 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 10 client.7940 handle_command_reply: tid=0 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 client.7940 shutdown 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 msgr2=0x26d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.760 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x26b63b0 tx=0x7f10c406e000).stop 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 shutdown_connections 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2680fd0 0x2732990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2630e40 0x27324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10a7fff700 0 client.7940 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x23f4bb0 0x2733bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 --2- 172.21.15.181:0/363485159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x24627b0 0x26d0c40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 >> 172.21.15.181:0/363485159 conn(0x7f10a0002320 msgr2=0x267fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.761 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 shutdown_connections 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 1 -- 172.21.15.181:0/363485159 wait complete. 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10eab89740 20 client.7940 trim_cache size 0 max 16384 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 msgr2=0x7f10ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 0x7f10ac057520 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f10d0003340 tx=0x7f10d0016000).stop 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 msgr2=0x7f10d40d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f10c400eba0 tx=0x7f10c4006df0).stop 2020-11-16T22:36:28.762 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 shutdown_connections 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f10ac055060 0x7f10ac057520 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10d415fdc0 0x7f10d40cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d4061100 0x7f10d40d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 --2- 172.21.15.181:0/1253462731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d413e870 0x7f10d40d1c80 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 >> 172.21.15.181:0/1253462731 conn(0x7f10d40cf630 msgr2=0x7f10d40d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 shutdown_connections 2020-11-16T22:36:28.763 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.764+0000 7f10dae96700 1 -- 172.21.15.181:0/1253462731 wait complete. 2020-11-16T22:36:28.764 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:28.764 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0 2020-11-16T22:36:28.764 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:28.770 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:28.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.896+0000 7fcb12903700 1 Processor -- start 2020-11-16T22:36:28.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.896+0000 7fcb12903700 1 -- start start 2020-11-16T22:36:28.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0c0615f0 0x7fcb0c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 0x7fcb0c080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c160e40 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c1609a0 con 0x7fcb0c062790 2020-11-16T22:36:28.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c160ba0 con 0x7fcb0c0615f0 2020-11-16T22:36:28.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0c0615f0 0x7fcb0c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0c0615f0 0x7fcb0c0622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37944/0 (socket says 172.21.15.181:37944) 2020-11-16T22:36:28.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 -- 172.21.15.181:0/1910198744 learned_addr learned my addr 172.21.15.181:0/1910198744 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb10ea0700 1 --2- 172.21.15.181:0/1910198744 >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 0x7fcb0c080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 -- 172.21.15.181:0/1910198744 >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 msgr2=0x7fcb0c080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 --2- 172.21.15.181:0/1910198744 >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 0x7fcb0c080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 -- 172.21.15.181:0/1910198744 >> v1:172.21.15.175:6789/0 conn(0x7fcb0c15f5b0 legacy=0x7fcb0c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:28.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 -- 172.21.15.181:0/1910198744 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c060970 con 0x7fcb0c0615f0 2020-11-16T22:36:28.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb10ea0700 1 --2- 172.21.15.181:0/1910198744 >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 0x7fcb0c080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:28.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb037fe700 1 --2- 172.21.15.181:0/1910198744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c0622a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fcb0c0bcb90 tx=0x7fcaf4006eb0).ready entity=mon.1 client_cookie=496122c16ade624d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb0b7fe700 1 -- 172.21.15.181:0/1910198744 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4009890 con 0x7fcb0c0615f0 2020-11-16T22:36:28.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb0b7fe700 1 -- 172.21.15.181:0/1910198744 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40099f0 con 0x7fcb0c0615f0 2020-11-16T22:36:28.897 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb0b7fe700 1 -- 172.21.15.181:0/1910198744 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4009d70 con 0x7fcb0c0615f0 2020-11-16T22:36:28.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- 172.21.15.181:0/1910198744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 msgr2=0x7fcb0c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 --2- 172.21.15.181:0/1910198744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c0622a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fcb0c0bcb90 tx=0x7fcaf4006eb0).stop 2020-11-16T22:36:28.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- 172.21.15.181:0/1910198744 shutdown_connections 2020-11-16T22:36:28.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 --2- 172.21.15.181:0/1910198744 >> v2:172.21.15.175:3300/0 conn(0x7fcb0c062790 0x7fcb0c080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 --2- 172.21.15.181:0/1910198744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c0622a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- 172.21.15.181:0/1910198744 >> 172.21.15.181:0/1910198744 conn(0x7fcb0c0d69a0 msgr2=0x7fcb0c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:28.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- 172.21.15.181:0/1910198744 shutdown_connections 2020-11-16T22:36:28.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- 172.21.15.181:0/1910198744 wait complete. 2020-11-16T22:36:28.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 Processor -- start 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.900+0000 7fcb12903700 1 -- start start 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 0x7fcb0c163480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c145270 con 0x7fcb0c062790 2020-11-16T22:36:28.902 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c163fe0 con 0x7fcb0c0615f0 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c164280 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45794/0 (socket says 172.21.15.181:45794) 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb0bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37950/0 (socket says 172.21.15.181:37950) 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 -- 172.21.15.181:0/3947490017 learned_addr learned my addr 172.21.15.181:0/3947490017 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:28.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb037fe700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 0x7fcb0c163480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 -- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 msgr2=0x7fcb0c15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c15e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 -- 172.21.15.181:0/3947490017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 msgr2=0x7fcb0c163480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 0x7fcb0c163480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 -- 172.21.15.181:0/3947490017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c172910 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb037fe700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 0x7fcb0c163480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:28.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb10ea0700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fcb0400cca0 tx=0x7fcb04008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040079b0 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04007b10 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c172bb0 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007e90 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1730f0 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c069eb0 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.905 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcb04011620 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 0x7fcae8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:28.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb0406e050 con 0x7fcb0c15f5b0 2020-11-16T22:36:28.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.904+0000 7fcb0bfff700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 0x7fcae8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:28.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.908+0000 7fcb0bfff700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 0x7fcae8057550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fcafc00d350 tx=0x7fcafc007b30).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:28.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:28.916+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcb040442b0 con 0x7fcb0c15f5b0 2020-11-16T22:36:29.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.116+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb0c0600f0 con 0x7fcb0c15f5b0 2020-11-16T22:36:29.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb08ff9700 1 -- 172.21.15.181:0/3947490017 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7fcb040478d0 con 0x7fcb0c15f5b0 2020-11-16T22:36:29.108 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:29.108 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:29.109 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 msgr2=0x7fcae8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 0x7fcae8057550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fcafc00d350 tx=0x7fcafc007b30).stop 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 msgr2=0x7fcb0c163970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fcb0400cca0 tx=0x7fcb04008040).stop 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 shutdown_connections 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcae8055090 0x7fcae8057550 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0c15f5b0 0x7fcb0c163970 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0c0615f0 0x7fcb0c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 --2- 172.21.15.181:0/3947490017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb0c062790 0x7fcb0c163480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 >> 172.21.15.181:0/3947490017 conn(0x7fcb0c0d69a0 msgr2=0x7fcb0c061a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 shutdown_connections 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.120+0000 7fcb12903700 1 -- 172.21.15.181:0/3947490017 wait complete. 2020-11-16T22:36:29.111 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:29.123 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i scrub status 2020-11-16T22:36:29.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 Processor -- start 2020-11-16T22:36:29.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- start start 2020-11-16T22:36:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28d0161860 0x7f28d0062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28d0062740 0x7f28d00644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d00c8db0 con 0x7f28d0161480 2020-11-16T22:36:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d0165030 con 0x7f28d0062740 2020-11-16T22:36:29.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d00c8330 con 0x7f28d0161860 2020-11-16T22:36:29.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d5668700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f28d0161480 0x7f28d0164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52262/0 (socket says 172.21.15.181:52262) 2020-11-16T22:36:29.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d5668700 1 -- 172.21.15.181:0/2641752003 learned_addr learned my addr 172.21.15.181:0/2641752003 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:29.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 --2- 172.21.15.181:0/2641752003 >> v2:172.21.15.181:3300/0 conn(0x7f28d0161860 0x7f28d0062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d5e69700 1 --2- 172.21.15.181:0/2641752003 >> v2:172.21.15.175:3300/0 conn(0x7f28d0062740 0x7f28d00644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452168224 0 0) 0x7f28d00c8db0 con 0x7f28d0161480 2020-11-16T22:36:29.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b0003240 con 0x7f28d0161480 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 -- 172.21.15.181:0/2641752003 >> v2:172.21.15.175:3300/0 conn(0x7f28d0062740 msgr2=0x7f28d00644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 --2- 172.21.15.181:0/2641752003 >> v2:172.21.15.175:3300/0 conn(0x7f28d0062740 0x7f28d00644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 -- 172.21.15.181:0/2641752003 >> v1:172.21.15.181:6789/0 conn(0x7f28d0161480 legacy=0x7f28d0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3229131840 0 0) 0x7f28b0003240 con 0x7f28d0161480 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 -- 172.21.15.181:0/2641752003 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00c85d0 con 0x7f28d0161860 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d4e67700 1 --2- 172.21.15.181:0/2641752003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0161860 0x7f28d0062250 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f28d00ced60 tx=0x7f28c8006df0).ready entity=mon.1 client_cookie=fc885f69c842a7d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8001500 con 0x7f28d0161860 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c8011400 con 0x7f28d0161860 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28cffff700 1 -- 172.21.15.181:0/2641752003 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8011780 con 0x7f28d0161860 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- 172.21.15.181:0/2641752003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0161860 msgr2=0x7f28d0062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 --2- 172.21.15.181:0/2641752003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0161860 0x7f28d0062250 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f28d00ced60 tx=0x7f28c8006df0).stop 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.268+0000 7f28d666a700 1 -- 172.21.15.181:0/2641752003 shutdown_connections 2020-11-16T22:36:29.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 --2- 172.21.15.181:0/2641752003 >> v2:172.21.15.175:3300/0 conn(0x7f28d0062740 0x7f28d00644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 --2- 172.21.15.181:0/2641752003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0161860 0x7f28d0062250 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 -- 172.21.15.181:0/2641752003 >> 172.21.15.181:0/2641752003 conn(0x7f28d00d2dc0 msgr2=0x7f28d00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 -- 172.21.15.181:0/2641752003 shutdown_connections 2020-11-16T22:36:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 -- 172.21.15.181:0/2641752003 wait complete. 2020-11-16T22:36:29.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 Processor -- start 2020-11-16T22:36:29.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.272+0000 7f28d666a700 1 -- start start 2020-11-16T22:36:29.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 0x7f28d00cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d0161480 0x7f28d00ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 0x7f28d0146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0147e30 con 0x7f28d0161480 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00cd830 con 0x7f28d0062740 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00cafa0 con 0x7f28d0161860 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 0x7f28d00cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 0x7f28d0146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 0x7f28d00cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37966/0 (socket says 172.21.15.181:37966) 2020-11-16T22:36:29.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5668700 1 -- 172.21.15.181:0/4226404305 learned_addr learned my addr 172.21.15.181:0/4226404305 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:29.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d4e67700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d0161480 0x7f28d00ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.267 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 -- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 msgr2=0x7f28d00cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 0x7f28d00cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 -- 172.21.15.181:0/4226404305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d0161480 msgr2=0x7f28d00ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d0161480 0x7f28d00ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 -- 172.21.15.181:0/4226404305 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00cb240 con 0x7f28d0161860 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d5e69700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 0x7f28d0146110 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f28c000ed20 tx=0x7f28c00091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.268 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28cd7fa700 1 -- 172.21.15.181:0/4226404305 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000c9a0 con 0x7f28d0161860 2020-11-16T22:36:29.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28cd7fa700 1 -- 172.21.15.181:0/4226404305 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c000cb00 con 0x7f28d0161860 2020-11-16T22:36:29.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d0148060 con 0x7f28d0161860 2020-11-16T22:36:29.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28cd7fa700 1 -- 172.21.15.181:0/4226404305 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000ce80 con 0x7f28d0161860 2020-11-16T22:36:29.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.276+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d01485a0 con 0x7f28d0161860 2020-11-16T22:36:29.269 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.280+0000 7f28cd7fa700 1 -- 172.21.15.181:0/4226404305 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f28c0022070 con 0x7f28d0161860 2020-11-16T22:36:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.280+0000 7f28cd7fa700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 0x7f28ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.280+0000 7f28d5668700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 0x7f28ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.280+0000 7f28cd7fa700 1 -- 172.21.15.181:0/4226404305 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28c006e750 con 0x7f28d0161860 2020-11-16T22:36:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.280+0000 7f28d5668700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 0x7f28ac057520 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f28d00cbbc0 tx=0x7f28b80012d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 Processor -- start 2020-11-16T22:36:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 -- start start 2020-11-16T22:36:29.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11037b0 0x1324720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 0x1322670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ad4b0 con 0x1095bb0 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x1261930 con 0x1321fd0 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x11216f0 con 0x11037b0 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11037b0 0x1324720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37976/0 (socket says 172.21.15.181:37976) 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 -- 172.21.15.181:0/2041061235 learned_addr learned my addr 172.21.15.181:0/2041061235 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d4e67700 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 0x1322670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 -- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11037b0 msgr2=0x1324720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11037b0 0x1324720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 -- 172.21.15.181:0/2041061235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 msgr2=0x1322670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 0x1322670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 -- 172.21.15.181:0/2041061235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x111c5a0 con 0x1321fd0 2020-11-16T22:36:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d4e67700 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 0x1322670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:29.277 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28d5e69700 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f28c0047ff0 tx=0x7f28c0015000).ready entity=mon.1 client_cookie=1fc44e8aa2f0376b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.284+0000 7f28c67fc700 1 -- 172.21.15.181:0/2041061235 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0044e00 con 0x1321fd0 2020-11-16T22:36:29.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28c67fc700 1 -- 172.21.15.181:0/2041061235 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0029070 con 0x1321fd0 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28c67fc700 1 -- 172.21.15.181:0/2041061235 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0047c60 con 0x1321fd0 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 msgr2=0x131b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f28c0047ff0 tx=0x7f28c0015000).stop 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/2041061235 shutdown_connections 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1321fd0 0x131b700 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x11037b0 0x1324720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- 172.21.15.181:0/2041061235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1095bb0 0x1322670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/2041061235 >> 172.21.15.181:0/2041061235 conn(0x12d1e40 msgr2=0x131e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/2041061235 shutdown_connections 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/2041061235 wait complete. 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- start start 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 Processor -- start 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1321fd0 0x1321590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x124e6e0 con 0x1321fd0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x124e820 con 0x11037b0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x124fa90 con 0x1095bb0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 10 client.0 ll_register_callbacks cb 0x1244ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d4e67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.282 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d4e67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37982/0 (socket says 172.21.15.181:37982) 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45820/0 (socket says 172.21.15.181:45820) 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d4e67700 1 -- 172.21.15.181:0/3775034871 learned_addr learned my addr 172.21.15.181:0/3775034871 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5e69700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1321fd0 0x1321590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 -- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 msgr2=0x13210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 -- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1321fd0 msgr2=0x1321590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.283 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1321fd0 0x1321590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 -- 172.21.15.181:0/3775034871 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x123f280 con 0x1095bb0 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d4e67700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28d5668700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f28b8017a70 tx=0x7f28b8019040).ready entity=mon.2 client_cookie=66c394ed49c00529 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f289f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b801c040 con 0x1095bb0 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28b80139d0 con 0x1095bb0 2020-11-16T22:36:29.284 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13026c0 con 0x1095bb0 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28b8015740 con 0x1095bb0 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 10 client.7952 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.288+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x1074fd0 con 0x1095bb0 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13026c0 con 0x1095bb0 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f28b801f070 con 0x1095bb0 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 10 client.7952 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:29.285 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 10 client.7952 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 20 client.7952 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1376b50 0x13244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 4 client.7952 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:29.286 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x113fc50 con 0x1376b50 2020-11-16T22:36:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28d4e67700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1376b50 0x13244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.287 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28d4e67700 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1376b50 0x13244b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=98985aed92943938 server_cookie=59e89b7184c96621 in_seq=0 out_seq=0 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 10 client.7952 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 1 -- 172.21.15.181:0/3775034871 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x113fc50 con 0x1376b50 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 10 client.7952 handle_command_reply: tid=0 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 client.7952 shutdown 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 msgr2=0x133c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f28b8017a70 tx=0x7f28b8019040).stop 2020-11-16T22:36:29.288 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 shutdown_connections 2020-11-16T22:36:29.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1376b50 0x13244b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1321fd0 0x1321590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x1095bb0 0x133c110 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f289f7fe700 0 client.7952 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:29.289 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 --2- 172.21.15.181:0/3775034871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11037b0 0x13210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 >> 172.21.15.181:0/3775034871 conn(0x12d1e40 msgr2=0x1355270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 shutdown_connections 2020-11-16T22:36:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 1 -- 172.21.15.181:0/3775034871 wait complete. 2020-11-16T22:36:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.292+0000 7f28e635d740 20 client.7952 trim_cache size 0 max 16384 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 Processor -- start 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 -- start start 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1322fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1374820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1226e40 con 0x11037b0 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x131ec10 con 0x1095bb0 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13757a0 con 0x12d1e40 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d4e67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1322fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d4e67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37990/0 (socket says 172.21.15.181:37990) 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d5668700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1322fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45828/0 (socket says 172.21.15.181:45828) 2020-11-16T22:36:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d4e67700 1 -- 172.21.15.181:0/485772964 learned_addr learned my addr 172.21.15.181:0/485772964 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.312+0000 7f28d5e69700 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1374820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 -- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 msgr2=0x1322fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1322fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 -- 172.21.15.181:0/485772964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 msgr2=0x1374820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1374820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 -- 172.21.15.181:0/485772964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40081d0 con 0x1095bb0 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f28c8006fd0 tx=0x7f28c8027280).ready entity=mon.1 client_cookie=9387c61e9fa34330 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28c6ffd700 1 -- 172.21.15.181:0/485772964 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8026070 con 0x1095bb0 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28c6ffd700 1 -- 172.21.15.181:0/485772964 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c8011b80 con 0x1095bb0 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28c6ffd700 1 -- 172.21.15.181:0/485772964 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8020450 con 0x1095bb0 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 msgr2=0x131e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f28c8006fd0 tx=0x7f28c8027280).stop 2020-11-16T22:36:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- 172.21.15.181:0/485772964 shutdown_connections 2020-11-16T22:36:29.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1374820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.500 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1322fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- 172.21.15.181:0/485772964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x131e720 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- 172.21.15.181:0/485772964 >> 172.21.15.181:0/485772964 conn(0x7f28a4002320 msgr2=0x13244b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- 172.21.15.181:0/485772964 shutdown_connections 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- 172.21.15.181:0/485772964 wait complete. 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- start start 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 Processor -- start 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x13d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1351b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x1316fd0 con 0x11037b0 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x136bf70 con 0x1095bb0 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x13171c0 con 0x12d1e40 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28e635d740 10 client.0 ll_register_callbacks cb 0x1322af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45838/0 (socket says 172.21.15.181:45838) 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 -- 172.21.15.181:0/330684769 learned_addr learned my addr 172.21.15.181:0/330684769 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5668700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x13d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d4e67700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1351b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 -- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 msgr2=0x13d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x13d3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 -- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 msgr2=0x1351b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.316+0000 7f28d5e69700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1351b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d5e69700 1 -- 172.21.15.181:0/330684769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x10b3720 con 0x12d1e40 2020-11-16T22:36:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d5e69700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f28c00092c0 tx=0x7f28c001d070).ready entity=mon.2 client_cookie=e1a8d3ef9f448b56 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0044560 con 0x12d1e40 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28c0029070 con 0x12d1e40 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28c0005a50 con 0x12d1e40 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c00446c0 con 0x12d1e40 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 10 client.7958 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:29.706 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x10bee00 con 0x12d1e40 2020-11-16T22:36:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0029070 con 0x12d1e40 2020-11-16T22:36:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f28c006e0d0 con 0x12d1e40 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 10 client.7958 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 10 client.7958 resolve_mds: resolved ID 'i' to GID 8043 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 20 client.7958 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1321fd0 0x1353030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 4 client.7958 mds_command: new command op to 8043 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 --> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1146f50 con 0x1321fd0 2020-11-16T22:36:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d5668700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1321fd0 0x1353030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d5668700 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1321fd0 0x1353030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=27ad187d0b91b721 server_cookie=721d5772803c9be5 in_seq=0 out_seq=0 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 10 client.7958 ms_handle_connect on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 1 -- 172.21.15.181:0/330684769 <== mds.1 v2:172.21.15.175:6838/3163754567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1146f50 con 0x1321fd0 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 10 client.7958 handle_command_reply: tid=0 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 client.7958 shutdown 2020-11-16T22:36:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 msgr2=0x1352050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f28c00092c0 tx=0x7f28c001d070).stop 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 shutdown_connections 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:6838/3163754567,v1:172.21.15.175:6840/3163754567] conn(0x1321fd0 0x1353030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x12d1e40 0x1352050 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f289f7fe700 0 client.7958 ms_handle_reset on v2:172.21.15.175:6838/3163754567 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1095bb0 0x13d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 --2- 172.21.15.181:0/330684769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x11037b0 0x1351b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 >> 172.21.15.181:0/330684769 conn(0x7f28a4002320 msgr2=0x13244b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 shutdown_connections 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 1 -- 172.21.15.181:0/330684769 wait complete. 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28e635d740 20 client.7958 trim_cache size 0 max 16384 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 msgr2=0x7f28ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 0x7f28ac057520 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f28d00cbbc0 tx=0x7f28b80012d0).stop 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 msgr2=0x7f28d0146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 0x7f28d0146110 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f28c000ed20 tx=0x7f28c00091d0).stop 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 shutdown_connections 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f28ac055060 0x7f28ac057520 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.320+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28d0161860 0x7f28d0146110 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.324+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28d0062740 0x7f28d00cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.324+0000 7f28d666a700 1 --2- 172.21.15.181:0/4226404305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d0161480 0x7f28d00ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.324+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 >> 172.21.15.181:0/4226404305 conn(0x7f28d00d2dc0 msgr2=0x7f28d00d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.324+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 shutdown_connections 2020-11-16T22:36:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:29.324+0000 7f28d666a700 1 -- 172.21.15.181:0/4226404305 wait complete. 2020-11-16T22:36:29.916 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:29.916 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:36:29.917 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:36:29.917 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:29.917 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:30.044 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 Processor -- start 2020-11-16T22:36:30.045 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 -- start start 2020-11-16T22:36:30.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 0x7f9d8c066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c064050 con 0x7f9d8c0b6c20 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0642f0 con 0x7f9d8c15e270 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d93b69700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c064590 con 0x7f9d8c063b80 2020-11-16T22:36:30.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91905700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45842/0 (socket says 172.21.15.181:45842) 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91905700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38002/0 (socket says 172.21.15.181:38002) 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 -- 172.21.15.181:0/3479367873 learned_addr learned my addr 172.21.15.181:0/3479367873 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d92106700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 0x7f9d8c066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 -- 172.21.15.181:0/3479367873 >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 msgr2=0x7f9d8c15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 -- 172.21.15.181:0/3479367873 >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 msgr2=0x7f9d8c066ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 0x7f9d8c066ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 -- 172.21.15.181:0/3479367873 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c064830 con 0x7f9d8c063b80 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91905700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d92106700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 0x7f9d8c066ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:30.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.056+0000 7f9d91104700 1 --2- 172.21.15.181:0/3479367873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f9d8c0d46a0 tx=0x7f9d74006eb0).ready entity=mon.2 client_cookie=a61bd159402dc64d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d90903700 1 -- 172.21.15.181:0/3479367873 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74009810 con 0x7f9d8c063b80 2020-11-16T22:36:30.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d90903700 1 -- 172.21.15.181:0/3479367873 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74009970 con 0x7f9d8c063b80 2020-11-16T22:36:30.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d90903700 1 -- 172.21.15.181:0/3479367873 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74009cf0 con 0x7f9d8c063b80 2020-11-16T22:36:30.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- 172.21.15.181:0/3479367873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 msgr2=0x7f9d8c0667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3479367873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f9d8c0d46a0 tx=0x7f9d74006eb0).stop 2020-11-16T22:36:30.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- 172.21.15.181:0/3479367873 shutdown_connections 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.175:3300/0 conn(0x7f9d8c0b6c20 0x7f9d8c066ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3479367873 >> v2:172.21.15.181:3300/0 conn(0x7f9d8c15e270 0x7f9d8c15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3479367873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c063b80 0x7f9d8c0667b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- 172.21.15.181:0/3479367873 >> 172.21.15.181:0/3479367873 conn(0x7f9d8c0d4790 msgr2=0x7f9d8c0d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- 172.21.15.181:0/3479367873 shutdown_connections 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- 172.21.15.181:0/3479367873 wait complete. 2020-11-16T22:36:30.255 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 Processor -- start 2020-11-16T22:36:30.256 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.060+0000 7f9d93b69700 1 -- start start 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c063b80 0x7f9d8c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 0x7f9d8c1720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c170f10 con 0x7f9d8c0b6c20 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c1711b0 con 0x7f9d8c063b80 2020-11-16T22:36:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c171450 con 0x7f9d8c15e270 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d91905700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c063b80 0x7f9d8c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45850/0 (socket says 172.21.15.181:45850) 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 -- 172.21.15.181:0/3819139973 learned_addr learned my addr 172.21.15.181:0/3819139973 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d91104700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 0x7f9d8c1720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 -- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c063b80 msgr2=0x7f9d8c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c063b80 0x7f9d8c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 -- 172.21.15.181:0/3819139973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 msgr2=0x7f9d8c1720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 0x7f9d8c1720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 -- 172.21.15.181:0/3819139973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c1716f0 con 0x7f9d8c15e270 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d91104700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 0x7f9d8c1720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d92106700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f9d8400ebf0 tx=0x7f9d84008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400c030 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84009740 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c171990 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84009ac0 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c170af0 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c170d40 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.064+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9d84013430 con 0x7f9d8c15e270 2020-11-16T22:36:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.068+0000 7f9d89ffb700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 0x7f9d68057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.068+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d8406dc80 con 0x7f9d8c15e270 2020-11-16T22:36:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.068+0000 7f9d91905700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 0x7f9d68057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.068+0000 7f9d91905700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 0x7f9d68057520 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9d7c006fd0 tx=0x7f9d7c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.072+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9d84043e80 con 0x7f9d8c15e270 2020-11-16T22:36:30.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d8c064a10 con 0x7f9d8c15e270 2020-11-16T22:36:30.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d89ffb700 1 -- 172.21.15.181:0/3819139973 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f9d840474a0 con 0x7f9d8c15e270 2020-11-16T22:36:30.264 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:30.264 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:30.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 msgr2=0x7f9d68057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.467 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 0x7f9d68057520 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9d7c006fd0 tx=0x7f9d7c006df0).stop 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 msgr2=0x7f9d8c1725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f9d8400ebf0 tx=0x7f9d84008040).stop 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 shutdown_connections 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9d68055060 0x7f9d68057520 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d8c15e270 0x7f9d8c1725c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.468 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d8c063b80 0x7f9d8c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 --2- 172.21.15.181:0/3819139973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d8c0b6c20 0x7f9d8c1720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 >> 172.21.15.181:0/3819139973 conn(0x7f9d8c0d4790 msgr2=0x7f9d8c15c060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:30.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 shutdown_connections 2020-11-16T22:36:30.469 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.276+0000 7f9d93b69700 1 -- 172.21.15.181:0/3819139973 wait complete. 2020-11-16T22:36:30.469 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:30.470 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e scrub status 2020-11-16T22:36:30.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 Processor -- start 2020-11-16T22:36:30.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 -- start start 2020-11-16T22:36:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39a4062960 0x7f39a40644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a40c8db0 con 0x7f39a4161480 2020-11-16T22:36:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a4165030 con 0x7f39a4161860 2020-11-16T22:36:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9e7a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a40c8330 con 0x7f39a4062960 2020-11-16T22:36:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f39a4161860 0x7f39a4062250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52320/0 (socket says 172.21.15.181:52320) 2020-11-16T22:36:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a3fff700 1 -- 172.21.15.181:0/1021736709 learned_addr learned my addr 172.21.15.181:0/1021736709 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9679700 1 --2- 172.21.15.181:0/1021736709 >> v2:172.21.15.175:3300/0 conn(0x7f39a4062960 0x7f39a40644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a37fe700 1 -- 172.21.15.181:0/1021736709 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360669614 0 0) 0x7f39a4165030 con 0x7f39a4161860 2020-11-16T22:36:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a37fe700 1 -- 172.21.15.181:0/1021736709 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3984003240 con 0x7f39a4161860 2020-11-16T22:36:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9679700 1 -- 172.21.15.181:0/1021736709 >> v1:172.21.15.181:6789/0 conn(0x7f39a4161860 legacy=0x7f39a4062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9679700 1 -- 172.21.15.181:0/1021736709 >> v1:172.21.15.175:6789/0 conn(0x7f39a4161480 legacy=0x7f39a4164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9679700 1 -- 172.21.15.181:0/1021736709 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40c85d0 con 0x7f39a4062960 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a9679700 1 --2- 172.21.15.181:0/1021736709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4062960 0x7f39a40644e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f399400b9a0 tx=0x7f3994006df0).ready entity=mon.0 client_cookie=fc1374fdd281a020 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a37fe700 1 -- 172.21.15.181:0/1021736709 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39940097b0 con 0x7f39a4062960 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a37fe700 1 -- 172.21.15.181:0/1021736709 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994009910 con 0x7f39a4062960 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.620+0000 7f39a37fe700 1 -- 172.21.15.181:0/1021736709 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994009c90 con 0x7f39a4062960 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- 172.21.15.181:0/1021736709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4062960 msgr2=0x7f39a40644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/1021736709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4062960 0x7f39a40644e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f399400b9a0 tx=0x7f3994006df0).stop 2020-11-16T22:36:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- 172.21.15.181:0/1021736709 shutdown_connections 2020-11-16T22:36:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/1021736709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4062960 0x7f39a40644e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- 172.21.15.181:0/1021736709 >> 172.21.15.181:0/1021736709 conn(0x7f39a40d2dc0 msgr2=0x7f39a40ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- 172.21.15.181:0/1021736709 shutdown_connections 2020-11-16T22:36:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- 172.21.15.181:0/1021736709 wait complete. 2020-11-16T22:36:30.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 Processor -- start 2020-11-16T22:36:30.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- start start 2020-11-16T22:36:30.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39a4062960 0x7f39a40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4161860 0x7f39a40600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b3fc0 con 0x7f39a4161860 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4174a90 con 0x7f39a4161480 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9e7a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40cf7d0 con 0x7f39a4062960 2020-11-16T22:36:30.617 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38022/0 (socket says 172.21.15.181:38022) 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a8e78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39a4062960 0x7f39a40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a3fff700 1 -- 172.21.15.181:0/4200675264 learned_addr learned my addr 172.21.15.181:0/4200675264 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.624+0000 7f39a9679700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4161860 0x7f39a40600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 -- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39a4062960 msgr2=0x7f39a40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39a4062960 0x7f39a40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 -- 172.21.15.181:0/4200675264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4161860 msgr2=0x7f39a40600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4161860 0x7f39a40600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 -- 172.21.15.181:0/4200675264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4176e60 con 0x7f39a4161480 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a3fff700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f398c00ca90 tx=0x7f398c007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a0ff9700 1 -- 172.21.15.181:0/4200675264 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c005b80 con 0x7f39a4161480 2020-11-16T22:36:30.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a0ff9700 1 -- 172.21.15.181:0/4200675264 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c005ce0 con 0x7f39a4161480 2020-11-16T22:36:30.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a4177160 con 0x7f39a4161480 2020-11-16T22:36:30.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a0ff9700 1 -- 172.21.15.181:0/4200675264 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c007340 con 0x7f39a4161480 2020-11-16T22:36:30.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a41778e0 con 0x7f39a4161480 2020-11-16T22:36:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.628+0000 7f39a0ff9700 1 -- 172.21.15.181:0/4200675264 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f398c010030 con 0x7f39a4161480 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.632+0000 7f39a0ff9700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 0x7f3980057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.632+0000 7f39a0ff9700 1 -- 172.21.15.181:0/4200675264 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f398c06dbb0 con 0x7f39a4161480 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.632+0000 7f39a8e78700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 0x7f3980057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.632+0000 7f39a8e78700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 0x7f3980057550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f39a40c91e0 tx=0x7f399c007b50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 Processor -- start 2020-11-16T22:36:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 -- start start 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x1199720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x1197670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x10224b0 con 0xf0abb0 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d6930 con 0x1196fd0 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0xf966f0 con 0xf787b0 2020-11-16T22:36:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a8e78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x1199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38032/0 (socket says 172.21.15.181:38032) 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 -- 172.21.15.181:0/3158470499 learned_addr learned my addr 172.21.15.181:0/3158470499 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a3fff700 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x1197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 -- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 msgr2=0x1199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x1199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.834 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 -- 172.21.15.181:0/3158470499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 msgr2=0x1197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x1197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 -- 172.21.15.181:0/3158470499 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xf915a0 con 0x1196fd0 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f39a9679700 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f399400be60 tx=0x7f3994015000).ready entity=mon.1 client_cookie=7621c109eb8bac4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f3999ffb700 1 -- 172.21.15.181:0/3158470499 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994009a80 con 0x1196fd0 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f3999ffb700 1 -- 172.21.15.181:0/3158470499 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994009be0 con 0x1196fd0 2020-11-16T22:36:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.636+0000 7f3999ffb700 1 -- 172.21.15.181:0/3158470499 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994011820 con 0x1196fd0 2020-11-16T22:36:31.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 msgr2=0x1190700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f399400be60 tx=0x7f3994015000).stop 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3158470499 shutdown_connections 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x1190700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x1199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3158470499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x1197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3158470499 >> 172.21.15.181:0/3158470499 conn(0x1146e40 msgr2=0x11933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.051 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3158470499 shutdown_connections 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3158470499 wait complete. 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 -- start start 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.640+0000 7f39b9b6d740 1 Processor -- start 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x11b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.052 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c36e0 con 0xf0abb0 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c3820 con 0x1196fd0 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x10c4a90 con 0xf787b0 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 10 client.0 ll_register_callbacks cb 0x10b9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38038/0 (socket says 172.21.15.181:38038) 2020-11-16T22:36:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 -- 172.21.15.181:0/3760598463 learned_addr learned my addr 172.21.15.181:0/3760598463 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a3fff700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a8e78700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x11b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 -- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 msgr2=0x11930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 -- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 msgr2=0x11b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x11b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.054 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 -- 172.21.15.181:0/3760598463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x10b4280 con 0x1196fd0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a9679700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f399400b820 tx=0x7f3994021070).ready entity=mon.1 client_cookie=86ec2c79db095d8f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994009a80 con 0x1196fd0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3994011830 con 0x1196fd0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x11776c0 con 0x1196fd0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3994028880 con 0x1196fd0 2020-11-16T22:36:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 10 client.8863 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0xee9fd0 con 0x1196fd0 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x11776c0 con 0x1196fd0 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f39940289e0 con 0x1196fd0 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 10 client.8863 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 10 client.8863 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 20 client.8863 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x11f6160 0x1198a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 4 client.8863 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xfb4c50 con 0x11f6160 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a8e78700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x11f6160 0x1198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f3972ffd700 10 client.8863 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.644+0000 7f39a8e78700 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x11f6160 0x1198a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=2489569b57050294 server_cookie=2834c22d1e9dd48c in_seq=0 out_seq=0 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f3972ffd700 1 -- 172.21.15.181:0/3760598463 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xfb4c50 con 0x11f6160 2020-11-16T22:36:31.057 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f3972ffd700 10 client.8863 handle_command_reply: tid=0 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 client.8863 shutdown 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 msgr2=0x11935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f399400b820 tx=0x7f3994021070).stop 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 shutdown_connections 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x11f6160 0x1198a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1196fd0 0x11935e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.058 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f3972ffd700 0 client.8863 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0xf0abb0 0x11b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/3760598463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 >> 172.21.15.181:0/3760598463 conn(0x1146e40 msgr2=0x1198340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 shutdown_connections 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 1 -- 172.21.15.181:0/3760598463 wait complete. 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.648+0000 7f39b9b6d740 20 client.8863 trim_cache size 0 max 16384 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.668+0000 7f39b9b6d740 1 Processor -- start 2020-11-16T22:36:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.668+0000 7f39b9b6d740 1 -- start start 2020-11-16T22:36:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x11b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x109be40 con 0x1146e40 2020-11-16T22:36:31.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b9de0 con 0xf0abb0 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x1199980 con 0xf787b0 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45888/0 (socket says 172.21.15.181:45888) 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 -- 172.21.15.181:0/2082207019 learned_addr learned my addr 172.21.15.181:0/2082207019 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a8e78700 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x11b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.263 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 -- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 msgr2=0x11c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 -- 172.21.15.181:0/2082207019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 msgr2=0x11b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x11b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 -- 172.21.15.181:0/2082207019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780081d0 con 0xf787b0 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39a9679700 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f399400c010 tx=0x7f399402c040).ready entity=mon.2 client_cookie=5ade170fea013e6e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f399a7fc700 1 -- 172.21.15.181:0/2082207019 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994009a80 con 0xf787b0 2020-11-16T22:36:31.264 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f399a7fc700 1 -- 172.21.15.181:0/2082207019 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994028e40 con 0xf787b0 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f399a7fc700 1 -- 172.21.15.181:0/2082207019 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994028880 con 0xf787b0 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 msgr2=0x11e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f399400c010 tx=0x7f399402c040).stop 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- 172.21.15.181:0/2082207019 shutdown_connections 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x11e71f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x11b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.265 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/2082207019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- 172.21.15.181:0/2082207019 >> 172.21.15.181:0/2082207019 conn(0x7f3978002320 msgr2=0x1195da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- 172.21.15.181:0/2082207019 shutdown_connections 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- 172.21.15.181:0/2082207019 wait complete. 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 -- start start 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.672+0000 7f39b9b6d740 1 Processor -- start 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.266 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x124bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x1248220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x11e2570 con 0x1146e40 2020-11-16T22:36:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x11e2f80 con 0xf0abb0 2020-11-16T22:36:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x11e27a0 con 0xf787b0 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 10 client.0 ll_register_callbacks cb 0x11979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x124bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a8e78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a8e78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38048/0 (socket says 172.21.15.181:38048) 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a8e78700 1 -- 172.21.15.181:0/336287631 learned_addr learned my addr 172.21.15.181:0/336287631 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a9679700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x1248220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 -- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 msgr2=0x11e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 -- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 msgr2=0x1248220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x1248220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 -- 172.21.15.181:0/336287631 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xf28720 con 0xf787b0 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:31.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a3fff700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x124bf70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f398c0479a0 tx=0x7f398c011000).ready entity=mon.2 client_cookie=10e9c0f1014d6baa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c030070 con 0xf787b0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3994009be0 con 0xf787b0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f398c044180 con 0xf787b0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f398c044340 con 0xf787b0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 10 client.7979 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0xf33e00 con 0xf787b0 2020-11-16T22:36:31.477 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994009be0 con 0xf787b0 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7f398c043860 con 0xf787b0 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 10 client.7979 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 10 client.7979 resolve_mds: resolved ID 'e' to GID 8040 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 20 client.7979 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x1196fd0 0x11e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 4 client.7979 mds_command: new command op to 8040 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:31.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 --> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xfbbf50 con 0x1196fd0 2020-11-16T22:36:31.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a8e78700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x1196fd0 0x11e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 10 client.7979 ms_handle_connect on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:31.682 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39a8e78700 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x1196fd0 0x11e1b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=7dbdff391287fc59 server_cookie=3472c457e3a68289 in_seq=0 out_seq=0 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 1 -- 172.21.15.181:0/336287631 <== mds.2 v2:172.21.15.175:6835/722500686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xfbbf50 con 0x1196fd0 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 10 client.7979 handle_command_reply: tid=0 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 client.7979 shutdown 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 msgr2=0x124bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x124bf70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f398c0479a0 tx=0x7f398c011000).stop 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 shutdown_connections 2020-11-16T22:36:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:6835/722500686,v1:172.21.15.175:6837/722500686] conn(0x1196fd0 0x11e1b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x1146e40 0x1248220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f3972ffd700 0 client.7979 ms_handle_reset on v2:172.21.15.175:6835/722500686 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xf0abb0 0x11e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 --2- 172.21.15.181:0/336287631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0xf787b0 0x124bf70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 >> 172.21.15.181:0/336287631 conn(0x7f3978002320 msgr2=0x11c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 shutdown_connections 2020-11-16T22:36:31.684 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.676+0000 7f39b9b6d740 1 -- 172.21.15.181:0/336287631 wait complete. 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39b9b6d740 20 client.7979 trim_cache size 0 max 16384 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 msgr2=0x7f3980057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 0x7f3980057550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f39a40c91e0 tx=0x7f399c007b50).stop 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 msgr2=0x7f39a40cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f398c00ca90 tx=0x7f398c007da0).stop 2020-11-16T22:36:31.685 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 shutdown_connections 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3980055090 0x7f3980057550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a4161860 0x7f39a40600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39a4062960 0x7f39a40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 --2- 172.21.15.181:0/4200675264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39a4161480 0x7f39a40cf2e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 >> 172.21.15.181:0/4200675264 conn(0x7f39a40d2dc0 msgr2=0x7f39a4147390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 shutdown_connections 2020-11-16T22:36:31.686 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:30.680+0000 7f39a9e7a700 1 -- 172.21.15.181:0/4200675264 wait complete. 2020-11-16T22:36:31.687 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:31.687 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:36:31.687 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:36:31.687 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:31.687 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 Processor -- start 2020-11-16T22:36:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 -- start start 2020-11-16T22:36:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 0x7f997c145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 0x7f997c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c081f20 con 0x7f997c160450 2020-11-16T22:36:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c060470 con 0x7f997c0d2a30 2020-11-16T22:36:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.832+0000 7f998099e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c060710 con 0x7f997c145770 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f997c160450 0x7f997c0d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52360/0 (socket says 172.21.15.181:52360) 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997a59c700 1 -- 172.21.15.181:0/2830844544 learned_addr learned my addr 172.21.15.181:0/2830844544 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 --2- 172.21.15.181:0/2830844544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 0x7f997c145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997b59e700 1 --2- 172.21.15.181:0/2830844544 >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 0x7f997c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f9979d9b700 1 -- 172.21.15.181:0/2830844544 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45624928 0 0) 0x7f997c081f20 con 0x7f997c160450 2020-11-16T22:36:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f9979d9b700 1 -- 172.21.15.181:0/2830844544 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f995c003240 con 0x7f997c160450 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 -- 172.21.15.181:0/2830844544 >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 msgr2=0x7f997c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 --2- 172.21.15.181:0/2830844544 >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 0x7f997c1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 -- 172.21.15.181:0/2830844544 >> v1:172.21.15.181:6789/0 conn(0x7f997c160450 legacy=0x7f997c0d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 -- 172.21.15.181:0/2830844544 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c0609b0 con 0x7f997c145770 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997b59e700 1 --2- 172.21.15.181:0/2830844544 >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 0x7f997c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f997ad9d700 1 --2- 172.21.15.181:0/2830844544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 0x7f997c145b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f996c009c90 tx=0x7f996c0091d0).ready entity=mon.2 client_cookie=7b38e887bd7295f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f9979d9b700 1 -- 172.21.15.181:0/2830844544 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00ba70 con 0x7f997c145770 2020-11-16T22:36:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f9979d9b700 1 -- 172.21.15.181:0/2830844544 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00bbd0 con 0x7f997c145770 2020-11-16T22:36:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f9979d9b700 1 -- 172.21.15.181:0/2830844544 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c011400 con 0x7f997c145770 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 -- 172.21.15.181:0/2830844544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 msgr2=0x7f997c145b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 --2- 172.21.15.181:0/2830844544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 0x7f997c145b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f996c009c90 tx=0x7f996c0091d0).stop 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 -- 172.21.15.181:0/2830844544 shutdown_connections 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 --2- 172.21.15.181:0/2830844544 >> v2:172.21.15.175:3300/0 conn(0x7f997c0d2a30 0x7f997c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 --2- 172.21.15.181:0/2830844544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c145770 0x7f997c145b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 -- 172.21.15.181:0/2830844544 >> 172.21.15.181:0/2830844544 conn(0x7f997c0d6760 msgr2=0x7f997c0d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 -- 172.21.15.181:0/2830844544 shutdown_connections 2020-11-16T22:36:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.836+0000 7f998099e700 1 -- 172.21.15.181:0/2830844544 wait complete. 2020-11-16T22:36:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.840+0000 7f998099e700 1 Processor -- start 2020-11-16T22:36:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.840+0000 7f998099e700 1 -- start start 2020-11-16T22:36:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f997c0d2a30 0x7f997c15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 0x7f997c172bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c13de80 con 0x7f997c0d2a30 2020-11-16T22:36:32.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15c490 con 0x7f997c145770 2020-11-16T22:36:32.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15c730 con 0x7f997c160450 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38064/0 (socket says 172.21.15.181:38064) 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997a59c700 1 -- 172.21.15.181:0/471458522 learned_addr learned my addr 172.21.15.181:0/471458522 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 0x7f997c172bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997ad9d700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f997c0d2a30 0x7f997c15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 -- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 msgr2=0x7f997c1726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.035 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 -- 172.21.15.181:0/471458522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f997c0d2a30 msgr2=0x7f997c15aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f997c0d2a30 0x7f997c15aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 -- 172.21.15.181:0/471458522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c15c9d0 con 0x7f997c160450 2020-11-16T22:36:32.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997a59c700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:32.036 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f997b59e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 0x7f997c172bb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9974006fd0 tx=0x7f9974008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400f5e0 con 0x7f997c160450 2020-11-16T22:36:32.037 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400f740 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c15cc70 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400fae0 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c15b060 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c15b2e0 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.844+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f997401a030 con 0x7f997c160450 2020-11-16T22:36:32.038 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.848+0000 7f9972ffd700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 0x7f9958057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.848+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f997406d870 con 0x7f997c160450 2020-11-16T22:36:32.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.848+0000 7f997ad9d700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 0x7f9958057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.848+0000 7f997ad9d700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 0x7f9958057520 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f996c00b770 tx=0x7f996c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.039 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:31.852+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9974043ad0 con 0x7f997c160450 2020-11-16T22:36:32.046 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.056+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f997c044f60 con 0x7f997c160450 2020-11-16T22:36:32.047 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f9972ffd700 1 -- 172.21.15.181:0/471458522 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f99740470f0 con 0x7f997c160450 2020-11-16T22:36:32.047 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:32.047 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:32.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 msgr2=0x7f9958057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 0x7f9958057520 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f996c00b770 tx=0x7f996c016000).stop 2020-11-16T22:36:32.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 msgr2=0x7f997c172bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 0x7f997c172bb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9974006fd0 tx=0x7f9974008040).stop 2020-11-16T22:36:32.049 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 shutdown_connections 2020-11-16T22:36:32.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9958055060 0x7f9958057520 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f997c160450 0x7f997c172bb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.050 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f997c0d2a30 0x7f997c15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 --2- 172.21.15.181:0/471458522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f997c145770 0x7f997c1726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 >> 172.21.15.181:0/471458522 conn(0x7f997c0d6760 msgr2=0x7f997c0d5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 shutdown_connections 2020-11-16T22:36:32.254 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.060+0000 7f998099e700 1 -- 172.21.15.181:0/471458522 wait complete. 2020-11-16T22:36:32.255 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:32.256 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub status 2020-11-16T22:36:32.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.404+0000 7fe83ffff700 1 Processor -- start 2020-11-16T22:36:32.394 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.404+0000 7fe83ffff700 1 -- start start 2020-11-16T22:36:32.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400cf390 con 0x7fe8400d0a00 2020-11-16T22:36:32.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400cf5d0 con 0x7fe8400cef10 2020-11-16T22:36:32.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84015ebb0 con 0x7fe840140550 2020-11-16T22:36:32.396 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe8400cef10 0x7fe8400d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52378/0 (socket says 172.21.15.181:52378) 2020-11-16T22:36:32.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45912/0 (socket says 172.21.15.181:45912) 2020-11-16T22:36:32.397 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83e7fc700 1 -- 172.21.15.181:0/1619310040 learned_addr learned my addr 172.21.15.181:0/1619310040 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:32.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 -- 172.21.15.181:0/1619310040 >> v1:172.21.15.181:6789/0 conn(0x7fe8400cef10 legacy=0x7fe8400d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 -- 172.21.15.181:0/1619310040 >> v1:172.21.15.175:6789/0 conn(0x7fe8400d0a00 legacy=0x7fe840063ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 -- 172.21.15.181:0/1619310040 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400cf870 con 0x7fe840140550 2020-11-16T22:36:32.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83effd700 1 --2- 172.21.15.181:0/1619310040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fe82800b9d0 tx=0x7fe828006df0).ready entity=mon.2 client_cookie=bf7f2b46edeaa924 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83dffb700 1 -- 172.21.15.181:0/1619310040 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828009870 con 0x7fe840140550 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83dffb700 1 -- 172.21.15.181:0/1619310040 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8280099d0 con 0x7fe840140550 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83dffb700 1 -- 172.21.15.181:0/1619310040 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828009d50 con 0x7fe840140550 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- 172.21.15.181:0/1619310040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 msgr2=0x7fe840140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 --2- 172.21.15.181:0/1619310040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fe82800b9d0 tx=0x7fe828006df0).stop 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- 172.21.15.181:0/1619310040 shutdown_connections 2020-11-16T22:36:32.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 --2- 172.21.15.181:0/1619310040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe840140550 0x7fe840140930 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- 172.21.15.181:0/1619310040 >> 172.21.15.181:0/1619310040 conn(0x7fe8400d4570 msgr2=0x7fe8400d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- 172.21.15.181:0/1619310040 shutdown_connections 2020-11-16T22:36:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.408+0000 7fe83ffff700 1 -- 172.21.15.181:0/1619310040 wait complete. 2020-11-16T22:36:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 Processor -- start 2020-11-16T22:36:32.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 -- start start 2020-11-16T22:36:32.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe840140550 0x7fe8400ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84016d1e0 con 0x7fe840140550 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84016d430 con 0x7fe8400d0a00 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84016d6d0 con 0x7fe8400cef10 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45918/0 (socket says 172.21.15.181:45918) 2020-11-16T22:36:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38078/0 (socket says 172.21.15.181:38078) 2020-11-16T22:36:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83effd700 1 -- 172.21.15.181:0/2140271334 learned_addr learned my addr 172.21.15.181:0/2140271334 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.412+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe840140550 0x7fe8400ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 -- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 msgr2=0x7fe8400b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 -- 172.21.15.181:0/2140271334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe840140550 msgr2=0x7fe8400ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe840140550 0x7fe8400ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 -- 172.21.15.181:0/2140271334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84016d970 con 0x7fe8400cef10 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83effd700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fe82800b920 tx=0x7fe828015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 -- 172.21.15.181:0/2140271334 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828011e50 con 0x7fe8400cef10 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 -- 172.21.15.181:0/2140271334 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82801e460 con 0x7fe8400cef10 2020-11-16T22:36:32.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 -- 172.21.15.181:0/2140271334 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82801e800 con 0x7fe8400cef10 2020-11-16T22:36:32.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84016dc10 con 0x7fe8400cef10 2020-11-16T22:36:32.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84016e150 con 0x7fe8400cef10 2020-11-16T22:36:32.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 -- 172.21.15.181:0/2140271334 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe828025030 con 0x7fe8400cef10 2020-11-16T22:36:32.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 0x7fe81c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe836ffd700 1 -- 172.21.15.181:0/2140271334 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe82806f590 con 0x7fe8400cef10 2020-11-16T22:36:32.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.416+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 0x7fe81c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 0x7fe81c057520 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fe830007fd0 tx=0x7fe830007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 Processor -- start 2020-11-16T22:36:32.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 -- start start 2020-11-16T22:36:32.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x291e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x291e890 0x2953c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x27a94b0 con 0x26ff7b0 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x285d930 con 0x2691bb0 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x271d6f0 con 0x291e890 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38084/0 (socket says 172.21.15.181:38084) 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 -- 172.21.15.181:0/905958756 learned_addr learned my addr 172.21.15.181:0/905958756 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x291e890 0x2953c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83effd700 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x291e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 -- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x291e890 msgr2=0x2953c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x291e890 0x2953c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 -- 172.21.15.181:0/905958756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 msgr2=0x291e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x291e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.420+0000 7fe83e7fc700 1 -- 172.21.15.181:0/905958756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27185a0 con 0x2691bb0 2020-11-16T22:36:32.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fe830004bc0 tx=0x7fe83000a000).ready entity=mon.1 client_cookie=e9edd4db52af5a64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe817fff700 1 -- 172.21.15.181:0/905958756 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830017b10 con 0x2691bb0 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe817fff700 1 -- 172.21.15.181:0/905958756 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830017c70 con 0x2691bb0 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe817fff700 1 -- 172.21.15.181:0/905958756 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830014400 con 0x2691bb0 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 msgr2=0x2953730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fe830004bc0 tx=0x7fe83000a000).stop 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- 172.21.15.181:0/905958756 shutdown_connections 2020-11-16T22:36:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x291e890 0x2953c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x291e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- 172.21.15.181:0/905958756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2691bb0 0x2953730 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- 172.21.15.181:0/905958756 >> 172.21.15.181:0/905958756 conn(0x28cde40 msgr2=0x291a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- 172.21.15.181:0/905958756 shutdown_connections 2020-11-16T22:36:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- 172.21.15.181:0/905958756 wait complete. 2020-11-16T22:36:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- start start 2020-11-16T22:36:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 Processor -- start 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2917700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x2917bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x284a6e0 con 0x26ff7b0 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x284a820 con 0x291e890 2020-11-16T22:36:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x284ba90 con 0x2691bb0 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe853d95740 10 client.0 ll_register_callbacks cb 0x2840ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe83f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe83f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38094/0 (socket says 172.21.15.181:38094) 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.424+0000 7fe83f7fe700 1 -- 172.21.15.181:0/2842299102 learned_addr learned my addr 172.21.15.181:0/2842299102 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83effd700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2917700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x2917bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 -- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 msgr2=0x2917700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2917700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 -- 172.21.15.181:0/2842299102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 msgr2=0x2917bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x2917bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 -- 172.21.15.181:0/2842299102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x283b280 con 0x291e890 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x2917bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fe8380117f0 tx=0x7fe838007e20).ready entity=mon.1 client_cookie=a9d004638862eaa3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838002240 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe8380023a0 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28fe6c0 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8380017d0 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 10 client.8887 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x2670fd0 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28fe6c0 con 0x291e890 2020-11-16T22:36:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fe83801c020 con 0x291e890 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 10 client.8887 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 10 client.8887 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 20 client.8887 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2958050 0x2982050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 4 client.8887 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x273bc50 con 0x2958050 2020-11-16T22:36:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83effd700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2958050 0x2982050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 10 client.8887 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:32.423 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe83effd700 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2958050 0x2982050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c3815cd335e82705 server_cookie=ff8fa7d8626f2d0 in_seq=0 out_seq=0 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 1 -- 172.21.15.181:0/2842299102 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x273bc50 con 0x2958050 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 10 client.8887 handle_command_reply: tid=0 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 client.8887 shutdown 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 msgr2=0x2985240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fe8380117f0 tx=0x7fe838007e20).stop 2020-11-16T22:36:32.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 shutdown_connections 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x2958050 0x2982050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x291e890 0x2985240 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2917700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 --2- 172.21.15.181:0/2842299102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x2917bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 >> 172.21.15.181:0/2842299102 conn(0x28cde40 msgr2=0x2971f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe8157fa700 0 client.8887 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 shutdown_connections 2020-11-16T22:36:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 1 -- 172.21.15.181:0/2842299102 wait complete. 2020-11-16T22:36:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.428+0000 7fe853d95740 20 client.8887 trim_cache size 0 max 16384 2020-11-16T22:36:32.442 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.452+0000 7fe853d95740 1 Processor -- start 2020-11-16T22:36:32.443 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.452+0000 7fe853d95740 1 -- start start 2020-11-16T22:36:32.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 0x2912f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26ff7b0 0x291a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x2822e40 con 0x28cde40 2020-11-16T22:36:32.444 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x28fe260 con 0x26ff7b0 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x291d2d0 con 0x2691bb0 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45940/0 (socket says 172.21.15.181:45940) 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26ff7b0 0x291a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 -- 172.21.15.181:0/2497285335 learned_addr learned my addr 172.21.15.181:0/2497285335 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83effd700 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 0x2912f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.445 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 -- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26ff7b0 msgr2=0x291a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26ff7b0 0x291a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 -- 172.21.15.181:0/2497285335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 msgr2=0x2912f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 0x2912f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 -- 172.21.15.181:0/2497285335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0081c0 con 0x2691bb0 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83effd700 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 0x2912f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:32.446 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fe830015000 tx=0x7fe830014000).ready entity=mon.2 client_cookie=2d15fc07c8536236 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe834ff9700 1 -- 172.21.15.181:0/2497285335 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830028070 con 0x2691bb0 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe834ff9700 1 -- 172.21.15.181:0/2497285335 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830004760 con 0x2691bb0 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe834ff9700 1 -- 172.21.15.181:0/2497285335 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830014a00 con 0x2691bb0 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 msgr2=0x2919560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fe830015000 tx=0x7fe830014000).stop 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- 172.21.15.181:0/2497285335 shutdown_connections 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x26ff7b0 0x291a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.448 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x28cde40 0x2912f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 --2- 172.21.15.181:0/2497285335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2919560 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- 172.21.15.181:0/2497285335 >> 172.21.15.181:0/2497285335 conn(0x7fe80c002320 msgr2=0x293cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- 172.21.15.181:0/2497285335 shutdown_connections 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.456+0000 7fe853d95740 1 -- 172.21.15.181:0/2497285335 wait complete. 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- start start 2020-11-16T22:36:32.449 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 Processor -- start 2020-11-16T22:36:32.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x29d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28cde40 0x294e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x29d2e00 con 0x26ff7b0 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x29d8ca0 con 0x28cde40 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x29d8f40 con 0x2691bb0 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 10 client.0 ll_register_callbacks cb 0x29825c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:36:32.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45946/0 (socket says 172.21.15.181:45946) 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 -- 172.21.15.181:0/3456489325 learned_addr learned my addr 172.21.15.181:0/3456489325 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83f7fe700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28cde40 0x294e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x29d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 -- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28cde40 msgr2=0x294e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28cde40 0x294e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 -- 172.21.15.181:0/3456489325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 msgr2=0x29d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x29d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 -- 172.21.15.181:0/3456489325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26af720 con 0x2691bb0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe83effd700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fe828006900 tx=0x7fe82806f040).ready entity=mon.2 client_cookie=5dff3d46c824728a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe8157fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828048ce0 con 0x2691bb0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe82801eb70 con 0x2691bb0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe830017c70 con 0x2691bb0 2020-11-16T22:36:32.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe82806f0b0 con 0x2691bb0 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 10 client.8003 fetch_fsmap learned FSMap version 125 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.460+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x26bae00 con 0x2691bb0 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830017c70 con 0x2691bb0 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 125) v1 ==== 2611+0+0 (secure 0 0 0) 0x7fe82806f210 con 0x2691bb0 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 10 client.8003 fetch_fsmap finished waiting for FSMap version 125 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 10 client.8003 resolve_mds: resolved ID 'd' to GID 8101 2020-11-16T22:36:32.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 20 client.8003 populate_metadata read hostname 'smithi181' 2020-11-16T22:36:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x291e890 0x29126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 4 client.8003 mds_command: new command op to 8101 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:36:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 --> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x2742f50 con 0x291e890 2020-11-16T22:36:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x291e890 0x29126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 10 client.8003 ms_handle_connect on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83e7fc700 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x291e890 0x29126c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=aee24f195be7aa73 server_cookie=ee3b9b935ddb39a4 in_seq=0 out_seq=0 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 1 -- 172.21.15.181:0/3456489325 <== mds.0 v2:172.21.15.181:6836/2750102703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x2742f50 con 0x291e890 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 10 client.8003 handle_command_reply: tid=0 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 client.8003 shutdown 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 msgr2=0x2960980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fe828006900 tx=0x7fe82806f040).stop 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 shutdown_connections 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:6836/2750102703,v1:172.21.15.181:6837/2750102703] conn(0x291e890 0x29126c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x28cde40 0x294e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x2691bb0 0x2960980 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 --2- 172.21.15.181:0/3456489325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x26ff7b0 0x29d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 >> 172.21.15.181:0/3456489325 conn(0x7fe80c002320 msgr2=0x2919560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.459 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe8157fa700 0 client.8003 ms_handle_reset on v2:172.21.15.181:6836/2750102703 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 shutdown_connections 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 1 -- 172.21.15.181:0/3456489325 wait complete. 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe853d95740 20 client.8003 trim_cache size 0 max 16384 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 msgr2=0x7fe81c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 0x7fe81c057520 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fe830007fd0 tx=0x7fe830007b80).stop 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 msgr2=0x7fe84015d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fe82800b920 tx=0x7fe828015000).stop 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 shutdown_connections 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe81c055060 0x7fe81c057520 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe840140550 0x7fe8400ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8400cef10 0x7fe84015d760 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 --2- 172.21.15.181:0/2140271334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8400d0a00 0x7fe8400b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 >> 172.21.15.181:0/2140271334 conn(0x7fe8400d4570 msgr2=0x7fe8400b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 shutdown_connections 2020-11-16T22:36:32.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.464+0000 7fe83ffff700 1 -- 172.21.15.181:0/2140271334 wait complete. 2020-11-16T22:36:32.462 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:32.462 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2020-11-16T22:36:32.462 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {} 2020-11-16T22:36:32.463 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:32.469 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:36:32.470 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.608+0000 7f203d401700 1 Processor -- start 2020-11-16T22:36:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.608+0000 7f203d401700 1 -- start start 2020-11-16T22:36:32.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f203d401700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f203815de80 0x7f20380b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f203d401700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203815e2e0 con 0x7f203815ff80 2020-11-16T22:36:32.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f203d401700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380b2a60 con 0x7f20380b2190 2020-11-16T22:36:32.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f203d401700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20380ce460 con 0x7f203815de80 2020-11-16T22:36:32.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f2037fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f20380b2190 0x7f20380cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52416/0 (socket says 172.21.15.181:52416) 2020-11-16T22:36:32.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f2037fff700 1 -- 172.21.15.181:0/1066597052 learned_addr learned my addr 172.21.15.181:0/1066597052 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:32.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f20377fe700 1 --2- 172.21.15.181:0/1066597052 >> v2:172.21.15.175:3300/0 conn(0x7f203815de80 0x7f20380b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f2036ffd700 1 -- 172.21.15.181:0/1066597052 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57867909 0 0) 0x7f20380b2a60 con 0x7f20380b2190 2020-11-16T22:36:32.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f2036ffd700 1 -- 172.21.15.181:0/1066597052 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2024003240 con 0x7f20380b2190 2020-11-16T22:36:32.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f20377fe700 1 -- 172.21.15.181:0/1066597052 >> v1:172.21.15.181:6789/0 conn(0x7f20380b2190 legacy=0x7f20380cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f20377fe700 1 -- 172.21.15.181:0/1066597052 >> v1:172.21.15.175:6789/0 conn(0x7f203815ff80 legacy=0x7f2038160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f20377fe700 1 -- 172.21.15.181:0/1066597052 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380ce700 con 0x7f203815de80 2020-11-16T22:36:32.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.612+0000 7f20377fe700 1 --2- 172.21.15.181:0/1066597052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f20380b2570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f202c00b9a0 tx=0x7f202c006df0).ready entity=mon.0 client_cookie=37031f62fff77758 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f2036ffd700 1 -- 172.21.15.181:0/1066597052 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0097b0 con 0x7f203815de80 2020-11-16T22:36:32.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f2036ffd700 1 -- 172.21.15.181:0/1066597052 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c009910 con 0x7f203815de80 2020-11-16T22:36:32.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f2036ffd700 1 -- 172.21.15.181:0/1066597052 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c009c90 con 0x7f203815de80 2020-11-16T22:36:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 -- 172.21.15.181:0/1066597052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 msgr2=0x7f20380b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 --2- 172.21.15.181:0/1066597052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f20380b2570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f202c00b9a0 tx=0x7f202c006df0).stop 2020-11-16T22:36:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 -- 172.21.15.181:0/1066597052 shutdown_connections 2020-11-16T22:36:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 --2- 172.21.15.181:0/1066597052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f20380b2570 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 -- 172.21.15.181:0/1066597052 >> 172.21.15.181:0/1066597052 conn(0x7f20380cfe90 msgr2=0x7f20380d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 -- 172.21.15.181:0/1066597052 shutdown_connections 2020-11-16T22:36:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.616+0000 7f203d401700 1 -- 172.21.15.181:0/1066597052 wait complete. 2020-11-16T22:36:32.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 Processor -- start 2020-11-16T22:36:32.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 -- start start 2020-11-16T22:36:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f2038141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203815ff80 0x7f20380b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203815bfc0 con 0x7f203815de80 2020-11-16T22:36:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203815c1b0 con 0x7f20380b2190 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203d401700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203815c450 con 0x7f203815ff80 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f2037fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f2037fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38118/0 (socket says 172.21.15.181:38118) 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f2037fff700 1 -- 172.21.15.181:0/1326118335 learned_addr learned my addr 172.21.15.181:0/1326118335 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f203cc00700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203815ff80 0x7f20380b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.620+0000 7f20377fe700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f2038141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 -- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203815ff80 msgr2=0x7f20380b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203815ff80 0x7f20380b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 -- 172.21.15.181:0/1326118335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 msgr2=0x7f2038141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f2038141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 -- 172.21.15.181:0/1326118335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203815c6f0 con 0x7f20380b2190 2020-11-16T22:36:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2037fff700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f202800c710 tx=0x7f2028007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20280072e0 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028007440 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203815c990 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028005820 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203816bd50 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2038044f60 con 0x7f20380b2190 2020-11-16T22:36:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f202801a060 con 0x7f20380b2190 2020-11-16T22:36:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 0x7f2010057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f20377fe700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 0x7f2010057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f202806d4e0 con 0x7f20380b2190 2020-11-16T22:36:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.624+0000 7f20377fe700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 0x7f2010057520 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f202c00c010 tx=0x7f202c017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.632+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2028043740 con 0x7f20380b2190 2020-11-16T22:36:32.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.832+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f203805c4a0 con 0x7f20380b2190 2020-11-16T22:36:32.822 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.832+0000 7f2017fff700 1 -- 172.21.15.181:0/1326118335 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f2028018070 con 0x7f20380b2190 2020-11-16T22:36:32.823 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:32.823 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:32.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 msgr2=0x7f2010057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 0x7f2010057520 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f202c00c010 tx=0x7f202c017000).stop 2020-11-16T22:36:32.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 msgr2=0x7f203815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.825 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f202800c710 tx=0x7f2028007a40).stop 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 shutdown_connections 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2010055060 0x7f2010057520 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203815ff80 0x7f20380b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20380b2190 0x7f203815a510 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 --2- 172.21.15.181:0/1326118335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203815de80 0x7f2038141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 >> 172.21.15.181:0/1326118335 conn(0x7f20380cfe90 msgr2=0x7f20380600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 shutdown_connections 2020-11-16T22:36:32.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.836+0000 7f203d401700 1 -- 172.21.15.181:0/1326118335 wait complete. 2020-11-16T22:36:32.827 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:32.838 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'i', 'e'] 2020-11-16T22:36:32.838 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:32.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 Processor -- start 2020-11-16T22:36:32.960 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 -- start start 2020-11-16T22:36:32.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.961 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcd540600f0 0x7fcd540605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd54065150 con 0x7fcd5415d460 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd54066a20 con 0x7fcd540600f0 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5ac35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540653f0 con 0x7fcd5415d080 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45968/0 (socket says 172.21.15.181:45968) 2020-11-16T22:36:32.962 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 -- 172.21.15.181:0/4073069669 learned_addr learned my addr 172.21.15.181:0/4073069669 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.963 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd5a434700 1 --2- 172.21.15.181:0/4073069669 >> v2:172.21.15.175:3300/0 conn(0x7fcd540600f0 0x7fcd540605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 -- 172.21.15.181:0/4073069669 >> v2:172.21.15.175:3300/0 conn(0x7fcd540600f0 msgr2=0x7fcd540605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:32.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 --2- 172.21.15.181:0/4073069669 >> v2:172.21.15.175:3300/0 conn(0x7fcd540600f0 0x7fcd540605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 -- 172.21.15.181:0/4073069669 >> v1:172.21.15.175:6789/0 conn(0x7fcd5415d460 legacy=0x7fcd54066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.964 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 -- 172.21.15.181:0/4073069669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54065690 con 0x7fcd5415d080 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd59c33700 1 --2- 172.21.15.181:0/4073069669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fcd44006fd0 tx=0x7fcd44006eb0).ready entity=mon.2 client_cookie=cb95e13d0b98b3f3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd58c31700 1 -- 172.21.15.181:0/4073069669 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44009810 con 0x7fcd5415d080 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd58c31700 1 -- 172.21.15.181:0/4073069669 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44009970 con 0x7fcd5415d080 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.972+0000 7fcd58c31700 1 -- 172.21.15.181:0/4073069669 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44009cf0 con 0x7fcd5415d080 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- 172.21.15.181:0/4073069669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 msgr2=0x7fcd540cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.965 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/4073069669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fcd44006fd0 tx=0x7fcd44006eb0).stop 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- 172.21.15.181:0/4073069669 shutdown_connections 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/4073069669 >> v2:172.21.15.175:3300/0 conn(0x7fcd540600f0 0x7fcd540605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/4073069669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d080 0x7fcd540cdd90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- 172.21.15.181:0/4073069669 >> 172.21.15.181:0/4073069669 conn(0x7fcd540cfe90 msgr2=0x7fcd540d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- 172.21.15.181:0/4073069669 shutdown_connections 2020-11-16T22:36:32.966 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- 172.21.15.181:0/4073069669 wait complete. 2020-11-16T22:36:32.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 Processor -- start 2020-11-16T22:36:32.968 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- start start 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 0x7fcd5415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5415d080 0x7fcd541596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415a6f0 con 0x7fcd5415d080 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415a8e0 con 0x7fcd540600f0 2020-11-16T22:36:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5ac35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415ab80 con 0x7fcd5415d460 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd59c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 0x7fcd5415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd59c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 0x7fcd5415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38132/0 (socket says 172.21.15.181:38132) 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd59c33700 1 -- 172.21.15.181:0/645847924 learned_addr learned my addr 172.21.15.181:0/645847924 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45978/0 (socket says 172.21.15.181:45978) 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd59432700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5415d080 0x7fcd541596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 -- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 msgr2=0x7fcd5415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 0x7fcd5415a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 -- 172.21.15.181:0/645847924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5415d080 msgr2=0x7fcd541596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5415d080 0x7fcd541596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.976+0000 7fcd5a434700 1 -- 172.21.15.181:0/645847924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5415ae20 con 0x7fcd5415d460 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd5a434700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fcd4c00ed50 tx=0x7fcd4c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00cb00 con 0x7fcd5415d460 2020-11-16T22:36:32.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c00cc80 con 0x7fcd5415d460 2020-11-16T22:36:32.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5415b0c0 con 0x7fcd5415d460 2020-11-16T22:36:32.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0123f0 con 0x7fcd5415d460 2020-11-16T22:36:32.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5416ea30 con 0x7fcd5415d460 2020-11-16T22:36:32.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd54060f20 con 0x7fcd5415d460 2020-11-16T22:36:32.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcd4c022070 con 0x7fcd5415d460 2020-11-16T22:36:32.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 0x7fcd30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:32.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd4c06e400 con 0x7fcd5415d460 2020-11-16T22:36:32.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd59c33700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 0x7fcd30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:32.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.980+0000 7fcd59c33700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 0x7fcd30057520 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fcd4400b1b0 tx=0x7fcd44015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:32.976 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:32.988+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcd54060f20 con 0x7fcd5415d460 2020-11-16T22:36:33.179 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcd54060f20 con 0x7fcd5415d460 2020-11-16T22:36:33.180 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd51ffb700 1 -- 172.21.15.181:0/645847924 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1796 (secure 0 0 0) 0x7fcd54060f20 con 0x7fcd5415d460 2020-11-16T22:36:33.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:33.180 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":598,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":117477,"bytes_used":64602112,"bytes_avail":773029511168,"bytes_total":773094113280,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":125,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"e","status":"up:active","gid":8040},{"filesystem_id":7,"rank":1,"name":"i","status":"up:active","gid":8043},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8101}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":38,"modified":"2020-11-16T22:36:28.855963+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2020-11-16T22:36:33.182 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 msgr2=0x7fcd30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 0x7fcd30057520 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fcd4400b1b0 tx=0x7fcd44015000).stop 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 msgr2=0x7fcd54159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fcd4c00ed50 tx=0x7fcd4c0091d0).stop 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 shutdown_connections 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fcd30055060 0x7fcd30057520 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:33.183 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5415d460 0x7fcd54159bb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd540600f0 0x7fcd5415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 --2- 172.21.15.181:0/645847924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5415d080 0x7fcd541596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 >> 172.21.15.181:0/645847924 conn(0x7fcd540cfe90 msgr2=0x7fcd540d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 shutdown_connections 2020-11-16T22:36:33.184 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:33.192+0000 7fcd5ac35700 1 -- 172.21.15.181:0/645847924 wait complete. 2020-11-16T22:36:33.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/d1/d2/d3]', 'e': 'paused paths [/d1/d2/d3]', 'i': 'paused'} 2020-11-16T22:36:34.196 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:34.328 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 Processor -- start 2020-11-16T22:36:34.329 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 -- start start 2020-11-16T22:36:34.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff00644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.330 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00c8db0 con 0x7f2ff0161860 2020-11-16T22:36:34.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff0165030 con 0x7f2ff0161480 2020-11-16T22:36:34.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff64b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00c8330 con 0x7f2ff0062960 2020-11-16T22:36:34.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff54b6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2ff0161480 0x7f2ff0164920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52444/0 (socket says 172.21.15.181:52444) 2020-11-16T22:36:34.331 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff54b6700 1 -- 172.21.15.181:0/378102022 learned_addr learned my addr 172.21.15.181:0/378102022 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:34.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff5cb7700 1 --2- 172.21.15.181:0/378102022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff00644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff5cb7700 1 -- 172.21.15.181:0/378102022 >> v1:172.21.15.181:6789/0 conn(0x7f2ff0161480 legacy=0x7f2ff0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff5cb7700 1 -- 172.21.15.181:0/378102022 >> v1:172.21.15.175:6789/0 conn(0x7f2ff0161860 legacy=0x7f2ff0062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.332 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2feffff700 1 -- 172.21.15.181:0/378102022 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464129890 0 0) 0x7f2ff0165030 con 0x7f2ff0161480 2020-11-16T22:36:34.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff5cb7700 1 -- 172.21.15.181:0/378102022 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00c85d0 con 0x7f2ff0062960 2020-11-16T22:36:34.333 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2ff5cb7700 1 --2- 172.21.15.181:0/378102022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff00644e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f2fe8009cc0 tx=0x7f2fe80091d0).ready entity=mon.2 client_cookie=4d8b95a957f055a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:34.334 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2feffff700 1 -- 172.21.15.181:0/378102022 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800bbc0 con 0x7f2ff0062960 2020-11-16T22:36:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.339+0000 7f2feffff700 1 -- 172.21.15.181:0/378102022 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe800bd20 con 0x7f2ff0062960 2020-11-16T22:36:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2feffff700 1 -- 172.21.15.181:0/378102022 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011610 con 0x7f2ff0062960 2020-11-16T22:36:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- 172.21.15.181:0/378102022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 msgr2=0x7f2ff00644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/378102022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff00644e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f2fe8009cc0 tx=0x7f2fe80091d0).stop 2020-11-16T22:36:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- 172.21.15.181:0/378102022 shutdown_connections 2020-11-16T22:36:34.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/378102022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff00644e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- 172.21.15.181:0/378102022 >> 172.21.15.181:0/378102022 conn(0x7f2ff00d2dc0 msgr2=0x7f2ff00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:34.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- 172.21.15.181:0/378102022 shutdown_connections 2020-11-16T22:36:34.336 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- 172.21.15.181:0/378102022 wait complete. 2020-11-16T22:36:34.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 Processor -- start 2020-11-16T22:36:34.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.343+0000 7f2ff64b8700 1 -- start start 2020-11-16T22:36:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ff0161480 0x7f2ff00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 0x7f2ff00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00cc2f0 con 0x7f2ff0161860 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0168980 con 0x7f2ff0161480 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00cb870 con 0x7f2ff0062960 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:45988/0 (socket says 172.21.15.181:45988) 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 -- 172.21.15.181:0/763335185 learned_addr learned my addr 172.21.15.181:0/763335185 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff4cb5700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ff0161480 0x7f2ff00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff5cb7700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 0x7f2ff00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 -- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ff0161480 msgr2=0x7f2ff00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ff0161480 0x7f2ff00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 -- 172.21.15.181:0/763335185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 msgr2=0x7f2ff00cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 0x7f2ff00cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 -- 172.21.15.181:0/763335185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00cbb10 con 0x7f2ff0062960 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff5cb7700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 0x7f2ff00cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff54b6700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f2fe000c790 tx=0x7f2fe0007a60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00072e0 con 0x7f2ff0062960 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0007440 con 0x7f2ff0062960 2020-11-16T22:36:34.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0005840 con 0x7f2ff0062960 2020-11-16T22:36:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff00cbdb0 con 0x7f2ff0062960 2020-11-16T22:36:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff015e540 con 0x7f2ff0062960 2020-11-16T22:36:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00cb6a0 con 0x7f2ff0062960 2020-11-16T22:36:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.347+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2fe0010070 con 0x7f2ff0062960 2020-11-16T22:36:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.351+0000 7f2fed7fa700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 0x7f2fc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.351+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2fe006d440 con 0x7f2ff0062960 2020-11-16T22:36:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.351+0000 7f2ff4cb5700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 0x7f2fc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.351+0000 7f2ff4cb5700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 0x7f2fc8057520 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2ff0066f80 tx=0x7f2fe4007ac0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.355+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2fe00436a0 con 0x7f2ff0062960 2020-11-16T22:36:34.544 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.555+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ff0044f60 con 0x7f2ff0062960 2020-11-16T22:36:34.545 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.555+0000 7f2fed7fa700 1 -- 172.21.15.181:0/763335185 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f2fe0046cc0 con 0x7f2ff0062960 2020-11-16T22:36:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:34.546 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 msgr2=0x7f2fc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 0x7f2fc8057520 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2ff0066f80 tx=0x7f2fe4007ac0).stop 2020-11-16T22:36:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 msgr2=0x7f2ff015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f2fe000c790 tx=0x7f2fe0007a60).stop 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 shutdown_connections 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f2fc8055060 0x7f2fc8057520 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff0161860 0x7f2ff00cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ff0062960 0x7f2ff015e050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 --2- 172.21.15.181:0/763335185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ff0161480 0x7f2ff00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 >> 172.21.15.181:0/763335185 conn(0x7f2ff00d2dc0 msgr2=0x7f2ff016a040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 shutdown_connections 2020-11-16T22:36:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.559+0000 7f2ff64b8700 1 -- 172.21.15.181:0/763335185 wait complete. 2020-11-16T22:36:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:34.753 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'i', 'e'] 2020-11-16T22:36:34.754 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:34.771 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:34.777 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:34.779 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:36:34.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 Processor -- start 2020-11-16T22:36:34.884 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 -- start start 2020-11-16T22:36:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9fc00816e0 0x7f9fc0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fc01492c0 0x7f9fc00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc0061b50 con 0x7f9fc0145a30 2020-11-16T22:36:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc00618b0 con 0x7f9fc00816e0 2020-11-16T22:36:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc00cb930 con 0x7f9fc01492c0 2020-11-16T22:36:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9fc0145a30 0x7f9fc0080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52458/0 (socket says 172.21.15.181:52458) 2020-11-16T22:36:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbeffd700 1 -- 172.21.15.181:0/2186505006 learned_addr learned my addr 172.21.15.181:0/2186505006 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 --2- 172.21.15.181:0/2186505006 >> v2:172.21.15.181:3300/0 conn(0x7f9fc01492c0 0x7f9fc00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbe7fc700 1 --2- 172.21.15.181:0/2186505006 >> v2:172.21.15.175:3300/0 conn(0x7f9fc00816e0 0x7f9fc0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbdffb700 1 -- 172.21.15.181:0/2186505006 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751346537 0 0) 0x7f9fc0061b50 con 0x7f9fc0145a30 2020-11-16T22:36:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbdffb700 1 -- 172.21.15.181:0/2186505006 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fac003240 con 0x7f9fc0145a30 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 -- 172.21.15.181:0/2186505006 >> v2:172.21.15.175:3300/0 conn(0x7f9fc00816e0 msgr2=0x7f9fc0145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 --2- 172.21.15.181:0/2186505006 >> v2:172.21.15.175:3300/0 conn(0x7f9fc00816e0 0x7f9fc0145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 -- 172.21.15.181:0/2186505006 >> v1:172.21.15.181:6789/0 conn(0x7f9fc0145a30 legacy=0x7f9fc0080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 -- 172.21.15.181:0/2186505006 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00cbbd0 con 0x7f9fc01492c0 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbf7fe700 1 --2- 172.21.15.181:0/2186505006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc01492c0 0x7f9fc00cc5b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f9fc00d78a0 tx=0x7f9fb0006df0).ready entity=mon.1 client_cookie=11ec9b994204dae9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.895+0000 7f9fbdffb700 1 -- 172.21.15.181:0/2186505006 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009640 con 0x7f9fc01492c0 2020-11-16T22:36:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbdffb700 1 -- 172.21.15.181:0/2186505006 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb00097a0 con 0x7f9fc01492c0 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbdffb700 1 -- 172.21.15.181:0/2186505006 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009b20 con 0x7f9fc01492c0 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- 172.21.15.181:0/2186505006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc01492c0 msgr2=0x7f9fc00cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2186505006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc01492c0 0x7f9fc00cc5b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f9fc00d78a0 tx=0x7f9fb0006df0).stop 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- 172.21.15.181:0/2186505006 shutdown_connections 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2186505006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc01492c0 0x7f9fc00cc5b0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2186505006 >> v2:172.21.15.175:3300/0 conn(0x7f9fc00816e0 0x7f9fc0145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- 172.21.15.181:0/2186505006 >> 172.21.15.181:0/2186505006 conn(0x7f9fc00d68c0 msgr2=0x7f9fc00d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- 172.21.15.181:0/2186505006 shutdown_connections 2020-11-16T22:36:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- 172.21.15.181:0/2186505006 wait complete. 2020-11-16T22:36:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 Processor -- start 2020-11-16T22:36:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- start start 2020-11-16T22:36:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 0x7f9fc0143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 0x7f9fc00d4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:35.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc01492c0 0x7f9fc00d5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:35.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc016f620 con 0x7f9fc01492c0 2020-11-16T22:36:35.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc016f8c0 con 0x7f9fc0145a30 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc016fb60 con 0x7f9fc00816e0 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 0x7f9fc00d4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 0x7f9fc0143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 0x7f9fc00d4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38162/0 (socket says 172.21.15.181:38162) 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbe7fc700 1 -- 172.21.15.181:0/2733461271 learned_addr learned my addr 172.21.15.181:0/2733461271 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbf7fe700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc01492c0 0x7f9fc00d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:35.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 -- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 msgr2=0x7f9fc00d4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:35.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 0x7f9fc00d4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 -- 172.21.15.181:0/2733461271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc01492c0 msgr2=0x7f9fc00d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:35.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc01492c0 0x7f9fc00d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.899+0000 7f9fbeffd700 1 -- 172.21.15.181:0/2733461271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc016fe00 con 0x7f9fc00816e0 2020-11-16T22:36:35.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fbeffd700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 0x7f9fc0143cf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f9fb800ca60 tx=0x7f9fb8007d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:35.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800ce30 con 0x7f9fc00816e0 2020-11-16T22:36:35.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8005b80 con 0x7f9fc00816e0 2020-11-16T22:36:35.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc01700a0 con 0x7f9fc00816e0 2020-11-16T22:36:35.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8007110 con 0x7f9fc00816e0 2020-11-16T22:36:35.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc01441e0 con 0x7f9fc00816e0 2020-11-16T22:36:35.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc0044f60 con 0x7f9fc00816e0 2020-11-16T22:36:35.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9fb8010030 con 0x7f9fc00816e0 2020-11-16T22:36:35.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 0x7f9f9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:35.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9fb806dce0 con 0x7f9fc00816e0 2020-11-16T22:36:35.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.903+0000 7f9fbe7fc700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 0x7f9f9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:35.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.907+0000 7f9fbe7fc700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 0x7f9f9c057520 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f9fc0060fe0 tx=0x7f9fb4006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:35.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:34.911+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9fb8043f40 con 0x7f9fc00816e0 2020-11-16T22:36:35.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.115+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9fc0170230 con 0x7f9fc00816e0 2020-11-16T22:36:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.115+0000 7f9fa6ffd700 1 -- 172.21.15.181:0/2733461271 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1796 (secure 0 0 0) 0x7f9fb8047560 con 0x7f9fc00816e0 2020-11-16T22:36:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 msgr2=0x7f9f9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 0x7f9f9c057520 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f9fc0060fe0 tx=0x7f9fb4006df0).stop 2020-11-16T22:36:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 msgr2=0x7f9fc0143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 0x7f9fc0143cf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f9fb800ca60 tx=0x7f9fb8007d60).stop 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 shutdown_connections 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f9f9c055060 0x7f9f9c057520 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fc01492c0 0x7f9fc00d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fc00816e0 0x7f9fc0143cf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 --2- 172.21.15.181:0/2733461271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fc0145a30 0x7f9fc00d4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 >> 172.21.15.181:0/2733461271 conn(0x7f9fc00d68c0 msgr2=0x7f9fc0080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 shutdown_connections 2020-11-16T22:36:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:35.119+0000 7f9fbffff700 1 -- 172.21.15.181:0/2733461271 wait complete. 2020-11-16T22:36:35.508 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:35.508 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":600,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":117477,"bytes_used":64602112,"bytes_avail":773029511168,"bytes_total":773094113280,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":125,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"e","status":"up:active","gid":8040},{"filesystem_id":7,"rank":1,"name":"i","status":"up:active","gid":8043},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8101}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":38,"modified":"2020-11-16T22:36:28.855963+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2020-11-16T22:36:35.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/d1/d2/d3]', 'e': 'paused paths [/d1/d2/d3]', 'i': 'paused'} 2020-11-16T22:36:36.509 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:36:36.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.647+0000 7f0c5bab6700 1 Processor -- start 2020-11-16T22:36:36.639 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.647+0000 7f0c5bab6700 1 -- start start 2020-11-16T22:36:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c5415f5b0 0x7f0c540d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c54160e40 con 0x7f0c540615f0 2020-11-16T22:36:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c541609a0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54160ba0 con 0x7f0c540d1830 2020-11-16T22:36:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46014/0 (socket says 172.21.15.181:46014) 2020-11-16T22:36:36.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c59852700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0c5415f5b0 0x7f0c540d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 -- 172.21.15.181:0/2806321025 learned_addr learned my addr 172.21.15.181:0/2806321025 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 -- 172.21.15.181:0/2806321025 >> v2:172.21.15.175:3300/0 conn(0x7f0c5415f5b0 msgr2=0x7f0c540d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 --2- 172.21.15.181:0/2806321025 >> v2:172.21.15.175:3300/0 conn(0x7f0c5415f5b0 0x7f0c540d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 -- 172.21.15.181:0/2806321025 >> v1:172.21.15.175:6789/0 conn(0x7f0c540615f0 legacy=0x7f0c540622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 -- 172.21.15.181:0/2806321025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54060970 con 0x7f0c540d1830 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5a053700 1 --2- 172.21.15.181:0/2806321025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f0c4c009d50 tx=0x7f0c4c009290).ready entity=mon.2 client_cookie=9e7d26145aabed8a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:36.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c58850700 1 -- 172.21.15.181:0/2806321025 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00bbc0 con 0x7f0c540d1830 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c58850700 1 -- 172.21.15.181:0/2806321025 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00bd20 con 0x7f0c540d1830 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c58850700 1 -- 172.21.15.181:0/2806321025 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011610 con 0x7f0c540d1830 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- 172.21.15.181:0/2806321025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 msgr2=0x7f0c540629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/2806321025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f0c4c009d50 tx=0x7f0c4c009290).stop 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- 172.21.15.181:0/2806321025 shutdown_connections 2020-11-16T22:36:36.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/2806321025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c540d1830 0x7f0c540629b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/2806321025 >> v2:172.21.15.175:3300/0 conn(0x7f0c5415f5b0 0x7f0c540d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- 172.21.15.181:0/2806321025 >> 172.21.15.181:0/2806321025 conn(0x7f0c540d69a0 msgr2=0x7f0c540d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:36.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- 172.21.15.181:0/2806321025 shutdown_connections 2020-11-16T22:36:36.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.651+0000 7f0c5bab6700 1 -- 172.21.15.181:0/2806321025 wait complete. 2020-11-16T22:36:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 Processor -- start 2020-11-16T22:36:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 -- start start 2020-11-16T22:36:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c540615f0 0x7f0c54162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c540d1830 0x7f0c541707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54162780 con 0x7f0c540615f0 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540d6350 con 0x7f0c540d1830 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5bab6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c541629c0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.647 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46018/0 (socket says 172.21.15.181:46018) 2020-11-16T22:36:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 -- 172.21.15.181:0/4174152161 learned_addr learned my addr 172.21.15.181:0/4174152161 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c59852700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c540615f0 0x7f0c54162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.648 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c59051700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c540d1830 0x7f0c541707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 -- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c540d1830 msgr2=0x7f0c541707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.649 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c540d1830 0x7f0c541707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 -- 172.21.15.181:0/4174152161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c540615f0 msgr2=0x7f0c54162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c540615f0 0x7f0c54162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 -- 172.21.15.181:0/4174152161 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54162c60 con 0x7f0c5415f5b0 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.655+0000 7f0c5a053700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f0c4c009fd0 tx=0x7f0c4c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00bbc0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.650 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00be90 con 0x7f0c5415f5b0 2020-11-16T22:36:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c54162f00 con 0x7f0c5415f5b0 2020-11-16T22:36:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c010cf0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c541636e0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0c4c017070 con 0x7f0c5415f5b0 2020-11-16T22:36:36.651 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c54069eb0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 0x7f0c2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c4c06fe40 con 0x7f0c5415f5b0 2020-11-16T22:36:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c59852700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 0x7f0c2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.652 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.659+0000 7f0c59852700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 0x7f0c2c057520 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f0c44006fd0 tx=0x7f0c44008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:36.657 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.667+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0c4c046120 con 0x7f0c5415f5b0 2020-11-16T22:36:36.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c541642f0 con 0x7f0c5415f5b0 2020-11-16T22:36:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c41ffb700 1 -- 172.21.15.181:0/4174152161 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4673 (secure 0 0 0) 0x7f0c4c015070 con 0x7f0c5415f5b0 2020-11-16T22:36:36.860 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:36.861 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":125,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4185480402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4185480402},{"type":"v1","addr":"172.21.15.181:6835","nonce":4185480402}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":7391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6839/341660022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":341660022},{"type":"v1","addr":"172.21.15.181:6839","nonce":341660022}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7412,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.175:6841/3307041078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6839","nonce":3307041078},{"type":"v1","addr":"172.21.15.175:6841","nonce":3307041078}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":7415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.181:6841/442659291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":442659291},{"type":"v1","addr":"172.21.15.181:6841","nonce":442659291}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":111},{"gid":8016,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3708019153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3708019153},{"type":"v1","addr":"172.21.15.175:6833","nonce":3708019153}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110},{"gid":8037,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1813723711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1813723711},{"type":"v1","addr":"172.21.15.175:6836","nonce":1813723711}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":110}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:35:22.567943+0000","modified":"2020-11-16T22:36:21.832449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8101,"mds_1":8043,"mds_2":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"e","rank":2,"incarnation":118,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6837/722500686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":722500686},{"type":"v1","addr":"172.21.15.175:6837","nonce":722500686}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_8043":{"gid":8043,"name":"i","rank":1,"incarnation":116,"state":"up:active","state_seq":17,"addr":"172.21.15.175:6840/3163754567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3163754567},{"type":"v1","addr":"172.21.15.175:6840","nonce":3163754567}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_8101":{"gid":8101,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":17,"addr":"172.21.15.181:6837/2750102703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2750102703},{"type":"v1","addr":"172.21.15.181:6837","nonce":2750102703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:36:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 msgr2=0x7f0c2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 0x7f0c2c057520 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f0c44006fd0 tx=0x7f0c44008040).stop 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 msgr2=0x7f0c54170cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f0c4c009fd0 tx=0x7f0c4c016000).stop 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 shutdown_connections 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f0c2c055060 0x7f0c2c057520 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c5415f5b0 0x7f0c54170cb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c540615f0 0x7f0c54162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 --2- 172.21.15.181:0/4174152161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c540d1830 0x7f0c541707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 >> 172.21.15.181:0/4174152161 conn(0x7f0c540d69a0 msgr2=0x7f0c54061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 shutdown_connections 2020-11-16T22:36:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:36.871+0000 7f0c5bab6700 1 -- 172.21.15.181:0/4174152161 wait complete. 2020-11-16T22:36:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 125 2020-11-16T22:36:36.875 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'i', 'e'] 2020-11-16T22:36:36.876 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:36.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 Processor -- start 2020-11-16T22:36:36.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 -- start start 2020-11-16T22:36:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 0x7fe594066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe594066a20 0x7fe5940c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940cb460 con 0x7fe5940cac00 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe594160e90 con 0x7fe594066a20 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe598f3f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe594162500 con 0x7fe5940cafe0 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe5937fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe5940cac00 0x7fe594160780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52486/0 (socket says 172.21.15.181:52486) 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe5937fe700 1 -- 172.21.15.181:0/1925745367 learned_addr learned my addr 172.21.15.181:0/1925745367 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 --2- 172.21.15.181:0/1925745367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 0x7fe594066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909134392 0 0) 0x7fe5940cb460 con 0x7fe5940cac00 2020-11-16T22:36:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe580003240 con 0x7fe5940cac00 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe593fff700 1 --2- 172.21.15.181:0/1925745367 >> v2:172.21.15.175:3300/0 conn(0x7fe594066a20 0x7fe5940c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 -- 172.21.15.181:0/1925745367 >> v2:172.21.15.175:3300/0 conn(0x7fe594066a20 msgr2=0x7fe5940c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 730383439 0 0) 0x7fe580003240 con 0x7fe5940cac00 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 --2- 172.21.15.181:0/1925745367 >> v2:172.21.15.175:3300/0 conn(0x7fe594066a20 0x7fe5940c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 -- 172.21.15.181:0/1925745367 >> v1:172.21.15.181:6789/0 conn(0x7fe5940cac00 legacy=0x7fe594160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 -- 172.21.15.181:0/1925745367 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5941627a0 con 0x7fe5940cafe0 2020-11-16T22:36:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.007+0000 7fe592ffd700 1 --2- 172.21.15.181:0/1925745367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 0x7fe594066530 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fe58800b9a0 tx=0x7fe588006df0).ready entity=mon.2 client_cookie=3fbcf2e01ce7db13 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584001860 con 0x7fe5940cafe0 2020-11-16T22:36:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588011400 con 0x7fe5940cafe0 2020-11-16T22:36:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe5927fc700 1 -- 172.21.15.181:0/1925745367 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588011780 con 0x7fe5940cafe0 2020-11-16T22:36:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- 172.21.15.181:0/1925745367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 msgr2=0x7fe594066530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1925745367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 0x7fe594066530 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fe58800b9a0 tx=0x7fe588006df0).stop 2020-11-16T22:36:37.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- 172.21.15.181:0/1925745367 shutdown_connections 2020-11-16T22:36:37.206 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1925745367 >> v2:172.21.15.175:3300/0 conn(0x7fe594066a20 0x7fe5940c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1925745367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5940cafe0 0x7fe594066530 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- 172.21.15.181:0/1925745367 >> 172.21.15.181:0/1925745367 conn(0x7fe5940d4590 msgr2=0x7fe5940d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:37.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- 172.21.15.181:0/1925745367 shutdown_connections 2020-11-16T22:36:37.207 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- 172.21.15.181:0/1925745367 wait complete. 2020-11-16T22:36:37.209 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 Processor -- start 2020-11-16T22:36:37.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.011+0000 7fe598f3f700 1 -- start start 2020-11-16T22:36:37.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 0x7fe59415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:37.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5940cac00 0x7fe5941635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:37.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:37.210 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59415d460 con 0x7fe5940cac00 2020-11-16T22:36:37.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59415cc80 con 0x7fe5940cafe0 2020-11-16T22:36:37.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59415cf20 con 0x7fe594066a20 2020-11-16T22:36:37.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:37.211 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe5937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 0x7fe59415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38190/0 (socket says 172.21.15.181:38190) 2020-11-16T22:36:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe5937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 0x7fe59415b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46030/0 (socket says 172.21.15.181:46030) 2020-11-16T22:36:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 -- 172.21.15.181:0/1735599668 learned_addr learned my addr 172.21.15.181:0/1735599668 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe592ffd700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5940cac00 0x7fe5941635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 -- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 msgr2=0x7fe59415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 0x7fe59415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 -- 172.21.15.181:0/1735599668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5940cac00 msgr2=0x7fe5941635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5940cac00 0x7fe5941635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 -- 172.21.15.181:0/1735599668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59415d1c0 con 0x7fe5940cafe0 2020-11-16T22:36:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe593fff700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fe58c00ebd0 tx=0x7fe58c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00b2e0 con 0x7fe5940cafe0 2020-11-16T22:36:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c00c820 con 0x7fe5940cafe0 2020-11-16T22:36:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00cbc0 con 0x7fe5940cafe0 2020-11-16T22:36:37.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59415dbc0 con 0x7fe5940cafe0 2020-11-16T22:36:37.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59415b790 con 0x7fe5940cafe0 2020-11-16T22:36:37.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.015+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59415e060 con 0x7fe5940cafe0 2020-11-16T22:36:37.618 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.019+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe58c01c460 con 0x7fe5940cafe0 2020-11-16T22:36:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.019+0000 7fe5737fe700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 0x7fe56c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.019+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe58c06dfb0 con 0x7fe5940cafe0 2020-11-16T22:36:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.019+0000 7fe5937fe700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 0x7fe56c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.019+0000 7fe5937fe700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 0x7fe56c057520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fe584001520 tx=0x7fe584001d20).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.027+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe59415e060 con 0x7fe5940cafe0 2020-11-16T22:36:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe59415e060 con 0x7fe5940cafe0 2020-11-16T22:36:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe5737fe700 1 -- 172.21.15.181:0/1735599668 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1691 (secure 0 0 0) 0x7fe59415e060 con 0x7fe5940cafe0 2020-11-16T22:36:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 msgr2=0x7fe56c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 0x7fe56c057520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fe584001520 tx=0x7fe584001d20).stop 2020-11-16T22:36:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 msgr2=0x7fe594163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fe58c00ebd0 tx=0x7fe58c006df0).stop 2020-11-16T22:36:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 shutdown_connections 2020-11-16T22:36:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe56c055060 0x7fe56c057520 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5940cafe0 0x7fe594163ab0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe594066a20 0x7fe59415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 --2- 172.21.15.181:0/1735599668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5940cac00 0x7fe5941635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 >> 172.21.15.181:0/1735599668 conn(0x7fe5940d4590 msgr2=0x7fe594140130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 shutdown_connections 2020-11-16T22:36:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:37.231+0000 7fe598f3f700 1 -- 172.21.15.181:0/1735599668 wait complete. 2020-11-16T22:36:37.828 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:37.828 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1605565602,"num_in_osds":8,"osd_in_since":1605565602,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":117477,"bytes_used":64602112,"bytes_avail":773029511168,"bytes_total":773094113280,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":125,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"e","status":"up:active","gid":8040},{"filesystem_id":7,"rank":1,"name":"i","status":"up:active","gid":8043},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8101}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":39,"modified":"2020-11-16T22:36:34.858023+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:36:38.030 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:36:38.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:36:38.032 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:36:38.066 INFO:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:36:38.083 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:36:38.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:36:38.100 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:36:38.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:36:38.101 INFO:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:36:38.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2020-11-16T22:36:38.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:38.188 INFO:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:36:38.207 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:36:38.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 Processor -- start 2020-11-16T22:36:38.337 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 -- start start 2020-11-16T22:36:38.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4008061b50 con 0x7f40081492c0 2020-11-16T22:36:38.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080618b0 con 0x7f4008145a30 2020-11-16T22:36:38.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400db8f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008149700 con 0x7f40080816e0 2020-11-16T22:36:38.339 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f4007fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f4007fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46040/0 (socket says 172.21.15.181:46040) 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f4007fff700 1 -- 172.21.15.181:0/2599218861 learned_addr learned my addr 172.21.15.181:0/2599218861 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.347+0000 7f400cb8d700 1 --1- 172.21.15.181:0/2599218861 >> v1:172.21.15.181:6789/0 conn(0x7f4008145a30 0x7f4008080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52500/0 (socket says 172.21.15.181:52500) 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f4007fff700 1 -- 172.21.15.181:0/2599218861 >> v1:172.21.15.181:6789/0 conn(0x7f4008145a30 legacy=0x7f4008080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f4007fff700 1 -- 172.21.15.181:0/2599218861 >> v1:172.21.15.175:6789/0 conn(0x7f40081492c0 legacy=0x7f40080cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:38.340 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f4007fff700 1 -- 172.21.15.181:0/2599218861 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080cb930 con 0x7f40080816e0 2020-11-16T22:36:38.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f4007fff700 1 --2- 172.21.15.181:0/2599218861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f3ff800ba20 tx=0x7f3ff8006eb0).ready entity=mon.2 client_cookie=bf861b607cc91d0a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.341 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f40077fe700 1 -- 172.21.15.181:0/2599218861 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80098d0 con 0x7f40080816e0 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f40077fe700 1 -- 172.21.15.181:0/2599218861 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff8009a30 con 0x7f40080816e0 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f40077fe700 1 -- 172.21.15.181:0/2599218861 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8009db0 con 0x7f40080816e0 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- 172.21.15.181:0/2599218861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 msgr2=0x7f4008145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 --2- 172.21.15.181:0/2599218861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f3ff800ba20 tx=0x7f3ff8006eb0).stop 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- 172.21.15.181:0/2599218861 shutdown_connections 2020-11-16T22:36:38.342 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 --2- 172.21.15.181:0/2599218861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40080816e0 0x7f4008145e10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- 172.21.15.181:0/2599218861 >> 172.21.15.181:0/2599218861 conn(0x7f40080d68c0 msgr2=0x7f40080d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:38.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- 172.21.15.181:0/2599218861 shutdown_connections 2020-11-16T22:36:38.343 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- 172.21.15.181:0/2599218861 wait complete. 2020-11-16T22:36:38.344 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 Processor -- start 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.351+0000 7f400db8f700 1 -- start start 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 0x7f400813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4008145a30 0x7f400813ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080cfd90 con 0x7f40080816e0 2020-11-16T22:36:38.345 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080d0030 con 0x7f4008145a30 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080d02d0 con 0x7f40081492c0 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f4007fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4008145a30 0x7f400813ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46046/0 (socket says 172.21.15.181:46046) 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 -- 172.21.15.181:0/3408555746 learned_addr learned my addr 172.21.15.181:0/3408555746 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:38.346 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400cb8d700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 0x7f400813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 -- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4008145a30 msgr2=0x7f400813ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4008145a30 0x7f400813ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 -- 172.21.15.181:0/3408555746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 msgr2=0x7f400813ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 0x7f400813ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 -- 172.21.15.181:0/3408555746 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080d0570 con 0x7f40081492c0 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400cb8d700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 0x7f400813ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400d38e700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f400000ca60 tx=0x7f4000007e70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000a730 con 0x7f40081492c0 2020-11-16T22:36:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000a890 con 0x7f40081492c0 2020-11-16T22:36:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000ac30 con 0x7f40081492c0 2020-11-16T22:36:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40080d0810 con 0x7f40081492c0 2020-11-16T22:36:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40080cf490 con 0x7f40081492c0 2020-11-16T22:36:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.355+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008044f60 con 0x7f40081492c0 2020-11-16T22:36:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.359+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4000019030 con 0x7f40081492c0 2020-11-16T22:36:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.359+0000 7f4004ff9700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 0x7f3fe4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.359+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f400006da60 con 0x7f40081492c0 2020-11-16T22:36:38.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.359+0000 7f400cb8d700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 0x7f3fe4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.350 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.359+0000 7f400cb8d700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 0x7f3fe4057520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f40080d43a0 tx=0x7f3ff0007120).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.353 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.363+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4000043ce0 con 0x7f40081492c0 2020-11-16T22:36:38.507 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.519+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4008068e30 con 0x7f40081492c0 2020-11-16T22:36:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f4004ff9700 1 -- 172.21.15.181:0/3408555746 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4000017020 con 0x7f40081492c0 2020-11-16T22:36:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 msgr2=0x7f3fe4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 0x7f3fe4057520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f40080d43a0 tx=0x7f3ff0007120).stop 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 msgr2=0x7f4008172c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f400000ca60 tx=0x7f4000007e70).stop 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 shutdown_connections 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f3fe4055060 0x7f3fe4057520 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40081492c0 0x7f4008172c70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40080816e0 0x7f400813ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 --2- 172.21.15.181:0/3408555746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4008145a30 0x7f400813ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 >> 172.21.15.181:0/3408555746 conn(0x7f40080d68c0 msgr2=0x7f40080d2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 shutdown_connections 2020-11-16T22:36:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.531+0000 7f400db8f700 1 -- 172.21.15.181:0/3408555746 wait complete. 2020-11-16T22:36:38.537 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2020-11-16T22:36:38.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.667+0000 7f94e2383700 1 Processor -- start 2020-11-16T22:36:38.658 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.667+0000 7f94e2383700 1 -- start start 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94dc0cf390 0x7f94dc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94dc163970 0x7f94dc163df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc065190 con 0x7f94dc0600f0 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0cfc60 con 0x7f94dc0cf390 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc065430 con 0x7f94dc163970 2020-11-16T22:36:38.660 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94dc163970 0x7f94dc163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94dc163970 0x7f94dc163df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38212/0 (socket says 172.21.15.181:38212) 2020-11-16T22:36:38.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 -- 172.21.15.181:0/3258394713 learned_addr learned my addr 172.21.15.181:0/3258394713 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:38.661 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e1381700 1 --2- 172.21.15.181:0/3258394713 >> v2:172.21.15.175:3300/0 conn(0x7f94dc0cf390 0x7f94dc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 -- 172.21.15.181:0/3258394713 >> v2:172.21.15.175:3300/0 conn(0x7f94dc0cf390 msgr2=0x7f94dc0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:38.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 --2- 172.21.15.181:0/3258394713 >> v2:172.21.15.175:3300/0 conn(0x7f94dc0cf390 0x7f94dc0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.662 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 -- 172.21.15.181:0/3258394713 >> v1:172.21.15.175:6789/0 conn(0x7f94dc0600f0 legacy=0x7f94dc060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 -- 172.21.15.181:0/3258394713 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0656d0 con 0x7f94dc163970 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e0b80700 1 --2- 172.21.15.181:0/3258394713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc163970 0x7f94dc163df0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f94dc0ca440 tx=0x7f94d8006eb0).ready entity=mon.1 client_cookie=d89dfe3876c5d484 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94d3fff700 1 -- 172.21.15.181:0/3258394713 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d80098d0 con 0x7f94dc163970 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94d3fff700 1 -- 172.21.15.181:0/3258394713 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8009a30 con 0x7f94dc163970 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94d3fff700 1 -- 172.21.15.181:0/3258394713 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8009db0 con 0x7f94dc163970 2020-11-16T22:36:38.663 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- 172.21.15.181:0/3258394713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc163970 msgr2=0x7f94dc163df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 --2- 172.21.15.181:0/3258394713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc163970 0x7f94dc163df0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f94dc0ca440 tx=0x7f94d8006eb0).stop 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- 172.21.15.181:0/3258394713 shutdown_connections 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 --2- 172.21.15.181:0/3258394713 >> v2:172.21.15.175:3300/0 conn(0x7f94dc0cf390 0x7f94dc0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 --2- 172.21.15.181:0/3258394713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc163970 0x7f94dc163df0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- 172.21.15.181:0/3258394713 >> 172.21.15.181:0/3258394713 conn(0x7f94dc0cfe90 msgr2=0x7f94dc0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- 172.21.15.181:0/3258394713 shutdown_connections 2020-11-16T22:36:38.664 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.671+0000 7f94e2383700 1 -- 172.21.15.181:0/3258394713 wait complete. 2020-11-16T22:36:38.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 Processor -- start 2020-11-16T22:36:38.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- start start 2020-11-16T22:36:38.666 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 0x7f94dc0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94dc0cf390 0x7f94dc140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 0x7f94dc161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0cbd80 con 0x7f94dc0cf390 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1701c0 con 0x7f94dc0600f0 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc161040 con 0x7f94dc163970 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1381700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 0x7f94dc0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.667 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 0x7f94dc161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1381700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 0x7f94dc0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38216/0 (socket says 172.21.15.181:38216) 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e0b80700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94dc0cf390 0x7f94dc140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1381700 1 -- 172.21.15.181:0/4005642554 learned_addr learned my addr 172.21.15.181:0/4005642554 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 -- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 msgr2=0x7f94dc0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 0x7f94dc0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.668 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 -- 172.21.15.181:0/4005642554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94dc0cf390 msgr2=0x7f94dc140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.669 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94dc0cf390 0x7f94dc140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 -- 172.21.15.181:0/4005642554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc1612e0 con 0x7f94dc163970 2020-11-16T22:36:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e1b82700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 0x7f94dc161cc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f94d400c810 tx=0x7f94d4007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400e560 con 0x7f94dc163970 2020-11-16T22:36:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400e6c0 con 0x7f94dc163970 2020-11-16T22:36:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc161580 con 0x7f94dc163970 2020-11-16T22:36:38.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400ea40 con 0x7f94dc163970 2020-11-16T22:36:38.871 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.675+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc140f60 con 0x7f94dc163970 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc170380 con 0x7f94dc163970 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f94d4010070 con 0x7f94dc163970 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94d17fa700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 0x7f94b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94e1381700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 0x7f94b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f94d406d8b0 con 0x7f94dc163970 2020-11-16T22:36:38.872 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.679+0000 7f94e1381700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 0x7f94b4057520 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f94cc00b6b0 tx=0x7f94cc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:38.873 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.687+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f94d4043b10 con 0x7f94dc163970 2020-11-16T22:36:38.882 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.891+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7f94dc044f60 con 0x7f94dc163970 2020-11-16T22:36:38.935 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.943+0000 7f94d17fa700 1 -- 172.21.15.181:0/4005642554 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v476) v1 ==== 165+0+0 (secure 0 0 0) 0x7f94d4047130 con 0x7f94dc163970 2020-11-16T22:36:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 msgr2=0x7f94b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 0x7f94b4057520 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f94cc00b6b0 tx=0x7f94cc008040).stop 2020-11-16T22:36:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 msgr2=0x7f94dc161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:38.937 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 0x7f94dc161cc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f94d400c810 tx=0x7f94d4007e20).stop 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 shutdown_connections 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f94b4055060 0x7f94b4057520 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94dc163970 0x7f94dc161cc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94dc0600f0 0x7f94dc0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 --2- 172.21.15.181:0/4005642554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94dc0cf390 0x7f94dc140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 >> 172.21.15.181:0/4005642554 conn(0x7f94dc0cfe90 msgr2=0x7f94dc060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:38.938 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 shutdown_connections 2020-11-16T22:36:38.939 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:38.947+0000 7f94e2383700 1 -- 172.21.15.181:0/4005642554 wait complete. 2020-11-16T22:36:38.950 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:36:38.951 INFO:tasks.cephfs_test_runner: 2020-11-16T22:36:38.951 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-16T22:36:38.951 INFO:tasks.cephfs_test_runner:Ran 6 tests in 539.914s 2020-11-16T22:36:38.952 INFO:tasks.cephfs_test_runner: 2020-11-16T22:36:38.952 INFO:tasks.cephfs_test_runner:OK 2020-11-16T22:36:38.952 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2020-11-16T22:36:39.162 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2020-11-16T22:36:39.171 DEBUG:tasks.check_counter:Getting stats from a 2020-11-16T22:36:39.172 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok perf dump 2020-11-16T22:36:39.296 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 0, 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2, 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 0, 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 140, 2020-11-16T22:36:39.297 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 18446744073709551614, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.001337550, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000010800, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000303566, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:39.298 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000386032, 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000386032 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:39.299 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 32, 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 29, 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 77056, 2020-11-16T22:36:39.300 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 13968, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.007554239, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001857877, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.006254730, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000853786, 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:39.301 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 29, 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.003075549, 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000106053 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:39.302 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2, 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 19, 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 267, 2020-11-16T22:36:39.303 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 69276, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.003610200, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001420081, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001844998, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:39.304 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 18, 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001248034, 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000069335 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:39.305 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:39.306 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:39.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.510 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2020-11-16T22:36:39.510 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2020-11-16T22:36:39.510 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2020-11-16T22:36:39.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:39.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:39.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:39.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:39.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:39.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:39.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:39.718 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 86852, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 38, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2020-11-16T22:36:39.719 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:39.720 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2020-11-16T22:36:39.721 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2020-11-16T22:36:39.722 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 29, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 71600, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 29, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:39.723 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 29, 2020-11-16T22:36:39.724 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 71600, 2020-11-16T22:36:39.724 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:39.724 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:39.724 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:39.724 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:39.926 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:39.926 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:39.926 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:39.927 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2020-11-16T22:36:39.928 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2020-11-16T22:36:39.928 DEBUG:tasks.check_counter:Getting stats from b 2020-11-16T22:36:39.928 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok perf dump 2020-11-16T22:36:40.056 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:40.057 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:40.057 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 0, 2020-11-16T22:36:40.057 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2, 2020-11-16T22:36:40.057 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 0, 2020-11-16T22:36:40.057 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 140, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 18446744073709551614, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.001994297, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000028828, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000501155, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:40.058 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000439997, 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000439997 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 5, 2020-11-16T22:36:40.060 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 22, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1728, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 69760, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.003953160, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001602741, 2020-11-16T22:36:40.061 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.002030734, 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 21, 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002241917, 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000106757 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.062 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.063 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 28, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 27, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 75120, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 13184, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:40.064 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.006121868, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001885552, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.003762617, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000874633, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 26, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002524661, 2020-11-16T22:36:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000097102 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.067 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2020-11-16T22:36:40.067 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2020-11-16T22:36:40.067 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2020-11-16T22:36:40.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.067 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2020-11-16T22:36:40.270 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:40.270 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:40.270 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:40.270 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:40.270 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:40.271 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:40.272 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:40.273 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:40.274 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 85650, 2020-11-16T22:36:40.275 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 37, 2020-11-16T22:36:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:40.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:40.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.481 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 29, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 71223, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:40.482 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 29, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 29, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 71223, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.483 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.484 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.484 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.484 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.484 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:40.485 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.b 2020-11-16T22:36:40.486 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.b 2020-11-16T22:36:40.690 DEBUG:tasks.check_counter:Getting stats from c 2020-11-16T22:36:40.690 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok perf dump 2020-11-16T22:36:40.817 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:40.817 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:40.817 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4, 2020-11-16T22:36:40.817 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1568, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 448, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 18446744073709551614, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.002438782, 2020-11-16T22:36:40.818 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000080158, 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000909770, 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3, 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000735409, 2020-11-16T22:36:40.819 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000245136 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.820 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1, 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 19, 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 160, 2020-11-16T22:36:40.821 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 69276, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.003833234, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001304732, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.002250953, 2020-11-16T22:36:40.822 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 18, 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001145101, 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000063616 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.823 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.824 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 28, 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 27, 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 75488, 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 13660, 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:40.825 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.005895029, 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001835544, 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.003312634, 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000859169, 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:40.826 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 27, 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002347595, 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000086947 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:40.827 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2020-11-16T22:36:40.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:41.030 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:41.030 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:41.030 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:41.031 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:41.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:41.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 88054, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 39, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2020-11-16T22:36:41.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2020-11-16T22:36:41.238 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:41.238 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:41.238 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:41.238 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:41.239 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:41.240 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:41.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 29, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 71600, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 29, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 29, 2020-11-16T22:36:41.242 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 71600, 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.243 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:41.245 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.c 2020-11-16T22:36:41.245 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.c 2020-11-16T22:36:41.245 DEBUG:tasks.check_counter:Getting stats from d 2020-11-16T22:36:41.246 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok perf dump 2020-11-16T22:36:41.373 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:41.373 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 154, 2020-11-16T22:36:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 107, 2020-11-16T22:36:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 36492, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 194976, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 24, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 18446744073709551615, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.028664788, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.005448519, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.015060591, 2020-11-16T22:36:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.005445842, 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 106, 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.071892777, 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000678233 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 67, 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000037821, 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000564 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:41.577 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 201, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 209, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 138237, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 78173, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.029272421, 2020-11-16T22:36:41.578 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.009502934, 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.020888169, 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002906099, 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 209, 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.024058655, 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000115113 2020-11-16T22:36:41.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.580 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:41.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 208, 2020-11-16T22:36:41.782 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000085041, 2020-11-16T22:36:41.782 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000408 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 123, 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 163, 2020-11-16T22:36:41.783 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 32271, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 297021, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.022411390, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.007880694, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.010125957, 2020-11-16T22:36:41.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.003649604, 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 160, 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.012792801, 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000079955 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:41.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 155, 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000062863, 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000405 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2020-11-16T22:36:41.786 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 132, 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.045680879, 2020-11-16T22:36:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000346067 2020-11-16T22:36:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2020-11-16T22:36:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2020-11-16T22:36:41.990 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2020-11-16T22:36:41.990 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2020-11-16T22:36:41.990 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000014416, 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000007208 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 19, 2020-11-16T22:36:41.991 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 18, 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 18, 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.030822780, 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001712376 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 1, 2020-11-16T22:36:41.992 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch": 0, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 16, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 16, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 5, 2020-11-16T22:36:41.993 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 11, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 15, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 2, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 5, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 5, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 4, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 24, 2020-11-16T22:36:41.994 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 22, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 25, 2020-11-16T22:36:41.995 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 1, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 4, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2020-11-16T22:36:41.996 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 10, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 2, 2020-11-16T22:36:41.997 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 10, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 14 2020-11-16T22:36:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2020-11-16T22:36:41.999 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2020-11-16T22:36:42.000 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 1, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 1, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2020-11-16T22:36:42.002 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 45, 2020-11-16T22:36:42.002 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 33, 2020-11-16T22:36:42.002 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 33, 2020-11-16T22:36:42.002 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 12, 2020-11-16T22:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2020-11-16T22:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 33, 2020-11-16T22:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 2, 2020-11-16T22:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 1, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 1, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 1, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4252719, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4274718, 2020-11-16T22:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 36, 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.227927006, 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006331305 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2020-11-16T22:36:42.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 19, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 21, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 17, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 20, 2020-11-16T22:36:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 3, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 16, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 18, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 2, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 9, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 9, 2020-11-16T22:36:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 38276, 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 338632 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 25, 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 19, 2020-11-16T22:36:42.211 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 6, 2020-11-16T22:36:42.212 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2020-11-16T22:36:42.212 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2020-11-16T22:36:42.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.212 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.414 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2020-11-16T22:36:42.414 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4, 2020-11-16T22:36:42.414 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.003919345, 2020-11-16T22:36:42.414 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000979836 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.415 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000754596, 2020-11-16T22:36:42.416 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000754596 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.417 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.418 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2020-11-16T22:36:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 8, 2020-11-16T22:36:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.015209120, 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001901140 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2020-11-16T22:36:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001086366, 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001086366 2020-11-16T22:36:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2020-11-16T22:36:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2020-11-16T22:36:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.832 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000592958, 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000592958 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.834 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3, 2020-11-16T22:36:42.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.009260395, 2020-11-16T22:36:43.038 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003086798 2020-11-16T22:36:43.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.038 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2020-11-16T22:36:43.038 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.038 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 0, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2020-11-16T22:36:43.040 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 0, 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 13, 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 13, 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 65 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2020-11-16T22:36:43.041 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:43.043 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:43.044 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:43.045 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 113023, 2020-11-16T22:36:43.046 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 75, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 264, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 3, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:43.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 5976, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 78, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2020-11-16T22:36:43.048 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 76208, 2020-11-16T22:36:43.049 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 449, 2020-11-16T22:36:43.250 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:43.250 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2020-11-16T22:36:43.250 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 97, 2020-11-16T22:36:43.251 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 97405, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 36, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 97, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 97, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 139 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 97, 2020-11-16T22:36:43.252 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 2, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 95, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 1, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 15, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2020-11-16T22:36:43.253 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 36, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 15, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2020-11-16T22:36:43.254 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 2, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 30, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2020-11-16T22:36:43.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2020-11-16T22:36:43.256 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2020-11-16T22:36:43.256 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2020-11-16T22:36:43.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2020-11-16T22:36:43.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2020-11-16T22:36:43.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2020-11-16T22:36:43.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2020-11-16T22:36:43.459 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2020-11-16T22:36:43.460 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 111, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 1, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 3, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 3, 2020-11-16T22:36:43.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 32, 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 2 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2020-11-16T22:36:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2020-11-16T22:36:43.463 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2020-11-16T22:36:43.464 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2020-11-16T22:36:43.464 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2020-11-16T22:36:43.666 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2020-11-16T22:36:43.666 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2020-11-16T22:36:43.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.666 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:43.666 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 473, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 160300, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 473, 2020-11-16T22:36:43.667 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 473, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 160300, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.668 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2020-11-16T22:36:43.669 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 97, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 97405, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 91, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 97405, 2020-11-16T22:36:43.670 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2020-11-16T22:36:43.671 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 97, 2020-11-16T22:36:43.672 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 97, 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 97, 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 97, 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.673 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2020-11-16T22:36:43.674 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:43.675 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x558896d359a0": { 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2020-11-16T22:36:43.676 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 45, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 80414, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 45, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2020-11-16T22:36:43.677 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 36, 2020-11-16T22:36:43.878 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 80414, 2020-11-16T22:36:43.878 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2020-11-16T22:36:43.878 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2020-11-16T22:36:43.878 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2020-11-16T22:36:43.879 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2020-11-16T22:36:43.879 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.879 INFO:teuthology.orchestra.run.smithi181.stdout: } 2020-11-16T22:36:43.879 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:43.880 INFO:tasks.check_counter:Daemon mds.d mds.exported=1 2020-11-16T22:36:43.881 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2020-11-16T22:36:43.881 DEBUG:tasks.check_counter:Getting stats from e 2020-11-16T22:36:43.881 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.e.asok perf dump 2020-11-16T22:36:44.008 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2020-11-16T22:36:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 57, 2020-11-16T22:36:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 45, 2020-11-16T22:36:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 24806, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 34168, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.009903926, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.002143308, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.005321681, 2020-11-16T22:36:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000867929, 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016040464, 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000348705 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 48, 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000014256, 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000297 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 197, 2020-11-16T22:36:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 168, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 135937, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 138869, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 14, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.029875904, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.008628038, 2020-11-16T22:36:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.024707789, 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.002869139, 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 168, 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.025807320, 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000153615 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 171, 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000080956, 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000473 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 18, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 43, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 4236, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 114647, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.007447490, 2020-11-16T22:36:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.002165743, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.004183530, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000542210, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 39, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.005526749, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000141711 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.017 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2020-11-16T22:36:44.017 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000005766, 2020-11-16T22:36:44.017 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000961 2020-11-16T22:36:44.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.218 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2020-11-16T22:36:44.218 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2020-11-16T22:36:44.218 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2020-11-16T22:36:44.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.218 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 67, 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.005967229, 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000089063 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000010355, 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000005177 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 2, 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 2, 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001411756, 2020-11-16T22:36:44.221 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000705878 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch": 0, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 14, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2020-11-16T22:36:44.222 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 18, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 18, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 17, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2020-11-16T22:36:44.223 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 4, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 5, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 2, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 3, 2020-11-16T22:36:44.224 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 1, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 1, 2020-11-16T22:36:44.225 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2020-11-16T22:36:44.226 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 1, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2020-11-16T22:36:44.227 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 1 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2020-11-16T22:36:44.228 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2020-11-16T22:36:44.229 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2020-11-16T22:36:44.230 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 5, 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 3, 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 3, 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 2, 2020-11-16T22:36:44.231 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 3, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 1, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 1, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 1, 2020-11-16T22:36:44.232 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4202276, 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4208277, 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 5, 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.028891947, 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.005778389 2020-11-16T22:36:44.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 21, 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 21, 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2020-11-16T22:36:44.234 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 20, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 21, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 1, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 18, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 18, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 3, 2020-11-16T22:36:44.235 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 3, 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 30320, 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 330440 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 2, 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2020-11-16T22:36:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 2, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 4, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.237 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.238 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.240 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2020-11-16T22:36:44.242 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2020-11-16T22:36:44.244 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2020-11-16T22:36:44.246 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001411756, 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000705878 2020-11-16T22:36:44.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2020-11-16T22:36:44.248 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2020-11-16T22:36:44.250 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.251 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2020-11-16T22:36:44.252 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.253 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.254 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.255 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 0, 2020-11-16T22:36:44.256 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 2, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 2, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1316 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.257 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:44.258 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:44.259 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:44.260 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:44.261 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:44.262 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 114944, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 63, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:44.263 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 5976, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 78, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:44.264 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 82040, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 344, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2020-11-16T22:36:44.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 55, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 27638, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 27, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 55, 2020-11-16T22:36:44.266 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 55, 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 87 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 55, 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2, 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 53, 2020-11-16T22:36:44.267 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 12, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 5, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 9, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2020-11-16T22:36:44.268 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2020-11-16T22:36:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 24, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2020-11-16T22:36:44.270 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2020-11-16T22:36:44.271 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2020-11-16T22:36:44.272 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2020-11-16T22:36:44.273 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 0, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 4, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 4, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 27, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2020-11-16T22:36:44.274 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 2 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.275 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2020-11-16T22:36:44.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 267, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 136036, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.277 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 267, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 267, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 136036, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.278 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.279 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 55, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 27638, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 49, 2020-11-16T22:36:44.280 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 27638, 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2020-11-16T22:36:44.281 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:44.282 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 55, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 55, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 55, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 55, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.283 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:44.284 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x5589566199a0": { 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 5, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 13973, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 5, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.287 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 5, 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 13973, 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.289 INFO:teuthology.orchestra.run.smithi175.stdout:} 2020-11-16T22:36:44.290 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2020-11-16T22:36:44.290 INFO:tasks.check_counter:Daemon mds.e mds.imported=1 2020-11-16T22:36:44.290 DEBUG:tasks.check_counter:Getting stats from f 2020-11-16T22:36:44.291 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.f.asok perf dump 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 33, 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 30, 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 76960, 2020-11-16T22:36:44.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14192, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 18446744073709551615, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.008160231, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.002115463, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.004251372, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.001008510, 2020-11-16T22:36:44.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 30, 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.003627228, 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000120907 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.417 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1, 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 21, 2020-11-16T22:36:44.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 160, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 69488, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.004752191, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.001254651, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.003126126, 2020-11-16T22:36:44.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 19, 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001328187, 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000069904 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.420 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 0, 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2020-11-16T22:36:44.421 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 0, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 0, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 18446744073709551615, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.000881416, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000000000, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000765920, 2020-11-16T22:36:44.422 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2020-11-16T22:36:44.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:44.425 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:44.426 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:44.427 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:44.428 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:44.429 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 88054, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 39, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2020-11-16T22:36:44.430 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:44.431 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:44.432 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 30, 2020-11-16T22:36:44.433 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 71353, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 30, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 30, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 71353, 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.434 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.435 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.435 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.435 INFO:teuthology.orchestra.run.smithi175.stdout:} 2020-11-16T22:36:44.436 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2020-11-16T22:36:44.436 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2020-11-16T22:36:44.437 DEBUG:tasks.check_counter:Getting stats from g 2020-11-16T22:36:44.437 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.g.asok perf dump 2020-11-16T22:36:44.558 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2020-11-16T22:36:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 0, 2020-11-16T22:36:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2020-11-16T22:36:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 0, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 0, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 18446744073709551614, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.001920493, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000000000, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000691929, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 5, 2020-11-16T22:36:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 22, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1728, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 69760, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.003539891, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.001528632, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.001702793, 2020-11-16T22:36:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 21, 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002428589, 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000115647 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 29, 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 28, 2020-11-16T22:36:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 75760, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 14080, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.006744218, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.001550277, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.004930305, 2020-11-16T22:36:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000916851, 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 27, 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002410650, 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000089283 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2020-11-16T22:36:44.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:44.569 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:44.570 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:44.571 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:44.572 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:44.573 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:44.574 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 88054, 2020-11-16T22:36:44.574 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 39, 2020-11-16T22:36:44.574 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:44.574 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:44.575 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:44.576 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.577 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 30, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 71730, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 30, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.578 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 30, 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 71730, 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.580 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.580 INFO:teuthology.orchestra.run.smithi175.stdout:} 2020-11-16T22:36:44.581 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2020-11-16T22:36:44.581 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2020-11-16T22:36:44.581 DEBUG:tasks.check_counter:Getting stats from h 2020-11-16T22:36:44.581 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.h.asok perf dump 2020-11-16T22:36:44.700 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2020-11-16T22:36:44.700 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 29, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 28, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 75392, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 13744, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 18446744073709551615, 2020-11-16T22:36:44.701 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.007776559, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.002093220, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.003659313, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000991750, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 27, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002588650, 2020-11-16T22:36:44.702 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000095875 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 5, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 23, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1728, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 69936, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 3, 2020-11-16T22:36:44.704 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.004366762, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.001304286, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.002887648, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 22, 2020-11-16T22:36:44.705 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002376109, 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000108004 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 107, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 140, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2020-11-16T22:36:44.707 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 18446744073709551615, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.000544365, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000019223, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000446261, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000366409, 2020-11-16T22:36:44.708 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000366409 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:44.710 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:44.711 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:44.712 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:44.713 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:44.714 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 86852, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 38, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:44.715 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:44.716 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 848, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2020-11-16T22:36:44.717 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:44.718 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 30, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 71353, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 30, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:44.719 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 30, 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 71353, 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.721 INFO:teuthology.orchestra.run.smithi175.stdout:} 2020-11-16T22:36:44.721 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2020-11-16T22:36:44.722 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2020-11-16T22:36:44.722 DEBUG:tasks.check_counter:Getting stats from i 2020-11-16T22:36:44.722 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.i.asok perf dump 2020-11-16T22:36:44.842 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2020-11-16T22:36:44.842 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2020-11-16T22:36:44.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 94, 2020-11-16T22:36:44.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 93, 2020-11-16T22:36:44.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 104353, 2020-11-16T22:36:44.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 31892, 2020-11-16T22:36:44.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.016697926, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.004633424, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.009890110, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.001352792, 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.844 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 93, 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.008956728, 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000096308 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 14, 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000011450, 2020-11-16T22:36:44.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000817 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 157, 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 153, 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 52685, 2020-11-16T22:36:44.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 252015, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 15, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.027622355, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.008311434, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.019517992, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.001369286, 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 152, 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016498387, 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000108542 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 222, 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000100157, 2020-11-16T22:36:44.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000451 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 51, 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 63, 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 17766, 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 32490, 2020-11-16T22:36:44.849 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.007874344, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.002476550, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.004754767, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000956364, 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2020-11-16T22:36:44.850 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 61, 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.023277289, 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000381594 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 48, 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000017613, 2020-11-16T22:36:44.851 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000366 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.852 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 66, 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.006976145, 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000105699 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.853 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000008685, 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000004342 2020-11-16T22:36:44.854 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 4, 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 3, 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.003219456, 2020-11-16T22:36:44.855 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001073152 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 1, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch": 0, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 13, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2020-11-16T22:36:44.856 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 16, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 16, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 16, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 2, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2020-11-16T22:36:44.857 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 4, 2020-11-16T22:36:44.858 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 6, 2020-11-16T22:36:44.858 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 5, 2020-11-16T22:36:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2020-11-16T22:36:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 1, 2020-11-16T22:36:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2020-11-16T22:36:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2020-11-16T22:36:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2020-11-16T22:36:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 2, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 1, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 2, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 1, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2020-11-16T22:36:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2020-11-16T22:36:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 3, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 3, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 3, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2020-11-16T22:36:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 2 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2020-11-16T22:36:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2020-11-16T22:36:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2020-11-16T22:36:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2020-11-16T22:36:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2020-11-16T22:36:45.270 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 1, 2020-11-16T22:36:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2020-11-16T22:36:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2020-11-16T22:36:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2020-11-16T22:36:45.271 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 7, 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 5, 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 5, 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 2, 2020-11-16T22:36:45.272 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 5, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 1, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 1, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2020-11-16T22:36:45.273 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 1, 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4208830, 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4214843, 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2020-11-16T22:36:45.274 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.028230819, 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004705136 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 19, 2020-11-16T22:36:45.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 21, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 2, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 18, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 21, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 3, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 16, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 18, 2020-11-16T22:36:45.276 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 2, 2020-11-16T22:36:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2020-11-16T22:36:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 5, 2020-11-16T22:36:45.277 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 5, 2020-11-16T22:36:45.478 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 31964, 2020-11-16T22:36:45.478 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 330440 2020-11-16T22:36:45.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.478 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 7, 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 4, 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 5, 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2020-11-16T22:36:45.479 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001440662, 2020-11-16T22:36:45.480 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001440662 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2020-11-16T22:36:45.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.484 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2020-11-16T22:36:45.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.484 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2020-11-16T22:36:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2020-11-16T22:36:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001778794, 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000889397 2020-11-16T22:36:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2020-11-16T22:36:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2020-11-16T22:36:45.902 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.902 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2020-11-16T22:36:45.904 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:45.905 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2020-11-16T22:36:45.906 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 0, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 2, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 2, 2020-11-16T22:36:45.907 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1316 2020-11-16T22:36:45.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:45.908 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2020-11-16T22:36:45.908 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2020-11-16T22:36:45.908 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2020-11-16T22:36:45.908 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2020-11-16T22:36:46.110 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2020-11-16T22:36:46.110 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2020-11-16T22:36:46.110 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2020-11-16T22:36:46.111 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_bytes": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Buffer_items": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_bytes": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Extent_items": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_bytes": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_Blob_items": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_bytes": 0, 2020-11-16T22:36:46.112 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_SharedBlob_items": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2020-11-16T22:36:46.113 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2020-11-16T22:36:46.114 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 119264, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 67, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2020-11-16T22:36:46.115 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2020-11-16T22:36:46.116 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2020-11-16T22:36:46.318 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2020-11-16T22:36:46.318 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2020-11-16T22:36:46.318 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2020-11-16T22:36:46.318 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 5976, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 78, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2020-11-16T22:36:46.319 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 74576, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 343, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2020-11-16T22:36:46.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 56, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 33932, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 27, 2020-11-16T22:36:46.321 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 56, 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 56, 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 90 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 56, 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2, 2020-11-16T22:36:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 54, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 12, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 6, 2020-11-16T22:36:46.323 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 9, 2020-11-16T22:36:46.324 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2020-11-16T22:36:46.324 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2020-11-16T22:36:46.324 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2020-11-16T22:36:46.526 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2020-11-16T22:36:46.526 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2020-11-16T22:36:46.526 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2020-11-16T22:36:46.526 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 24, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2020-11-16T22:36:46.527 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2020-11-16T22:36:46.528 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2020-11-16T22:36:46.529 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2020-11-16T22:36:46.530 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 0, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 4, 2020-11-16T22:36:46.531 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 4, 2020-11-16T22:36:46.532 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2020-11-16T22:36:46.532 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2020-11-16T22:36:46.532 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 28, 2020-11-16T22:36:46.532 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2020-11-16T22:36:46.734 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 3 2020-11-16T22:36:46.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2020-11-16T22:36:46.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2020-11-16T22:36:46.736 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 297, 2020-11-16T22:36:46.737 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 143115, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 297, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 297, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 143115, 2020-11-16T22:36:46.738 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2020-11-16T22:36:46.739 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:46.740 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:46.942 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 56, 2020-11-16T22:36:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 33932, 2020-11-16T22:36:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 50, 2020-11-16T22:36:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 33932, 2020-11-16T22:36:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2020-11-16T22:36:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:47.148 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 56, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 56, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 56, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 56, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2020-11-16T22:36:47.150 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:47.151 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:47.152 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:47.152 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:47.354 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:47.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2020-11-16T22:36:47.354 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x562e8a8779a0": { 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 7, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 20539, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2020-11-16T22:36:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 7, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 6, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 20539, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2020-11-16T22:36:47.356 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2020-11-16T22:36:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:47.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2020-11-16T22:36:47.357 INFO:teuthology.orchestra.run.smithi175.stdout:} 2020-11-16T22:36:47.358 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2020-11-16T22:36:47.359 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2020-11-16T22:36:47.359 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2020-11-16T22:36:47.368 INFO:tasks.kclient:Unmounting kernel clients... 2020-11-16T22:36:47.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.369 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:36:47.385 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:36:47.385 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:36:47.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.386 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2020-11-16T22:36:47.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.417 INFO:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-16T22:36:47.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.553 INFO:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2020-11-16T22:36:47.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.751 INFO:teuthology.orchestra.run.smithi106:> ip netns list 2020-11-16T22:36:47.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:36:47.764 INFO:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2020-11-16T22:36:47.842 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2020-11-16T22:36:47.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:36:47.845 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-11-16T22:36:47.854 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-11-16T22:36:47.855 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:47.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:47.999+0000 7f6747e56700 1 Processor -- start 2020-11-16T22:36:47.988 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:47.999+0000 7f6747e56700 1 -- start start 2020-11-16T22:36:47.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:47.999+0000 7f6747e56700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 0x7f6740061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:47.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:47.999+0000 7f6747e56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 0x7f674015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:47.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:47.999+0000 7f6747e56700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6740061b70 con 0x7f67400ce750 2020-11-16T22:36:47.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674015cda0 con 0x7f67400612a0 2020-11-16T22:36:47.991 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674015d040 con 0x7f674015fa10 2020-11-16T22:36:47.992 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 0x7f674015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6745bf2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 0x7f6740061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67453f1700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f67400ce750 0x7f67400cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.181:45642/0 (socket says 172.21.15.181:45642) 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67453f1700 1 -- 172.21.15.181:0/3078103902 learned_addr learned my addr 172.21.15.181:0/3078103902 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6744bf0700 1 -- 172.21.15.181:0/3078103902 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308180468 0 0) 0x7f6740061b70 con 0x7f67400ce750 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6744bf0700 1 -- 172.21.15.181:0/3078103902 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6728003240 con 0x7f67400ce750 2020-11-16T22:36:47.993 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 -- 172.21.15.181:0/3078103902 >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 msgr2=0x7f6740061680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 --2- 172.21.15.181:0/3078103902 >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 0x7f6740061680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 -- 172.21.15.181:0/3078103902 >> v1:172.21.15.175:6789/0 conn(0x7f67400ce750 legacy=0x7f67400cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 -- 172.21.15.181:0/3078103902 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674015d2e0 con 0x7f674015fa10 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6745bf2700 1 --2- 172.21.15.181:0/3078103902 >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 0x7f6740061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f67463f3700 1 --2- 172.21.15.181:0/3078103902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 0x7f674015fea0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f6738009c90 tx=0x7f67380091d0).ready entity=mon.2 client_cookie=643a2042b75109c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6744bf0700 1 -- 172.21.15.181:0/3078103902 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800b9c0 con 0x7f674015fa10 2020-11-16T22:36:47.994 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6744bf0700 1 -- 172.21.15.181:0/3078103902 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800bb20 con 0x7f674015fa10 2020-11-16T22:36:47.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6744bf0700 1 -- 172.21.15.181:0/3078103902 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800bea0 con 0x7f674015fa10 2020-11-16T22:36:47.995 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- 172.21.15.181:0/3078103902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 msgr2=0x7f674015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 --2- 172.21.15.181:0/3078103902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 0x7f674015fea0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f6738009c90 tx=0x7f67380091d0).stop 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- 172.21.15.181:0/3078103902 shutdown_connections 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 --2- 172.21.15.181:0/3078103902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f674015fa10 0x7f674015fea0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 --2- 172.21.15.181:0/3078103902 >> v2:172.21.15.175:3300/0 conn(0x7f67400612a0 0x7f6740061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- 172.21.15.181:0/3078103902 >> 172.21.15.181:0/3078103902 conn(0x7f67400cf890 msgr2=0x7f67400cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:47.996 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- 172.21.15.181:0/3078103902 shutdown_connections 2020-11-16T22:36:47.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.003+0000 7f6747e56700 1 -- 172.21.15.181:0/3078103902 wait complete. 2020-11-16T22:36:47.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 Processor -- start 2020-11-16T22:36:47.997 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 -- start start 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67400ce750 0x7f674016d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 0x7f674016db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cc720 con 0x7f67400ce750 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cc9c0 con 0x7f674015fa10 2020-11-16T22:36:47.999 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6747e56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400ccc60 con 0x7f67400612a0 2020-11-16T22:36:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6745bf2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6745bf2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46072/0 (socket says 172.21.15.181:46072) 2020-11-16T22:36:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f6745bf2700 1 -- 172.21.15.181:0/3335547026 learned_addr learned my addr 172.21.15.181:0/3335547026 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.007+0000 7f67463f3700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 0x7f674016db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.000 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f67453f1700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67400ce750 0x7f674016d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 -- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 msgr2=0x7f674016db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.001 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 0x7f674016db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 -- 172.21.15.181:0/3335547026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67400ce750 msgr2=0x7f674016d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67400ce750 0x7f674016d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 -- 172.21.15.181:0/3335547026 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674014fc70 con 0x7f67400612a0 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f67463f3700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 0x7f674016db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6745bf2700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f673c00ef50 tx=0x7f673c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.002 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00ca80 con 0x7f67400612a0 2020-11-16T22:36:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00cbe0 con 0x7f67400612a0 2020-11-16T22:36:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674014ff10 con 0x7f67400612a0 2020-11-16T22:36:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0123f0 con 0x7f67400612a0 2020-11-16T22:36:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674014e3c0 con 0x7f67400612a0 2020-11-16T22:36:48.003 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6740069eb0 con 0x7f67400612a0 2020-11-16T22:36:48.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f673c012760 con 0x7f67400612a0 2020-11-16T22:36:48.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 0x7f6718057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f67453f1700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 0x7f6718057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f673c06e310 con 0x7f67400612a0 2020-11-16T22:36:48.004 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.011+0000 7f67453f1700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 0x7f6718057570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f674005bd80 tx=0x7f6730009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.008 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.019+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f673c0445f0 con 0x7f67400612a0 2020-11-16T22:36:48.161 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.171+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f674015f7f0 con 0x7f67400612a0 2020-11-16T22:36:48.163 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f672dffb700 1 -- 172.21.15.181:0/3335547026 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f673c047c10 con 0x7f67400612a0 2020-11-16T22:36:48.164 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:48.164 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:23.571640+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:48.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 msgr2=0x7f6718057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 0x7f6718057570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f674005bd80 tx=0x7f6730009040).stop 2020-11-16T22:36:48.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 msgr2=0x7f674014ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.167 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f673c00ef50 tx=0x7f673c006df0).stop 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 shutdown_connections 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f67180550b0 0x7f6718057570 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674015fa10 0x7f674016db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67400612a0 0x7f674014ded0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 --2- 172.21.15.181:0/3335547026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67400ce750 0x7f674016d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.175+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 >> 172.21.15.181:0/3335547026 conn(0x7f67400cf890 msgr2=0x7f674015c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:48.168 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.179+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 shutdown_connections 2020-11-16T22:36:48.169 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.179+0000 7f6747e56700 1 -- 172.21.15.181:0/3335547026 wait complete. 2020-11-16T22:36:48.183 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:48.299 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 Processor -- start 2020-11-16T22:36:48.300 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 -- start start 2020-11-16T22:36:48.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa64815f5b0 0x7fa6480d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa648160e40 con 0x7fa6480d1830 2020-11-16T22:36:48.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa648160ba0 con 0x7fa6480615f0 2020-11-16T22:36:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa64d53a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6481609a0 con 0x7fa64815f5b0 2020-11-16T22:36:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa64815f5b0 0x7fa6480d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa64815f5b0 0x7fa6480d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38238/0 (socket says 172.21.15.181:38238) 2020-11-16T22:36:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 -- 172.21.15.181:0/3051048985 learned_addr learned my addr 172.21.15.181:0/3051048985 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:48.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa6467fc700 1 --1- 172.21.15.181:0/3051048985 >> v1:172.21.15.181:6789/0 conn(0x7fa6480615f0 0x7fa6480622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52546/0 (socket says 172.21.15.181:52546) 2020-11-16T22:36:48.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 -- 172.21.15.181:0/3051048985 >> v1:172.21.15.181:6789/0 conn(0x7fa6480615f0 legacy=0x7fa6480622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 -- 172.21.15.181:0/3051048985 >> v1:172.21.15.175:6789/0 conn(0x7fa6480d1830 legacy=0x7fa648080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 -- 172.21.15.181:0/3051048985 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648060970 con 0x7fa64815f5b0 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa646ffd700 1 --2- 172.21.15.181:0/3051048985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa64815f5b0 0x7fa6480d1340 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fa638006fd0 tx=0x7fa638006e50).ready entity=mon.1 client_cookie=6e58a0dabff00817 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa645ffb700 1 -- 172.21.15.181:0/3051048985 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638006030 con 0x7fa64815f5b0 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa645ffb700 1 -- 172.21.15.181:0/3051048985 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638006190 con 0x7fa64815f5b0 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.311+0000 7fa645ffb700 1 -- 172.21.15.181:0/3051048985 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638006510 con 0x7fa64815f5b0 2020-11-16T22:36:48.304 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- 172.21.15.181:0/3051048985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa64815f5b0 msgr2=0x7fa6480d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3051048985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa64815f5b0 0x7fa6480d1340 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fa638006fd0 tx=0x7fa638006e50).stop 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- 172.21.15.181:0/3051048985 shutdown_connections 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3051048985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa64815f5b0 0x7fa6480d1340 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- 172.21.15.181:0/3051048985 >> 172.21.15.181:0/3051048985 conn(0x7fa6480d69a0 msgr2=0x7fa6480d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- 172.21.15.181:0/3051048985 shutdown_connections 2020-11-16T22:36:48.305 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- 172.21.15.181:0/3051048985 wait complete. 2020-11-16T22:36:48.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 Processor -- start 2020-11-16T22:36:48.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- start start 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6480d1830 0x7fa64815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815f5b0 0x7fa648080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648062d90 con 0x7fa6480d1830 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b9c70 con 0x7fa6480615f0 2020-11-16T22:36:48.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa64d53a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b9f10 con 0x7fa64815f5b0 2020-11-16T22:36:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa646ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa646ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38244/0 (socket says 172.21.15.181:38244) 2020-11-16T22:36:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa646ffd700 1 -- 172.21.15.181:0/3935357118 learned_addr learned my addr 172.21.15.181:0/3935357118 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.315+0000 7fa6477fe700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815f5b0 0x7fa648080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa6467fc700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6480d1830 0x7fa64815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 -- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815f5b0 msgr2=0x7fa648080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815f5b0 0x7fa648080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 -- 172.21.15.181:0/3935357118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6480d1830 msgr2=0x7fa64815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.309 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6480d1830 0x7fa64815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 -- 172.21.15.181:0/3935357118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480ba1b0 con 0x7fa6480615f0 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa646ffd700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa64805d520 tx=0x7fa638015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638006830 con 0x7fa6480615f0 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638005a30 con 0x7fa6480615f0 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638005dd0 con 0x7fa6480615f0 2020-11-16T22:36:48.310 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6480ba450 con 0x7fa6480615f0 2020-11-16T22:36:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6480b9930 con 0x7fa6480615f0 2020-11-16T22:36:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa648069eb0 con 0x7fa6480615f0 2020-11-16T22:36:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa63801a040 con 0x7fa6480615f0 2020-11-16T22:36:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.319+0000 7fa63effd700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 0x7fa624057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.323+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa63806f1b0 con 0x7fa6480615f0 2020-11-16T22:36:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.323+0000 7fa6467fc700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 0x7fa624057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.323+0000 7fa6467fc700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 0x7fa624057520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fa6480d6910 tx=0x7fa630007b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.318 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.327+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa6380454c0 con 0x7fa6480615f0 2020-11-16T22:36:48.466 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.475+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6480b9af0 con 0x7fa624055060 2020-11-16T22:36:48.476 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa63effd700 1 -- 172.21.15.181:0/3935357118 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+104598 (secure 0 0 0) 0x7fa630004cb0 con 0x7fa624055060 2020-11-16T22:36:48.477 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 msgr2=0x7fa624057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 0x7fa624057520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fa6480d6910 tx=0x7fa630007b80).stop 2020-11-16T22:36:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 msgr2=0x7fa6480628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa64805d520 tx=0x7fa638015000).stop 2020-11-16T22:36:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 shutdown_connections 2020-11-16T22:36:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fa624055060 0x7fa624057520 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815f5b0 0x7fa648080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480615f0 0x7fa6480628a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 --2- 172.21.15.181:0/3935357118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6480d1830 0x7fa64815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 >> 172.21.15.181:0/3935357118 conn(0x7fa6480d69a0 msgr2=0x7fa6481636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 shutdown_connections 2020-11-16T22:36:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.487+0000 7fa64d53a700 1 -- 172.21.15.181:0/3935357118 wait complete. 2020-11-16T22:36:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:36:48.493 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2020-11-16T22:36:46.861368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117916,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":259,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":63096,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":51968,"kb_avail":754911624,"statfs":{"total":773094113280,"available":773029502976,"internally_reserved":0,"allocated":11329536,"data_stored":5273368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53215232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":439,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003488"},"pg_stats":[{"pgid":"16.7","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.595364+0000","last_change":"2020-11-16T22:35:22.593356+0000","last_active":"2020-11-16T22:35:22.595364+0000","last_peered":"2020-11-16T22:35:22.595364+0000","last_clean":"2020-11-16T22:35:22.595364+0000","last_became_active":"2020-11-16T22:35:22.592949+0000","last_became_peered":"2020-11-16T22:35:22.592949+0000","last_unstale":"2020-11-16T22:35:22.595364+0000","last_undegraded":"2020-11-16T22:35:22.595364+0000","last_fullsized":"2020-11-16T22:35:22.595364+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.593947+0000","last_change":"2020-11-16T22:35:22.590884+0000","last_active":"2020-11-16T22:35:22.593947+0000","last_peered":"2020-11-16T22:35:22.593947+0000","last_clean":"2020-11-16T22:35:22.593947+0000","last_became_active":"2020-11-16T22:35:22.590435+0000","last_became_peered":"2020-11-16T22:35:22.590435+0000","last_unstale":"2020-11-16T22:35:22.593947+0000","last_undegraded":"2020-11-16T22:35:22.593947+0000","last_fullsized":"2020-11-16T22:35:22.593947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.590491+0000","last_change":"2020-11-16T22:35:22.587484+0000","last_active":"2020-11-16T22:35:22.590491+0000","last_peered":"2020-11-16T22:35:22.590491+0000","last_clean":"2020-11-16T22:35:22.590491+0000","last_became_active":"2020-11-16T22:35:22.586786+0000","last_became_peered":"2020-11-16T22:35:22.586786+0000","last_unstale":"2020-11-16T22:35:22.590491+0000","last_undegraded":"2020-11-16T22:35:22.590491+0000","last_fullsized":"2020-11-16T22:35:22.590491+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.591364+0000","last_change":"2020-11-16T22:35:22.588060+0000","last_active":"2020-11-16T22:35:22.591364+0000","last_peered":"2020-11-16T22:35:22.591364+0000","last_clean":"2020-11-16T22:35:22.591364+0000","last_became_active":"2020-11-16T22:35:22.586336+0000","last_became_peered":"2020-11-16T22:35:22.586336+0000","last_unstale":"2020-11-16T22:35:22.591364+0000","last_undegraded":"2020-11-16T22:35:22.591364+0000","last_fullsized":"2020-11-16T22:35:22.591364+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.607868+0000","last_change":"2020-11-16T22:35:22.603425+0000","last_active":"2020-11-16T22:35:22.607868+0000","last_peered":"2020-11-16T22:35:22.607868+0000","last_clean":"2020-11-16T22:35:22.607868+0000","last_became_active":"2020-11-16T22:35:22.602895+0000","last_became_peered":"2020-11-16T22:35:22.602895+0000","last_unstale":"2020-11-16T22:35:22.607868+0000","last_undegraded":"2020-11-16T22:35:22.607868+0000","last_fullsized":"2020-11-16T22:35:22.607868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.599330+0000","last_change":"2020-11-16T22:35:22.593614+0000","last_active":"2020-11-16T22:35:22.599330+0000","last_peered":"2020-11-16T22:35:22.599330+0000","last_clean":"2020-11-16T22:35:22.599330+0000","last_became_active":"2020-11-16T22:35:22.593172+0000","last_became_peered":"2020-11-16T22:35:22.593172+0000","last_unstale":"2020-11-16T22:35:22.599330+0000","last_undegraded":"2020-11-16T22:35:22.599330+0000","last_fullsized":"2020-11-16T22:35:22.599330+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.589327+0000","last_change":"2020-11-16T22:35:22.588442+0000","last_active":"2020-11-16T22:35:22.589327+0000","last_peered":"2020-11-16T22:35:22.589327+0000","last_clean":"2020-11-16T22:35:22.589327+0000","last_became_active":"2020-11-16T22:35:22.588015+0000","last_became_peered":"2020-11-16T22:35:22.588015+0000","last_unstale":"2020-11-16T22:35:22.589327+0000","last_undegraded":"2020-11-16T22:35:22.589327+0000","last_fullsized":"2020-11-16T22:35:22.589327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.602537+0000","last_change":"2020-11-16T22:35:22.601873+0000","last_active":"2020-11-16T22:35:22.602537+0000","last_peered":"2020-11-16T22:35:22.602537+0000","last_clean":"2020-11-16T22:35:22.602537+0000","last_became_active":"2020-11-16T22:35:22.601487+0000","last_became_peered":"2020-11-16T22:35:22.601487+0000","last_unstale":"2020-11-16T22:35:22.602537+0000","last_undegraded":"2020-11-16T22:35:22.602537+0000","last_fullsized":"2020-11-16T22:35:22.602537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"111'29","reported_seq":"40","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.153498+0000","last_change":"2020-11-16T22:35:22.192135+0000","last_active":"2020-11-16T22:36:38.153498+0000","last_peered":"2020-11-16T22:36:38.153498+0000","last_clean":"2020-11-16T22:36:38.153498+0000","last_became_active":"2020-11-16T22:35:22.191742+0000","last_became_peered":"2020-11-16T22:35:22.191742+0000","last_unstale":"2020-11-16T22:36:38.153498+0000","last_undegraded":"2020-11-16T22:36:38.153498+0000","last_fullsized":"2020-11-16T22:36:38.153498+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.6","version":"111'14","reported_seq":"25","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:01.326219+0000","last_change":"2020-11-16T22:35:22.196234+0000","last_active":"2020-11-16T22:36:01.326219+0000","last_peered":"2020-11-16T22:36:01.326219+0000","last_clean":"2020-11-16T22:36:01.326219+0000","last_became_active":"2020-11-16T22:35:22.195870+0000","last_became_peered":"2020-11-16T22:35:22.195870+0000","last_unstale":"2020-11-16T22:36:01.326219+0000","last_undegraded":"2020-11-16T22:36:01.326219+0000","last_fullsized":"2020-11-16T22:36:01.326219+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"111'19","reported_seq":"31","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:12.379446+0000","last_change":"2020-11-16T22:35:21.258469+0000","last_active":"2020-11-16T22:36:12.379446+0000","last_peered":"2020-11-16T22:36:12.379446+0000","last_clean":"2020-11-16T22:36:12.379446+0000","last_became_active":"2020-11-16T22:35:21.257651+0000","last_became_peered":"2020-11-16T22:35:21.257651+0000","last_unstale":"2020-11-16T22:36:12.379446+0000","last_undegraded":"2020-11-16T22:36:12.379446+0000","last_fullsized":"2020-11-16T22:36:12.379446+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"111'55","reported_seq":"68","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.154153+0000","last_change":"2020-11-16T22:35:21.130500+0000","last_active":"2020-11-16T22:36:38.154153+0000","last_peered":"2020-11-16T22:36:38.154153+0000","last_clean":"2020-11-16T22:36:38.154153+0000","last_became_active":"2020-11-16T22:35:21.130104+0000","last_became_peered":"2020-11-16T22:35:21.130104+0000","last_unstale":"2020-11-16T22:36:38.154153+0000","last_undegraded":"2020-11-16T22:36:38.154153+0000","last_fullsized":"2020-11-16T22:36:38.154153+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":81610,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"111'26","reported_seq":"38","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:04.895494+0000","last_change":"2020-11-16T22:35:21.178478+0000","last_active":"2020-11-16T22:36:04.895494+0000","last_peered":"2020-11-16T22:36:04.895494+0000","last_clean":"2020-11-16T22:36:04.895494+0000","last_became_active":"2020-11-16T22:35:21.177888+0000","last_became_peered":"2020-11-16T22:35:21.177888+0000","last_unstale":"2020-11-16T22:36:04.895494+0000","last_undegraded":"2020-11-16T22:36:04.895494+0000","last_fullsized":"2020-11-16T22:36:04.895494+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20641,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"111'19","reported_seq":"30","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:12.136875+0000","last_change":"2020-11-16T22:35:22.196350+0000","last_active":"2020-11-16T22:36:12.136875+0000","last_peered":"2020-11-16T22:36:12.136875+0000","last_clean":"2020-11-16T22:36:12.136875+0000","last_became_active":"2020-11-16T22:35:22.195927+0000","last_became_peered":"2020-11-16T22:35:22.195927+0000","last_unstale":"2020-11-16T22:36:12.136875+0000","last_undegraded":"2020-11-16T22:36:12.136875+0000","last_fullsized":"2020-11-16T22:36:12.136875+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1","version":"111'24","reported_seq":"37","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.154106+0000","last_change":"2020-11-16T22:35:21.179123+0000","last_active":"2020-11-16T22:36:38.154106+0000","last_peered":"2020-11-16T22:36:38.154106+0000","last_clean":"2020-11-16T22:36:38.154106+0000","last_became_active":"2020-11-16T22:35:21.178511+0000","last_became_peered":"2020-11-16T22:35:21.178511+0000","last_unstale":"2020-11-16T22:36:38.154106+0000","last_undegraded":"2020-11-16T22:36:38.154106+0000","last_fullsized":"2020-11-16T22:36:38.154106+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14153,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.0","version":"111'23","reported_seq":"34","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:01.330524+0000","last_change":"2020-11-16T22:35:22.266939+0000","last_active":"2020-11-16T22:36:01.330524+0000","last_peered":"2020-11-16T22:36:01.330524+0000","last_clean":"2020-11-16T22:36:01.330524+0000","last_became_active":"2020-11-16T22:35:22.266066+0000","last_became_peered":"2020-11-16T22:35:22.266066+0000","last_unstale":"2020-11-16T22:36:01.330524+0000","last_undegraded":"2020-11-16T22:36:01.330524+0000","last_fullsized":"2020-11-16T22:36:01.330524+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"74","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.581875+0000","last_change":"2020-11-16T22:26:48.228531+0000","last_active":"2020-11-16T22:35:22.581875+0000","last_peered":"2020-11-16T22:35:22.581875+0000","last_clean":"2020-11-16T22:35:22.581875+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:35:22.581875+0000","last_undegraded":"2020-11-16T22:35:22.581875+0000","last_fullsized":"2020-11-16T22:35:22.581875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.587008+0000","last_change":"2020-11-16T22:26:48.226107+0000","last_active":"2020-11-16T22:35:22.587008+0000","last_peered":"2020-11-16T22:35:22.587008+0000","last_clean":"2020-11-16T22:35:22.587008+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:35:22.587008+0000","last_undegraded":"2020-11-16T22:35:22.587008+0000","last_fullsized":"2020-11-16T22:35:22.587008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"77","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.575716+0000","last_change":"2020-11-16T22:26:48.145112+0000","last_active":"2020-11-16T22:35:22.575716+0000","last_peered":"2020-11-16T22:35:22.575716+0000","last_clean":"2020-11-16T22:35:22.575716+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:35:22.575716+0000","last_undegraded":"2020-11-16T22:35:22.575716+0000","last_fullsized":"2020-11-16T22:35:22.575716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.586204+0000","last_change":"2020-11-16T22:26:48.223877+0000","last_active":"2020-11-16T22:35:22.586204+0000","last_peered":"2020-11-16T22:35:22.586204+0000","last_clean":"2020-11-16T22:35:22.586204+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:35:22.586204+0000","last_undegraded":"2020-11-16T22:35:22.586204+0000","last_fullsized":"2020-11-16T22:35:22.586204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"96","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.578850+0000","last_change":"2020-11-16T22:26:44.683337+0000","last_active":"2020-11-16T22:35:22.578850+0000","last_peered":"2020-11-16T22:35:22.578850+0000","last_clean":"2020-11-16T22:35:22.578850+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:35:22.578850+0000","last_undegraded":"2020-11-16T22:35:22.578850+0000","last_fullsized":"2020-11-16T22:35:22.578850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_clean_scrub_stamp":"2020-11-16T22:26:39.662675+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.586201+0000","last_change":"2020-11-16T22:26:48.143290+0000","last_active":"2020-11-16T22:35:22.586201+0000","last_peered":"2020-11-16T22:35:22.586201+0000","last_clean":"2020-11-16T22:35:22.586201+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:35:22.586201+0000","last_undegraded":"2020-11-16T22:35:22.586201+0000","last_fullsized":"2020-11-16T22:35:22.586201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.588135+0000","last_change":"2020-11-16T22:26:48.141598+0000","last_active":"2020-11-16T22:35:22.588135+0000","last_peered":"2020-11-16T22:35:22.588135+0000","last_clean":"2020-11-16T22:35:22.588135+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:35:22.588135+0000","last_undegraded":"2020-11-16T22:35:22.588135+0000","last_fullsized":"2020-11-16T22:35:22.588135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"88","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.580628+0000","last_change":"2020-11-16T22:26:48.228444+0000","last_active":"2020-11-16T22:35:22.580628+0000","last_peered":"2020-11-16T22:35:22.580628+0000","last_clean":"2020-11-16T22:35:22.580628+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:35:22.580628+0000","last_undegraded":"2020-11-16T22:35:22.580628+0000","last_fullsized":"2020-11-16T22:35:22.580628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.588195+0000","last_change":"2020-11-16T22:26:48.225986+0000","last_active":"2020-11-16T22:35:22.588195+0000","last_peered":"2020-11-16T22:35:22.588195+0000","last_clean":"2020-11-16T22:35:22.588195+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:35:22.588195+0000","last_undegraded":"2020-11-16T22:35:22.588195+0000","last_fullsized":"2020-11-16T22:35:22.588195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":8,"stat_sum":{"num_bytes":117916,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":257,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":235832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":16,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542268,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7744,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364096,"statfs":{"total":96636764160,"available":96628834304,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.762,"15min":1.766},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.65100000000000002},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.835},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.774,"15min":1.784},"min":{"1min":1.2070000000000001,"5min":1.014,"15min":0.68899999999999995},"max":{"1min":2.702,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":1.605}]},{"osd":1,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":2.105,"15min":2.0209999999999999},"min":{"1min":1.2849999999999999,"5min":0.93000000000000005,"15min":0.92500000000000004},"max":{"1min":2.7269999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":2.379},{"interface":"front","average":{"1min":1.871,"5min":1.8029999999999999,"15min":1.76},"min":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":0.73099999999999998},"max":{"1min":2.5830000000000002,"5min":2.7749999999999999,"15min":2.883},"last":1.516}]},{"osd":2,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.3279999999999998,"5min":2.1520000000000001,"15min":2.0259999999999998},"min":{"1min":1.669,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":3.0529999999999999,"5min":3.0790000000000002,"15min":3.4780000000000002},"last":2.1880000000000002},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.2810000000000001,"15min":2.29},"min":{"1min":1.794,"5min":1.3,"15min":1.105},"max":{"1min":3.129,"5min":3.98,"15min":3.98},"last":2.028}]},{"osd":3,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.7789999999999999,"5min":2.5840000000000001,"15min":2.5609999999999999},"min":{"1min":1.546,"5min":1.4910000000000001,"15min":1.2190000000000001},"max":{"1min":3.6850000000000001,"5min":3.6850000000000001,"15min":3.9969999999999999},"last":3.1829999999999998},{"interface":"front","average":{"1min":2.9430000000000001,"5min":2.8460000000000001,"15min":2.823},"min":{"1min":2.2250000000000001,"5min":1.6020000000000001,"15min":1.2689999999999999},"max":{"1min":3.8420000000000001,"5min":4.266,"15min":4.266},"last":3.1349999999999998}]},{"osd":4,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.5329999999999999,"5min":2.3069999999999999,"15min":2.2290000000000001},"min":{"1min":1.718,"5min":1.5169999999999999,"15min":1.018},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.9279999999999999},{"interface":"front","average":{"1min":2.996,"5min":2.7789999999999999,"15min":2.7690000000000001},"min":{"1min":2.3029999999999999,"5min":1.748,"15min":1.1479999999999999},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9940000000000002},"last":2.5800000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.9910000000000001,"5min":2.9100000000000001,"15min":2.944},"min":{"1min":2.2480000000000002,"5min":1.4810000000000001,"15min":1.3540000000000001},"max":{"1min":4.3339999999999996,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":2.7480000000000002},{"interface":"front","average":{"1min":2.7690000000000001,"5min":2.746,"15min":2.7170000000000001},"min":{"1min":1.899,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":3.5259999999999998,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":2.2869999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.9830000000000001,"5min":2.8700000000000001,"15min":2.8650000000000002},"min":{"1min":2.1190000000000002,"5min":1.7569999999999999,"15min":1.2989999999999999},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":2.4860000000000002},{"interface":"front","average":{"1min":3.3100000000000001,"5min":3.2040000000000002,"15min":3.1699999999999999},"min":{"1min":2.6640000000000001,"5min":2.1629999999999998,"15min":1.222},"max":{"1min":4.5510000000000002,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":3.407}]}]},{"osd":5,"up_from":14,"seq":60129542267,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7508,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94364332,"statfs":{"total":96636764160,"available":96629075968,"internally_reserved":0,"allocated":1388544,"data_stored":630469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.696,"15min":1.6499999999999999},"min":{"1min":1.335,"5min":0.98699999999999999,"15min":0.97499999999999998},"max":{"1min":3.206,"5min":3.206,"15min":3.206},"last":2.2210000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.768,"15min":1.7490000000000001},"min":{"1min":1.083,"5min":1.083,"15min":0.94099999999999995},"max":{"1min":2.617,"5min":3.1469999999999998,"15min":3.1579999999999999},"last":1.379}]},{"osd":1,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":1.7669999999999999,"5min":1.768,"15min":1.6850000000000001},"min":{"1min":1.4450000000000001,"5min":1.1779999999999999,"15min":1.0469999999999999},"max":{"1min":2.8330000000000002,"5min":3.423,"15min":3.423},"last":1.6679999999999999},{"interface":"front","average":{"1min":2.347,"5min":2.1899999999999999,"15min":2.1709999999999998},"min":{"1min":1.2909999999999999,"5min":1.01,"15min":1.01},"max":{"1min":3.5270000000000001,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":2.4430000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":2.2759999999999998,"15min":2.2170000000000001},"min":{"1min":1.764,"5min":1.282,"15min":1.131},"max":{"1min":3.3849999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.97},{"interface":"front","average":{"1min":2.1560000000000001,"5min":2.101,"15min":2.069},"min":{"1min":1.4019999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":2.4990000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":2.4940000000000002,"15min":2.4449999999999998},"min":{"1min":1.6479999999999999,"5min":1.6419999999999999,"15min":1.1850000000000001},"max":{"1min":3.0019999999999998,"5min":3.2650000000000001,"15min":3.3340000000000001},"last":2.0699999999999998},{"interface":"front","average":{"1min":2.766,"5min":2.6739999999999999,"15min":2.5630000000000002},"min":{"1min":1.869,"5min":1.377,"15min":1.377},"max":{"1min":3.7130000000000001,"5min":3.7130000000000001,"15min":3.7130000000000001},"last":2.605}]},{"osd":4,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4119999999999999,"5min":2.3759999999999999,"15min":2.3639999999999999},"min":{"1min":1.905,"5min":1.3380000000000001,"15min":1.214},"max":{"1min":3.335,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":2.0249999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":2.8210000000000002,"15min":2.8069999999999999},"min":{"1min":2.246,"5min":1.9390000000000001,"15min":1.6879999999999999},"max":{"1min":3.915,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":2.9329999999999998}]},{"osd":6,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9369999999999998,"5min":2.8250000000000002,"15min":2.8300000000000001},"min":{"1min":2.2410000000000001,"5min":1.8109999999999999,"15min":1.6379999999999999},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":2.29},{"interface":"front","average":{"1min":3.3159999999999998,"5min":3.1560000000000001,"15min":3.1379999999999999},"min":{"1min":2.504,"5min":2.278,"15min":1.8680000000000001},"max":{"1min":4.6920000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":3.254}]},{"osd":7,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":2.875,"15min":2.887},"min":{"1min":2.2559999999999998,"5min":1.913,"15min":1.6000000000000001},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6469999999999998},"last":3.089},{"interface":"front","average":{"1min":3.3250000000000002,"5min":3.2450000000000001,"15min":3.1970000000000001},"min":{"1min":2.8170000000000002,"5min":2.1749999999999998,"15min":1.913},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":3.177}]}]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7716,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94364124,"statfs":{"total":96636764160,"available":96628862976,"internally_reserved":0,"allocated":1404928,"data_stored":644510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.581,"15min":1.611},"min":{"1min":1.24,"5min":0.95799999999999996,"15min":0.94899999999999995},"max":{"1min":2.871,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6180000000000001,"15min":1.639},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":1.879}]},{"osd":1,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":1.925,"15min":1.853},"min":{"1min":1.2390000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":2.1389999999999998},{"interface":"front","average":{"1min":2.379,"5min":2.4820000000000002,"15min":2.5089999999999999},"min":{"1min":1.417,"5min":1.417,"15min":1.417},"max":{"1min":3.4790000000000001,"5min":3.875,"15min":3.875},"last":2.621}]},{"osd":2,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.1720000000000002,"15min":2.2109999999999999},"min":{"1min":1.5,"5min":1.171,"15min":1.1259999999999999},"max":{"1min":2.8399999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":2.5619999999999998},{"interface":"front","average":{"1min":2.3559999999999999,"5min":2.327,"15min":2.3439999999999999},"min":{"1min":1.7330000000000001,"5min":1.583,"15min":1.258},"max":{"1min":3.0289999999999999,"5min":3.1739999999999999,"15min":3.4860000000000002},"last":2.4990000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:35:54 2020","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":2.226,"15min":2.2570000000000001},"min":{"1min":1.403,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":2.73,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.9830000000000001},{"interface":"front","average":{"1min":2.823,"5min":2.665,"15min":2.7410000000000001},"min":{"1min":2.0139999999999998,"5min":1.6559999999999999,"15min":1.5009999999999999},"max":{"1min":3.5070000000000001,"5min":3.7320000000000002,"15min":4.0830000000000002},"last":3.0819999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.5369999999999999,"5min":2.6080000000000001,"15min":2.6429999999999998},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.1509999999999998,"5min":5.6399999999999997,"15min":5.6399999999999997},"last":2.3380000000000001},{"interface":"front","average":{"1min":2.79,"5min":2.6680000000000001,"15min":2.8010000000000002},"min":{"1min":1.9890000000000001,"5min":1.171,"15min":1.171},"max":{"1min":3.6619999999999999,"5min":4.766,"15min":4.766},"last":2.8809999999999998}]},{"osd":5,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6869999999999998,"5min":2.5230000000000001,"15min":2.5249999999999999},"min":{"1min":1.821,"5min":1.5129999999999999,"15min":1.5129999999999999},"max":{"1min":3.4089999999999998,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":2.9359999999999999},{"interface":"front","average":{"1min":3.0259999999999998,"5min":3.0179999999999998,"15min":3.0800000000000001},"min":{"1min":2.2959999999999998,"5min":1.516,"15min":1.516},"max":{"1min":3.8079999999999998,"5min":6.0380000000000003,"15min":6.0380000000000003},"last":2.8340000000000001}]},{"osd":7,"last update":"Mon Nov 16 22:35:54 2020","interfaces":[{"interface":"back","average":{"1min":3.1240000000000001,"5min":3.073,"15min":3.21},"min":{"1min":2.5529999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"max":{"1min":3.7050000000000001,"5min":5.1379999999999999,"15min":5.3220000000000001},"last":3.5249999999999999},{"interface":"front","average":{"1min":2.984,"5min":3.0750000000000002,"15min":3.077},"min":{"1min":2.0910000000000002,"5min":2.0459999999999998,"15min":2.0459999999999998},"max":{"1min":3.7549999999999999,"5min":5.2569999999999997,"15min":5.2569999999999997},"last":3.3239999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8384,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94363456,"statfs":{"total":96636764160,"available":96628178944,"internally_reserved":0,"allocated":1499136,"data_stored":731943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.7410000000000001,"15min":1.8160000000000001},"min":{"1min":1.2410000000000001,"5min":0.96799999999999997,"15min":0.65200000000000002},"max":{"1min":3.0139999999999998,"5min":3.02,"15min":3.02},"last":1.3089999999999999},{"interface":"front","average":{"1min":2.2949999999999999,"5min":2.1720000000000002,"15min":2.1829999999999998},"min":{"1min":1.472,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.956,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":2.2639999999999998}]},{"osd":1,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.8029999999999999,"15min":1.7430000000000001},"min":{"1min":1.294,"5min":1.0580000000000001,"15min":0.74199999999999999},"max":{"1min":3.2360000000000002,"5min":3.302,"15min":3.302},"last":2.141},{"interface":"front","average":{"1min":2.0059999999999998,"5min":2.0259999999999998,"15min":1.8979999999999999},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.0660000000000001},"max":{"1min":3.0030000000000001,"5min":3.1869999999999998,"15min":3.3620000000000001},"last":1.6850000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.9410000000000001,"15min":1.9370000000000001},"min":{"1min":1.5069999999999999,"5min":1.29,"15min":1.054},"max":{"1min":2.931,"5min":4.117,"15min":4.117},"last":1.534},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.2309999999999999,"15min":2.113},"min":{"1min":1.3759999999999999,"5min":1.198,"15min":1.0129999999999999},"max":{"1min":3.0939999999999999,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.7769999999999999}]},{"osd":3,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":2.3759999999999999,"15min":2.2610000000000001},"min":{"1min":1.2370000000000001,"5min":1.206,"15min":0.94599999999999995},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":2.0800000000000001},{"interface":"front","average":{"1min":2.577,"5min":2.7290000000000001,"15min":2.75},"min":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.2719999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":3.0150000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4489999999999998},"min":{"1min":1.8779999999999999,"5min":1.361,"15min":1.075},"max":{"1min":3.448,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":2.734},{"interface":"front","average":{"1min":2.9809999999999999,"5min":2.9430000000000001,"15min":2.9060000000000001},"min":{"1min":2.258,"5min":1.6890000000000001,"15min":1.1160000000000001},"max":{"1min":4.0819999999999999,"5min":4.9859999999999998,"15min":4.9859999999999998},"last":2.9279999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.8999999999999999,"5min":2.851,"15min":2.8370000000000002},"min":{"1min":2.2080000000000002,"5min":1.78,"15min":1.5529999999999999},"max":{"1min":3.6850000000000001,"5min":4.3739999999999997,"15min":4.5220000000000002},"last":2.4380000000000002},{"interface":"front","average":{"1min":2.9700000000000002,"5min":2.8660000000000001,"15min":2.8180000000000001},"min":{"1min":2.1899999999999999,"5min":1.8959999999999999,"15min":1.6359999999999999},"max":{"1min":3.6429999999999998,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.899}]},{"osd":7,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.1259999999999999,"5min":3.1539999999999999,"15min":3.1110000000000002},"min":{"1min":2.3460000000000001,"5min":2.242,"15min":1.857},"max":{"1min":3.9449999999999998,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":2.6459999999999999},{"interface":"front","average":{"1min":3.069,"5min":2.9980000000000002,"15min":2.9670000000000001},"min":{"1min":2.544,"5min":1.8440000000000001,"15min":1.496},"max":{"1min":3.7890000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":2.5270000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542267,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7824,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94364016,"statfs":{"total":96636764160,"available":96628752384,"internally_reserved":0,"allocated":1384448,"data_stored":630427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.677,"15min":1.6339999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3199999999999998,"5min":2.6469999999999998,"15min":2.8159999999999998},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.665,"5min":1.72,"15min":1.694},"min":{"1min":1.1279999999999999,"5min":1.0449999999999999,"15min":1.02},"max":{"1min":2.0720000000000001,"5min":2.653,"15min":2.653},"last":1.516}]},{"osd":2,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6799999999999999,"15min":1.649},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.998,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.4570000000000001},{"interface":"front","average":{"1min":2.1259999999999999,"5min":2.1520000000000001,"15min":2.089},"min":{"1min":1.0149999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.641,"5min":2.9119999999999999,"15min":3.4119999999999999},"last":1.6739999999999999}]},{"osd":3,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1280000000000001,"5min":2.1619999999999999,"15min":2.0409999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.7160000000000002,"5min":3.1509999999999998,"15min":3.2559999999999998},"last":2.302},{"interface":"front","average":{"1min":2.3450000000000002,"5min":2.379,"15min":2.29},"min":{"1min":1.056,"5min":1.056,"15min":0.99399999999999999},"max":{"1min":3.2469999999999999,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":2.4180000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.5379999999999998,"5min":2.5649999999999999,"15min":2.6920000000000002},"min":{"1min":1.603,"5min":1.5129999999999999,"15min":1.5029999999999999},"max":{"1min":3.2210000000000001,"5min":3.8559999999999999,"15min":4.8479999999999999},"last":2.3740000000000001},{"interface":"front","average":{"1min":2.3889999999999998,"5min":2.4279999999999999,"15min":2.4169999999999998},"min":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.1930000000000001},"max":{"1min":3.319,"5min":3.956,"15min":4.0380000000000003},"last":1.964}]},{"osd":5,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8650000000000002,"5min":2.923,"15min":2.9540000000000002},"min":{"1min":2.2530000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"max":{"1min":3.5449999999999999,"5min":3.8700000000000001,"15min":5.0659999999999998},"last":2.2280000000000002},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.7530000000000001,"15min":2.661},"min":{"1min":1.8,"5min":1.6779999999999999,"15min":1.6180000000000001},"max":{"1min":3.3239999999999998,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":2.7240000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":2.8809999999999998,"15min":2.7989999999999999},"min":{"1min":1.772,"5min":1.772,"15min":1.772},"max":{"1min":3.3410000000000002,"5min":4.2599999999999998,"15min":4.2599999999999998},"last":2.8330000000000002},{"interface":"front","average":{"1min":3.125,"5min":3.1629999999999998,"15min":3.2010000000000001},"min":{"1min":2.2210000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"max":{"1min":3.9060000000000001,"5min":4.3490000000000002,"15min":5.2329999999999997},"last":2.6280000000000001}]},{"osd":7,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":3.3340000000000001,"5min":3.3370000000000002,"15min":3.3530000000000002},"min":{"1min":2.3999999999999999,"5min":2.1320000000000001,"15min":2.0489999999999999},"max":{"1min":4.2779999999999996,"5min":4.5910000000000002,"15min":5.6379999999999999},"last":2.5009999999999999},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.004,"15min":2.9399999999999999},"min":{"1min":2.036,"5min":2.036,"15min":1.895},"max":{"1min":3.5150000000000001,"5min":4.6059999999999999,"15min":4.6059999999999999},"last":3.0270000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542267,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7732,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94364108,"statfs":{"total":96636764160,"available":96628846592,"internally_reserved":0,"allocated":1486848,"data_stored":725455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.667,"15min":1.6339999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.70199999999999996},"max":{"1min":2.8450000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.79,"5min":1.7330000000000001,"15min":1.6859999999999999},"min":{"1min":1.3819999999999999,"5min":1.119,"15min":0.83699999999999997},"max":{"1min":2.6819999999999999,"5min":2.6829999999999998,"15min":2.718},"last":1.3839999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":1.6619999999999999,"5min":1.8029999999999999,"15min":1.716},"min":{"1min":1.3440000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.2789999999999999,"5min":10.023,"15min":10.023},"last":1.575},{"interface":"front","average":{"1min":2.1960000000000002,"5min":2.2440000000000002,"15min":2.097},"min":{"1min":1.345,"5min":0.95499999999999996,"15min":0.80400000000000005},"max":{"1min":3.2530000000000001,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":2.488}]},{"osd":3,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.0059999999999998,"5min":2.0609999999999999,"15min":2.1320000000000001},"min":{"1min":1.339,"5min":1.1319999999999999,"15min":0.81000000000000005},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":3.3319999999999999},"last":2.3399999999999999},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2519999999999998,"15min":2.2130000000000001},"min":{"1min":1.387,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.214,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":1.9379999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.226,"5min":2.383,"15min":2.3410000000000002},"min":{"1min":1.7410000000000001,"5min":1.008,"15min":1.008},"max":{"1min":3.0379999999999998,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":2.4169999999999998},{"interface":"front","average":{"1min":2.581,"5min":2.6739999999999999,"15min":2.6760000000000002},"min":{"1min":1.7969999999999999,"5min":1.022,"15min":1.022},"max":{"1min":3.379,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":2.0680000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":2.9750000000000001,"15min":3.0179999999999998},"min":{"1min":1.581,"5min":1.276,"15min":1.276},"max":{"1min":3.7639999999999998,"5min":5.351,"15min":5.351},"last":2.274},{"interface":"front","average":{"1min":2.778,"5min":2.7320000000000002,"15min":2.8050000000000002},"min":{"1min":2.153,"5min":1.4279999999999999,"15min":1.3839999999999999},"max":{"1min":3.4809999999999999,"5min":3.7349999999999999,"15min":3.863},"last":2.8140000000000001}]},{"osd":6,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.016,"5min":3.0009999999999999,"15min":3},"min":{"1min":2.2480000000000002,"5min":1.5269999999999999,"15min":1.5269999999999999},"max":{"1min":3.931,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":3.202},{"interface":"front","average":{"1min":2.8130000000000002,"5min":2.9239999999999999,"15min":2.8690000000000002},"min":{"1min":2.3180000000000001,"5min":1.8149999999999999,"15min":1.4670000000000001},"max":{"1min":3.6930000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":2.903}]},{"osd":7,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.1139999999999999,"5min":3.1230000000000002,"15min":3.0979999999999999},"min":{"1min":2.2000000000000002,"5min":2.008,"15min":1.7010000000000001},"max":{"1min":4.1239999999999997,"5min":4.1239999999999997,"15min":4.7430000000000003},"last":2.7200000000000002},{"interface":"front","average":{"1min":3.2610000000000001,"5min":3.1800000000000002,"15min":3.2120000000000002},"min":{"1min":2.6619999999999999,"5min":1.712,"15min":1.712},"max":{"1min":4.3899999999999997,"5min":4.3899999999999997,"15min":4.4029999999999996},"last":3.5760000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542268,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7936,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94363904,"statfs":{"total":96636764160,"available":96628637696,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.671,"15min":1.6519999999999999},"min":{"1min":0.84699999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":3.0710000000000002},"last":1.837},{"interface":"front","average":{"1min":1.845,"5min":1.776,"15min":1.7929999999999999},"min":{"1min":1.1950000000000001,"5min":1.032,"15min":0.81899999999999995},"max":{"1min":2.484,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":2.246}]},{"osd":1,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.946,"5min":1.897,"15min":1.837},"min":{"1min":1.1240000000000001,"5min":1.0069999999999999,"15min":0.91500000000000004},"max":{"1min":2.6400000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":2.1890000000000001},{"interface":"front","average":{"1min":2.0089999999999999,"5min":1.986,"15min":1.944},"min":{"1min":1.2929999999999999,"5min":0.81499999999999995,"15min":0.80500000000000005},"max":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.752}]},{"osd":3,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.911,"5min":1.8859999999999999,"15min":1.8620000000000001},"min":{"1min":1.2989999999999999,"5min":1.0840000000000001,"15min":0.92900000000000005},"max":{"1min":2.4929999999999999,"5min":3.0840000000000001,"15min":3.9660000000000002},"last":1.9279999999999999},{"interface":"front","average":{"1min":2.544,"5min":2.581,"15min":2.464},"min":{"1min":1.5860000000000001,"5min":1.1830000000000001,"15min":1.1499999999999999},"max":{"1min":3.238,"5min":3.944,"15min":3.944},"last":2.5830000000000002}]},{"osd":4,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":2.5979999999999999,"15min":2.512},"min":{"1min":1.782,"5min":1.5800000000000001,"15min":1.498},"max":{"1min":3.423,"5min":5.0209999999999999,"15min":5.0209999999999999},"last":2.407},{"interface":"front","average":{"1min":2.4249999999999998,"5min":2.5470000000000002,"15min":2.569},"min":{"1min":1.452,"5min":1.452,"15min":1.3260000000000001},"max":{"1min":3.1520000000000001,"5min":3.7400000000000002,"15min":3.8010000000000002},"last":2.4700000000000002}]},{"osd":5,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.6930000000000001,"5min":2.7909999999999999,"15min":2.8010000000000002},"min":{"1min":2.0219999999999998,"5min":1.6080000000000001,"15min":1.6080000000000001},"max":{"1min":3.5310000000000001,"5min":3.738,"15min":4.0259999999999998},"last":3.3809999999999998},{"interface":"front","average":{"1min":2.2440000000000002,"5min":2.4100000000000001,"15min":2.4540000000000002},"min":{"1min":1.639,"5min":1.639,"15min":1.363},"max":{"1min":2.9140000000000001,"5min":3.7970000000000002,"15min":4.3070000000000004},"last":3.5640000000000001}]},{"osd":6,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.589,"5min":2.6800000000000002,"15min":2.681},"min":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":3.6699999999999999,"5min":4.1059999999999999,"15min":4.165},"last":3.2639999999999998},{"interface":"front","average":{"1min":2.907,"5min":3.0430000000000001,"15min":2.9470000000000001},"min":{"1min":2.2360000000000002,"5min":2.0550000000000002,"15min":1.9930000000000001},"max":{"1min":3.5369999999999999,"5min":4.4450000000000003,"15min":4.4450000000000003},"last":2.8660000000000001}]},{"osd":7,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":3.0800000000000001,"15min":3.0790000000000002},"min":{"1min":2.4260000000000002,"5min":1.903,"15min":1.8879999999999999},"max":{"1min":3.8410000000000002,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":2.9980000000000002},{"interface":"front","average":{"1min":2.7850000000000001,"5min":2.9710000000000001,"15min":3.0089999999999999},"min":{"1min":1.488,"5min":1.488,"15min":1.488},"max":{"1min":3.6120000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":3.7549999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542266,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8252,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":6848,"kb_avail":94363588,"statfs":{"total":96636764160,"available":96628314112,"internally_reserved":0,"allocated":1429504,"data_stored":651180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7012352},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.677,"15min":1.6359999999999999},"min":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":0.73599999999999999},"max":{"1min":2.621,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.8169999999999999,"5min":1.7729999999999999,"15min":1.774},"min":{"1min":1.3660000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":3.6739999999999999},"last":1.9990000000000001}]},{"osd":1,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.6579999999999999,"15min":1.6339999999999999},"min":{"1min":1.1579999999999999,"5min":0.872,"15min":0.81100000000000005},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.77},"last":2.2229999999999999},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.1779999999999999,"15min":2.173},"min":{"1min":1.5800000000000001,"5min":1.25,"15min":1.099},"max":{"1min":5.258,"5min":5.258,"15min":5.258},"last":2.492}]},{"osd":2,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.1499999999999999,"5min":2.2010000000000001,"15min":2.1800000000000002},"min":{"1min":1.4750000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":3.1030000000000002,"5min":3.214,"15min":3.597},"last":2.2639999999999998},{"interface":"front","average":{"1min":2.488,"5min":2.3999999999999999,"15min":2.3759999999999999},"min":{"1min":1.776,"5min":1.3700000000000001,"15min":1.075},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0990000000000002},"last":2.0600000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.48,"5min":2.5779999999999998,"15min":2.5329999999999999},"min":{"1min":1.782,"5min":1.298,"15min":1.2150000000000001},"max":{"1min":3.5070000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":2.5920000000000001},{"interface":"front","average":{"1min":2.371,"5min":2.4009999999999998,"15min":2.403},"min":{"1min":1.758,"5min":1.403,"15min":1.1699999999999999},"max":{"1min":3.3210000000000002,"5min":3.681,"15min":3.6829999999999998},"last":1.9059999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7879999999999998,"15min":2.7530000000000001},"min":{"1min":1.9750000000000001,"5min":1.5600000000000001,"15min":1.2709999999999999},"max":{"1min":3.5950000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":2.645},{"interface":"front","average":{"1min":3.0529999999999999,"5min":2.9620000000000002,"15min":3.0009999999999999},"min":{"1min":2.1120000000000001,"5min":1.857,"15min":1.8360000000000001},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":2.9199999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.1680000000000001,"5min":3.0600000000000001,"15min":3.1080000000000001},"min":{"1min":2.4159999999999999,"5min":1.615,"15min":1.615},"max":{"1min":4.3970000000000002,"5min":4.3970000000000002,"15min":4.5359999999999996},"last":3.3079999999999998},{"interface":"front","average":{"1min":3.032,"5min":3.0609999999999999,"15min":3.093},"min":{"1min":2.1890000000000001,"5min":2.04,"15min":1.7310000000000001},"max":{"1min":3.895,"5min":4.1959999999999997,"15min":4.5590000000000002},"last":3.0110000000000001}]},{"osd":7,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.194,"5min":3.2309999999999999,"15min":3.1669999999999998},"min":{"1min":2.4380000000000002,"5min":2.0259999999999998,"15min":1.8240000000000001},"max":{"1min":4.343,"5min":4.343,"15min":4.6470000000000002},"last":2.8109999999999999},{"interface":"front","average":{"1min":2.9159999999999999,"5min":2.9649999999999999,"15min":2.9409999999999998},"min":{"1min":2.0390000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"max":{"1min":3.831,"5min":3.98,"15min":3.98},"last":2.4529999999999998}]}]}]}} 2020-11-16T22:36:48.497 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:48.623 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 Processor -- start 2020-11-16T22:36:48.624 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- start start 2020-11-16T22:36:48.625 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 0x7f7234167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7234166bb0 0x7f7234167040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7234168390 con 0x7f7234167ac0 2020-11-16T22:36:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340b5f00 con 0x7f72340b3930 2020-11-16T22:36:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340b2060 con 0x7f7234166bb0 2020-11-16T22:36:48.626 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7234166bb0 0x7f7234167040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7238947700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7234167ac0 0x7f72340b3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52556/0 (socket says 172.21.15.181:52556) 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7238947700 1 -- 172.21.15.181:0/1723115415 learned_addr learned my addr 172.21.15.181:0/1723115415 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 -- 172.21.15.181:0/1723115415 >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 msgr2=0x7f7234167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7233fff700 1 --2- 172.21.15.181:0/1723115415 >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 0x7f7234167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 --2- 172.21.15.181:0/1723115415 >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 0x7f7234167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.627 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 -- 172.21.15.181:0/1723115415 >> v1:172.21.15.181:6789/0 conn(0x7f7234167ac0 legacy=0x7f72340b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 -- 172.21.15.181:0/1723115415 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340b2300 con 0x7f7234166bb0 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7233fff700 1 --2- 172.21.15.181:0/1723115415 >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 0x7f7234167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f72337fe700 1 -- 172.21.15.181:0/1723115415 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031335851 0 0) 0x7f7234168390 con 0x7f7234167ac0 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f7239148700 1 --2- 172.21.15.181:0/1723115415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234167040 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f7234168990 tx=0x7f722c0091d0).ready entity=mon.1 client_cookie=19d2f722eb72b542 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f72337fe700 1 -- 172.21.15.181:0/1723115415 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c00bbc0 con 0x7f7234166bb0 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f72337fe700 1 -- 172.21.15.181:0/1723115415 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c00bd20 con 0x7f7234166bb0 2020-11-16T22:36:48.628 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f72337fe700 1 -- 172.21.15.181:0/1723115415 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c011610 con 0x7f7234166bb0 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- 172.21.15.181:0/1723115415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 msgr2=0x7f7234167040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 --2- 172.21.15.181:0/1723115415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234167040 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f7234168990 tx=0x7f722c0091d0).stop 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- 172.21.15.181:0/1723115415 shutdown_connections 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 --2- 172.21.15.181:0/1723115415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234167040 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 --2- 172.21.15.181:0/1723115415 >> v2:172.21.15.175:3300/0 conn(0x7f72340b3930 0x7f7234167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- 172.21.15.181:0/1723115415 >> 172.21.15.181:0/1723115415 conn(0x7f7234060bc0 msgr2=0x7f72340d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- 172.21.15.181:0/1723115415 shutdown_connections 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.635+0000 7f723abab700 1 -- 172.21.15.181:0/1723115415 wait complete. 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 Processor -- start 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- start start 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72340b3930 0x7f7234166080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72341778c0 con 0x7f72340b3930 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234061c90 con 0x7f7234166bb0 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7234061f30 con 0x7f7234167ac0 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7233fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46102/0 (socket says 172.21.15.181:46102) 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 -- 172.21.15.181:0/3381638361 learned_addr learned my addr 172.21.15.181:0/3381638361 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7233fff700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234163ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38260/0 (socket says 172.21.15.181:38260) 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7238947700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72340b3930 0x7f7234166080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 -- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 msgr2=0x7f7234163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 -- 172.21.15.181:0/3381638361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72340b3930 msgr2=0x7f7234166080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72340b3930 0x7f7234166080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 -- 172.21.15.181:0/3381638361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340621d0 con 0x7f7234167ac0 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7239148700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f722c0106d0 tx=0x7f722c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c00bbc0 con 0x7f7234167ac0 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c011e70 con 0x7f7234167ac0 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c010d20 con 0x7f7234167ac0 2020-11-16T22:36:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340624d0 con 0x7f7234167ac0 2020-11-16T22:36:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.639+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7234062ed0 con 0x7f7234167ac0 2020-11-16T22:36:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723405d2e0 con 0x7f7234167ac0 2020-11-16T22:36:48.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f722c024030 con 0x7f7234167ac0 2020-11-16T22:36:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f7230ff9700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 0x7f7210057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f722c06f980 con 0x7f7234167ac0 2020-11-16T22:36:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f7238947700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 0x7f7210057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.643+0000 7f7238947700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 0x7f7210057520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f721c0011a0 tx=0x7f721c007a00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:48.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.651+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f723405d2e0 con 0x7f7234167ac0 2020-11-16T22:36:48.793 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.803+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f723405d2e0 con 0x7f7234167ac0 2020-11-16T22:36:48.795 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f7230ff9700 1 -- 172.21.15.181:0/3381638361 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f722c030210 con 0x7f7234167ac0 2020-11-16T22:36:48.795 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:48.796 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"fsid":"0da1f45c-24e6-440b-b459-36ddfb20a11a","created":"2020-11-16T22:26:34.846483+0000","modified":"2020-11-16T22:35:23.571640+0000","last_up_change":"2020-11-16T22:26:42.695021+0000","last_in_change":"2020-11-16T22:26:42.695021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:26:38.724005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:26:46.352024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:35:20.412645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:35:21.362082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ddae4f96-c4ce-4f57-920e-c80b7e795739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":15203},{"type":"v1","addr":"172.21.15.181:6810","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":15203},{"type":"v1","addr":"172.21.15.181:6814","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":15203},{"type":"v1","addr":"172.21.15.181:6822","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":15203},{"type":"v1","addr":"172.21.15.181:6818","nonce":15203}]},"public_addr":"172.21.15.181:6810/15203","cluster_addr":"172.21.15.181:6814/15203","heartbeat_back_addr":"172.21.15.181:6822/15203","heartbeat_front_addr":"172.21.15.181:6818/15203","state":["exists","up"]},{"osd":1,"uuid":"47600a3e-3e36-414c-af17-37a175ef1fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":15204},{"type":"v1","addr":"172.21.15.181:6811","nonce":15204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":15204},{"type":"v1","addr":"172.21.15.181:6815","nonce":15204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":15204},{"type":"v1","addr":"172.21.15.181:6823","nonce":15204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":15204},{"type":"v1","addr":"172.21.15.181:6819","nonce":15204}]},"public_addr":"172.21.15.181:6811/15204","cluster_addr":"172.21.15.181:6815/15204","heartbeat_back_addr":"172.21.15.181:6823/15204","heartbeat_front_addr":"172.21.15.181:6819/15204","state":["exists","up"]},{"osd":2,"uuid":"0a826a69-584a-40e6-8be5-9dad35970e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":15202},{"type":"v1","addr":"172.21.15.181:6801","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":15202},{"type":"v1","addr":"172.21.15.181:6803","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":15202},{"type":"v1","addr":"172.21.15.181:6807","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":15202},{"type":"v1","addr":"172.21.15.181:6805","nonce":15202}]},"public_addr":"172.21.15.181:6801/15202","cluster_addr":"172.21.15.181:6803/15202","heartbeat_back_addr":"172.21.15.181:6807/15202","heartbeat_front_addr":"172.21.15.181:6805/15202","state":["exists","up"]},{"osd":3,"uuid":"dcbcd29f-2431-4d31-9f4b-84cb393eb95e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":15205},{"type":"v1","addr":"172.21.15.181:6825","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":15205},{"type":"v1","addr":"172.21.15.181:6827","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":15205},{"type":"v1","addr":"172.21.15.181:6831","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":15205},{"type":"v1","addr":"172.21.15.181:6829","nonce":15205}]},"public_addr":"172.21.15.181:6825/15205","cluster_addr":"172.21.15.181:6827/15205","heartbeat_back_addr":"172.21.15.181:6831/15205","heartbeat_front_addr":"172.21.15.181:6829/15205","state":["exists","up"]},{"osd":4,"uuid":"e6eb7f9b-c36a-46bb-b1b5-756336fffc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":15329},{"type":"v1","addr":"172.21.15.175:6801","nonce":15329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":15329},{"type":"v1","addr":"172.21.15.175:6803","nonce":15329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":15329},{"type":"v1","addr":"172.21.15.175:6807","nonce":15329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":15329},{"type":"v1","addr":"172.21.15.175:6805","nonce":15329}]},"public_addr":"172.21.15.175:6801/15329","cluster_addr":"172.21.15.175:6803/15329","heartbeat_back_addr":"172.21.15.175:6807/15329","heartbeat_front_addr":"172.21.15.175:6805/15329","state":["exists","up"]},{"osd":5,"uuid":"210445e9-3a4c-4b4a-af5d-b0399ac33558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":15351},{"type":"v1","addr":"172.21.15.175:6825","nonce":15351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":15351},{"type":"v1","addr":"172.21.15.175:6827","nonce":15351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":15351},{"type":"v1","addr":"172.21.15.175:6831","nonce":15351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":15351},{"type":"v1","addr":"172.21.15.175:6829","nonce":15351}]},"public_addr":"172.21.15.175:6825/15351","cluster_addr":"172.21.15.175:6827/15351","heartbeat_back_addr":"172.21.15.175:6831/15351","heartbeat_front_addr":"172.21.15.175:6829/15351","state":["exists","up"]},{"osd":6,"uuid":"a4d749e2-3789-4d8b-ab3f-56d1b413d88a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":15330},{"type":"v1","addr":"172.21.15.175:6809","nonce":15330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":15330},{"type":"v1","addr":"172.21.15.175:6811","nonce":15330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":15330},{"type":"v1","addr":"172.21.15.175:6815","nonce":15330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":15330},{"type":"v1","addr":"172.21.15.175:6813","nonce":15330}]},"public_addr":"172.21.15.175:6809/15330","cluster_addr":"172.21.15.175:6811/15330","heartbeat_back_addr":"172.21.15.175:6815/15330","heartbeat_front_addr":"172.21.15.175:6813/15330","state":["exists","up"]},{"osd":7,"uuid":"af0ccec7-e3da-4514-8621-b26a9adc9245","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":15354},{"type":"v1","addr":"172.21.15.175:6817","nonce":15354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":15354},{"type":"v1","addr":"172.21.15.175:6819","nonce":15354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":15354},{"type":"v1","addr":"172.21.15.175:6824","nonce":15354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":15354},{"type":"v1","addr":"172.21.15.175:6821","nonce":15354}]},"public_addr":"172.21.15.175:6817/15354","cluster_addr":"172.21.15.175:6819/15354","heartbeat_back_addr":"172.21.15.175:6824/15354","heartbeat_front_addr":"172.21.15.175:6821/15354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.671501+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.746502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.690265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.757386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.415714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.700792+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.640641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:26:40.689253+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:48.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 msgr2=0x7f7210057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 0x7f7210057520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f721c0011a0 tx=0x7f721c007a00).stop 2020-11-16T22:36:48.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 msgr2=0x7f72341643b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:48.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f722c0106d0 tx=0x7f722c016000).stop 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 shutdown_connections 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7210055060 0x7f7210057520 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7234167ac0 0x7f72341643b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72340b3930 0x7f7234166080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 --2- 172.21.15.181:0/3381638361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7234166bb0 0x7f7234163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 >> 172.21.15.181:0/3381638361 conn(0x7f7234060bc0 msgr2=0x7f72340b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:48.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 shutdown_connections 2020-11-16T22:36:48.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:48.807+0000 7f723abab700 1 -- 172.21.15.181:0/3381638361 wait complete. 2020-11-16T22:36:49.811 INFO:tasks.ceph:Scrubbing osd.0 2020-11-16T22:36:49.811 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:49.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.951+0000 7f3903507700 1 Processor -- start 2020-11-16T22:36:49.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.951+0000 7f3903507700 1 -- start start 2020-11-16T22:36:49.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.951+0000 7f3903507700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38fc15d280 0x7f38fc060c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.951+0000 7f3903507700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38fc061160 0x7f38fc080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc162240 con 0x7f38fc061160 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc1624e0 con 0x7f38fc15d280 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc162780 con 0x7f38fc0814c0 2020-11-16T22:36:49.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902505700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38fc15d280 0x7f38fc060c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46112/0 (socket says 172.21.15.181:46112) 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 -- 172.21.15.181:0/3964063121 learned_addr learned my addr 172.21.15.181:0/3964063121 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3901d04700 1 --2- 172.21.15.181:0/3964063121 >> v2:172.21.15.175:3300/0 conn(0x7f38fc061160 0x7f38fc080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 -- 172.21.15.181:0/3964063121 >> v2:172.21.15.181:3300/0 conn(0x7f38fc15d280 msgr2=0x7f38fc060c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 --2- 172.21.15.181:0/3964063121 >> v2:172.21.15.181:3300/0 conn(0x7f38fc15d280 0x7f38fc060c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 -- 172.21.15.181:0/3964063121 >> v2:172.21.15.175:3300/0 conn(0x7f38fc061160 msgr2=0x7f38fc080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:49.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 --2- 172.21.15.181:0/3964063121 >> v2:172.21.15.175:3300/0 conn(0x7f38fc061160 0x7f38fc080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 -- 172.21.15.181:0/3964063121 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc162a20 con 0x7f38fc0814c0 2020-11-16T22:36:49.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3902d06700 1 --2- 172.21.15.181:0/3964063121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f38fc05fc50 tx=0x7f38f4009290).ready entity=mon.2 client_cookie=f7cec721f1afc627 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:49.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3901503700 1 -- 172.21.15.181:0/3964063121 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400bbc0 con 0x7f38fc0814c0 2020-11-16T22:36:49.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3901503700 1 -- 172.21.15.181:0/3964063121 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400bd20 con 0x7f38fc0814c0 2020-11-16T22:36:49.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3901503700 1 -- 172.21.15.181:0/3964063121 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011610 con 0x7f38fc0814c0 2020-11-16T22:36:49.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- 172.21.15.181:0/3964063121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 msgr2=0x7f38fc0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 --2- 172.21.15.181:0/3964063121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f38fc05fc50 tx=0x7f38f4009290).stop 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- 172.21.15.181:0/3964063121 shutdown_connections 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 --2- 172.21.15.181:0/3964063121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc0814c0 0x7f38fc0d1100 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 --2- 172.21.15.181:0/3964063121 >> v2:172.21.15.181:3300/0 conn(0x7f38fc15d280 0x7f38fc060c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 --2- 172.21.15.181:0/3964063121 >> v2:172.21.15.175:3300/0 conn(0x7f38fc061160 0x7f38fc080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- 172.21.15.181:0/3964063121 >> 172.21.15.181:0/3964063121 conn(0x7f38fc0d6680 msgr2=0x7f38fc0d6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- 172.21.15.181:0/3964063121 shutdown_connections 2020-11-16T22:36:49.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 -- 172.21.15.181:0/3964063121 wait complete. 2020-11-16T22:36:49.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.955+0000 7f3903507700 1 Processor -- start 2020-11-16T22:36:49.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- start start 2020-11-16T22:36:49.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc061160 0x7f38fc1705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38fc0814c0 0x7f38fc158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 0x7f38fc0bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc170ab0 con 0x7f38fc0814c0 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0bae00 con 0x7f38fc15d280 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0bb0a0 con 0x7f38fc061160 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 0x7f38fc0bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902505700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc061160 0x7f38fc1705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 0x7f38fc0bbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38274/0 (socket says 172.21.15.181:38274) 2020-11-16T22:36:49.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 -- 172.21.15.181:0/1406534861 learned_addr learned my addr 172.21.15.181:0/1406534861 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3901d04700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38fc0814c0 0x7f38fc158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 -- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc061160 msgr2=0x7f38fc1705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc061160 0x7f38fc1705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 -- 172.21.15.181:0/1406534861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38fc0814c0 msgr2=0x7f38fc158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38fc0814c0 0x7f38fc158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0bb340 con 0x7f38fc15d280 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902505700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38fc061160 0x7f38fc1705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3902d06700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 0x7f38fc0bbfc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f38fc1628c0 tx=0x7f38f4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400bbc0 con 0x7f38fc15d280 2020-11-16T22:36:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400be90 con 0x7f38fc15d280 2020-11-16T22:36:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0bb5e0 con 0x7f38fc15d280 2020-11-16T22:36:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4010cf0 con 0x7f38fc15d280 2020-11-16T22:36:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.959+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc158e10 con 0x7f38fc15d280 2020-11-16T22:36:49.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f38fc0bbc00 con 0x7f38fc15d280 2020-11-16T22:36:49.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f38f4023030 con 0x7f38fc15d280 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38f406f450 con 0x7f38fc15d280 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f3902505700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f38d405c5a0 0x7f38d405e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38d405f010 con 0x7f38d405c5a0 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38f40719b0 con 0x7f38fc15d280 2020-11-16T22:36:49.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f3901d04700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f38d405c5a0 0x7f38d405e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:49.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f3902505700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f38fc0cbd20 tx=0x7f38f8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:49.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.963+0000 7f3901d04700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f38d405c5a0 0x7f38d405e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:49.956 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.967+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== osd.0 v2:172.21.15.181:6808/15203 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f38d405f010 con 0x7f38d405c5a0 2020-11-16T22:36:49.974 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.983+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 --> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f38fc044f60 con 0x7f38d405c5a0 2020-11-16T22:36:49.975 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f38ea7fc700 1 -- 172.21.15.181:0/1406534861 <== osd.0 v2:172.21.15.181:6808/15203 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f38fc044f60 con 0x7f38d405c5a0 2020-11-16T22:36:49.975 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:49.976 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:49.976 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f38d405c5a0 msgr2=0x7f38d405e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6808/15203,v1:172.21.15.181:6810/15203] conn(0x7f38d405c5a0 0x7f38d405e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 msgr2=0x7f38d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f38d4055060 0x7f38d4057520 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f38fc0cbd20 tx=0x7f38f8006df0).stop 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 msgr2=0x7f38fc0bbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:49.977 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 --2- 172.21.15.181:0/1406534861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38fc15d280 0x7f38fc0bbfc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f38fc1628c0 tx=0x7f38f4016000).stop 2020-11-16T22:36:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3902d06700 1 -- 172.21.15.181:0/1406534861 reap_dead start 2020-11-16T22:36:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 shutdown_connections 2020-11-16T22:36:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 >> 172.21.15.181:0/1406534861 conn(0x7f38fc0d6680 msgr2=0x7f38fc060a40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 shutdown_connections 2020-11-16T22:36:49.978 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:49.987+0000 7f3903507700 1 -- 172.21.15.181:0/1406534861 wait complete. 2020-11-16T22:36:49.994 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2020-11-16T22:36:50.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.131+0000 7f21e0011700 1 Processor -- start 2020-11-16T22:36:50.123 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.131+0000 7f21e0011700 1 -- start start 2020-11-16T22:36:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 0x7f21d8161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21d8160810 0x7f21d80600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8160cc0 con 0x7f21d80c85e0 2020-11-16T22:36:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80c8a40 con 0x7f21d8160810 2020-11-16T22:36:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cf030 con 0x7f21d8160430 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21d8160810 0x7f21d80600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21dddad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 0x7f21d8161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21d8160810 0x7f21d80600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38286/0 (socket says 172.21.15.181:38286) 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 -- 172.21.15.181:0/194664197 learned_addr learned my addr 172.21.15.181:0/194664197 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 -- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 msgr2=0x7f21d8161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 0x7f21d8161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 -- 172.21.15.181:0/194664197 >> v1:172.21.15.175:6789/0 conn(0x7f21d80c85e0 legacy=0x7f21d8162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 -- 172.21.15.181:0/194664197 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8161a10 con 0x7f21d8160810 2020-11-16T22:36:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21dddad700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 0x7f21d8161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21de5ae700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80600f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f21d0009d50 tx=0x7f21d0009290).ready entity=mon.1 client_cookie=ce7da19ffa08fc8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21dcdab700 1 -- 172.21.15.181:0/194664197 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000bbc0 con 0x7f21d8160810 2020-11-16T22:36:50.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21dcdab700 1 -- 172.21.15.181:0/194664197 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000bd20 con 0x7f21d8160810 2020-11-16T22:36:50.127 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21dcdab700 1 -- 172.21.15.181:0/194664197 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0011610 con 0x7f21d8160810 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 msgr2=0x7f21d80600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80600f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f21d0009d50 tx=0x7f21d0009290).stop 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- 172.21.15.181:0/194664197 shutdown_connections 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80600f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 --2- 172.21.15.181:0/194664197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d8160430 0x7f21d8161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.128 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- 172.21.15.181:0/194664197 >> 172.21.15.181:0/194664197 conn(0x7f21d80d3680 msgr2=0x7f21d80d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- 172.21.15.181:0/194664197 shutdown_connections 2020-11-16T22:36:50.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- 172.21.15.181:0/194664197 wait complete. 2020-11-16T22:36:50.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 Processor -- start 2020-11-16T22:36:50.129 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.135+0000 7f21e0011700 1 -- start start 2020-11-16T22:36:50.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.130 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 0x7f21d80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80d1740 con 0x7f21d8160430 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80c96b0 con 0x7f21d8160810 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80d1970 con 0x7f21d80c85e0 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46130/0 (socket says 172.21.15.181:46130) 2020-11-16T22:36:50.131 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 -- 172.21.15.181:0/3159277510 learned_addr learned my addr 172.21.15.181:0/3159277510 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21de5ae700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dd5ac700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 0x7f21d80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 -- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 msgr2=0x7f21d80b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 -- 172.21.15.181:0/3159277510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 msgr2=0x7f21d80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.132 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 0x7f21d80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 -- 172.21.15.181:0/3159277510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80d1c10 con 0x7f21d80c85e0 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21de5ae700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dd5ac700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 0x7f21d80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21dddad700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f21d400ef50 tx=0x7f21d4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d400cb30 con 0x7f21d80c85e0 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d400cc90 con 0x7f21d80c85e0 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4012600 con 0x7f21d80c85e0 2020-11-16T22:36:50.133 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80d1eb0 con 0x7f21d80c85e0 2020-11-16T22:36:50.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80d23f0 con 0x7f21d80c85e0 2020-11-16T22:36:50.134 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f21d4012970 con 0x7f21d80c85e0 2020-11-16T22:36:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.139+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8069eb0 con 0x7f21d80c85e0 2020-11-16T22:36:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.143+0000 7f21c5ffb700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 0x7f21b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.143+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f21d406e430 con 0x7f21d80c85e0 2020-11-16T22:36:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.143+0000 7f21dd5ac700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 0x7f21b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.135 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.143+0000 7f21dd5ac700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 0x7f21b0057520 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f21d81677f0 tx=0x7f21c8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.138 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.147+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f21d4044690 con 0x7f21d80c85e0 2020-11-16T22:36:50.297 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.307+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f21d80d2040 con 0x7f21b0055060 2020-11-16T22:36:50.298 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.307+0000 7f21c5ffb700 1 -- 172.21.15.181:0/3159277510 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f21d80d2040 con 0x7f21b0055060 2020-11-16T22:36:50.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 msgr2=0x7f21b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.301 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 0x7f21b0057520 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f21d81677f0 tx=0x7f21c8009040).stop 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 msgr2=0x7f21d80d2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f21d400ef50 tx=0x7f21d4006df0).stop 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 shutdown_connections 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f21b0055060 0x7f21b0057520 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21d8160810 0x7f21d80b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21d80c85e0 0x7f21d80d2b30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 --2- 172.21.15.181:0/3159277510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21d8160430 0x7f21d80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 >> 172.21.15.181:0/3159277510 conn(0x7f21d80d3680 msgr2=0x7f21d8163c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 shutdown_connections 2020-11-16T22:36:50.303 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.311+0000 7f21e0011700 1 -- 172.21.15.181:0/3159277510 wait complete. 2020-11-16T22:36:50.303 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 0 to deep-scrub 2020-11-16T22:36:50.318 INFO:tasks.ceph:Scrubbing osd.1 2020-11-16T22:36:50.318 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:50.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 Processor -- start 2020-11-16T22:36:50.447 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 -- start start 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f134415ad40 0x7f134415b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13441618b0 0x7f1344066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134415b610 con 0x7f1344065fd0 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1344066940 con 0x7f13441618b0 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134b0b5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1344066b30 con 0x7f134415ad40 2020-11-16T22:36:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f134415ad40 0x7f134415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f134415ad40 0x7f134415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38296/0 (socket says 172.21.15.181:38296) 2020-11-16T22:36:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 -- 172.21.15.181:0/1183197355 learned_addr learned my addr 172.21.15.181:0/1183197355 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a8b4700 1 --2- 172.21.15.181:0/1183197355 >> v2:172.21.15.175:3300/0 conn(0x7f13441618b0 0x7f1344066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 -- 172.21.15.181:0/1183197355 >> v2:172.21.15.175:3300/0 conn(0x7f13441618b0 msgr2=0x7f1344066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 --2- 172.21.15.181:0/1183197355 >> v2:172.21.15.175:3300/0 conn(0x7f13441618b0 0x7f1344066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 -- 172.21.15.181:0/1183197355 >> v1:172.21.15.175:6789/0 conn(0x7f1344065fd0 legacy=0x7f13441611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 -- 172.21.15.181:0/1183197355 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344066dd0 con 0x7f134415ad40 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f134a0b3700 1 --2- 172.21.15.181:0/1183197355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134415ad40 0x7f134415b120 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f13440d51a0 tx=0x7f1340006eb0).ready entity=mon.1 client_cookie=1ad176af70487a64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f13490b1700 1 -- 172.21.15.181:0/1183197355 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340009890 con 0x7f134415ad40 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f13490b1700 1 -- 172.21.15.181:0/1183197355 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13400099f0 con 0x7f134415ad40 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.459+0000 7f13490b1700 1 -- 172.21.15.181:0/1183197355 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340009d70 con 0x7f134415ad40 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- 172.21.15.181:0/1183197355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134415ad40 msgr2=0x7f134415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 --2- 172.21.15.181:0/1183197355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134415ad40 0x7f134415b120 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f13440d51a0 tx=0x7f1340006eb0).stop 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- 172.21.15.181:0/1183197355 shutdown_connections 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 --2- 172.21.15.181:0/1183197355 >> v2:172.21.15.175:3300/0 conn(0x7f13441618b0 0x7f1344066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 --2- 172.21.15.181:0/1183197355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134415ad40 0x7f134415b120 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- 172.21.15.181:0/1183197355 >> 172.21.15.181:0/1183197355 conn(0x7f13440d4460 msgr2=0x7f13440d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- 172.21.15.181:0/1183197355 shutdown_connections 2020-11-16T22:36:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- 172.21.15.181:0/1183197355 wait complete. 2020-11-16T22:36:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 Processor -- start 2020-11-16T22:36:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.463+0000 7f134b0b5700 1 -- start start 2020-11-16T22:36:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1344065fd0 0x7f1344157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134415ad40 0x7f134413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13441618b0 0x7f13441400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134416f3f0 con 0x7f134415ad40 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134416f690 con 0x7f13441618b0 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134416f930 con 0x7f1344065fd0 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a8b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13441618b0 0x7f13441400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1344065fd0 0x7f1344157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a8b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13441618b0 0x7f13441400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38304/0 (socket says 172.21.15.181:38304) 2020-11-16T22:36:50.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a8b4700 1 -- 172.21.15.181:0/2386829673 learned_addr learned my addr 172.21.15.181:0/2386829673 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f13498b2700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134415ad40 0x7f134413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 -- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13441618b0 msgr2=0x7f13441400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13441618b0 0x7f13441400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 -- 172.21.15.181:0/2386829673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134415ad40 msgr2=0x7f134413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134415ad40 0x7f134413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134416fbd0 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134a0b3700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1344065fd0 0x7f1344157840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f1340012000 tx=0x7f1340015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340011d50 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134001e460 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134416fe70 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134001e800 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1344154a70 con 0x7f1344065fd0 2020-11-16T22:36:50.460 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f134416f280 con 0x7f1344065fd0 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.467+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1340025030 con 0x7f1344065fd0 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f131c055060 0x7f131c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13498b2700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f131c055060 0x7f131c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f134006f610 con 0x7f1344065fd0 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f131c05c5a0 0x7f131c05e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.461 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f131c05f010 con 0x7f131c05c5a0 2020-11-16T22:36:50.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1340071c80 con 0x7f1344065fd0 2020-11-16T22:36:50.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13498b2700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f131c055060 0x7f131c057520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1344160e10 tx=0x7f1334007b70).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f134a8b4700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f131c05c5a0 0x7f131c05e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.462 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f134a8b4700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f131c05c5a0 0x7f131c05e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.463 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.471+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== osd.1 v2:172.21.15.181:6809/15204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f131c05f010 con 0x7f131c05c5a0 2020-11-16T22:36:50.480 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 --> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13441569b0 con 0x7f131c05c5a0 2020-11-16T22:36:50.481 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f13327fc700 1 -- 172.21.15.181:0/2386829673 <== osd.1 v2:172.21.15.181:6809/15204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f13441569b0 con 0x7f131c05c5a0 2020-11-16T22:36:50.481 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:50.481 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:50.481 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:50.482 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f131c05c5a0 msgr2=0x7f131c05e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6809/15204,v1:172.21.15.181:6811/15204] conn(0x7f131c05c5a0 0x7f131c05e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f131c055060 msgr2=0x7f131c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f131c055060 0x7f131c057520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1344160e10 tx=0x7f1334007b70).stop 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1344065fd0 msgr2=0x7f1344157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 --2- 172.21.15.181:0/2386829673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1344065fd0 0x7f1344157840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f1340012000 tx=0x7f1340015000).stop 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134a8b4700 1 -- 172.21.15.181:0/2386829673 reap_dead start 2020-11-16T22:36:50.483 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 shutdown_connections 2020-11-16T22:36:50.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 >> 172.21.15.181:0/2386829673 conn(0x7f13440d4460 msgr2=0x7f13440d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 shutdown_connections 2020-11-16T22:36:50.484 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.491+0000 7f134b0b5700 1 -- 172.21.15.181:0/2386829673 wait complete. 2020-11-16T22:36:50.502 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2020-11-16T22:36:50.631 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 Processor -- start 2020-11-16T22:36:50.632 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 -- start start 2020-11-16T22:36:50.633 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb46c15f5b0 0x7fb46c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c0615f0 0x7fb46c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c160e40 con 0x7fb46c0622a0 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c1609a0 con 0x7fb46c15f5b0 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb4738e6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c160ba0 con 0x7fb46c0615f0 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb470e81700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c0615f0 0x7fb46c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb46c15f5b0 0x7fb46c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.634 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471e83700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb46c0622a0 0x7fb46c062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52620/0 (socket says 172.21.15.181:52620) 2020-11-16T22:36:50.635 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471e83700 1 -- 172.21.15.181:0/3114443614 learned_addr learned my addr 172.21.15.181:0/3114443614 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 -- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c0615f0 msgr2=0x7fb46c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 --2- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c0615f0 0x7fb46c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 -- 172.21.15.181:0/3114443614 >> v1:172.21.15.181:6789/0 conn(0x7fb46c0622a0 legacy=0x7fb46c062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb45bfff700 1 -- 172.21.15.181:0/3114443614 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676003831 0 0) 0x7fb46c160e40 con 0x7fb46c0622a0 2020-11-16T22:36:50.636 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 -- 172.21.15.181:0/3114443614 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c060970 con 0x7fb46c15f5b0 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb471682700 1 --2- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c15f5b0 0x7fb46c0d1340 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fb46800b990 tx=0x7fb468006d90).ready entity=mon.1 client_cookie=ff0f5efe9811ca7a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb45bfff700 1 -- 172.21.15.181:0/3114443614 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468006030 con 0x7fb46c15f5b0 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb45bfff700 1 -- 172.21.15.181:0/3114443614 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468006190 con 0x7fb46c15f5b0 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.643+0000 7fb45bfff700 1 -- 172.21.15.181:0/3114443614 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468006510 con 0x7fb46c15f5b0 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c15f5b0 msgr2=0x7fb46c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c15f5b0 0x7fb46c0d1340 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fb46800b990 tx=0x7fb468006d90).stop 2020-11-16T22:36:50.637 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- 172.21.15.181:0/3114443614 shutdown_connections 2020-11-16T22:36:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c15f5b0 0x7fb46c0d1340 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3114443614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c0615f0 0x7fb46c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- 172.21.15.181:0/3114443614 >> 172.21.15.181:0/3114443614 conn(0x7fb46c0d69a0 msgr2=0x7fb46c0d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- 172.21.15.181:0/3114443614 shutdown_connections 2020-11-16T22:36:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- 172.21.15.181:0/3114443614 wait complete. 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 Processor -- start 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- start start 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb46c0622a0 0x7fb46c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c15f5b0 0x7fb46c0bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.640 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c16f4a0 con 0x7fb46c0622a0 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c16f740 con 0x7fb46c0615f0 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb4738e6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c16f9e0 con 0x7fb46c15f5b0 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471e83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c15f5b0 0x7fb46c0bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38318/0 (socket says 172.21.15.181:38318) 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 -- 172.21.15.181:0/3827839177 learned_addr learned my addr 172.21.15.181:0/3827839177 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.641 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb470e81700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb46c0622a0 0x7fb46c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 -- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c15f5b0 msgr2=0x7fb46c0bbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c15f5b0 0x7fb46c0bbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 -- 172.21.15.181:0/3827839177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb46c0622a0 msgr2=0x7fb46c153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb46c0622a0 0x7fb46c153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.647+0000 7fb471682700 1 -- 172.21.15.181:0/3827839177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c16fc80 con 0x7fb46c0615f0 2020-11-16T22:36:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb471682700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fb468006fd0 tx=0x7fb468015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680059b0 con 0x7fb46c0615f0 2020-11-16T22:36:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468005b10 con 0x7fb46c0615f0 2020-11-16T22:36:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c16ff20 con 0x7fb46c0615f0 2020-11-16T22:36:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46801e460 con 0x7fb46c0615f0 2020-11-16T22:36:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0bc390 con 0x7fb46c0615f0 2020-11-16T22:36:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c16f2d0 con 0x7fb46c0615f0 2020-11-16T22:36:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb468025030 con 0x7fb46c0615f0 2020-11-16T22:36:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 0x7fb444057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.644 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb46806f410 con 0x7fb46c0615f0 2020-11-16T22:36:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb470e81700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 0x7fb444057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.651+0000 7fb470e81700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 0x7fb444057520 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fb46c15a860 tx=0x7fb45c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.645 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.655+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb468045670 con 0x7fb46c0615f0 2020-11-16T22:36:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb46c069eb0 con 0x7fb444055060 2020-11-16T22:36:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4597fa700 1 -- 172.21.15.181:0/3827839177 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb46c069eb0 con 0x7fb444055060 2020-11-16T22:36:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 msgr2=0x7fb444057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 0x7fb444057520 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fb46c15a860 tx=0x7fb45c008040).stop 2020-11-16T22:36:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 msgr2=0x7fb46c153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.807+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fb468006fd0 tx=0x7fb468015000).stop 2020-11-16T22:36:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 shutdown_connections 2020-11-16T22:36:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb444055060 0x7fb444057520 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb46c15f5b0 0x7fb46c0bbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb46c0615f0 0x7fb46c153490 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.800 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 --2- 172.21.15.181:0/3827839177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb46c0622a0 0x7fb46c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 >> 172.21.15.181:0/3827839177 conn(0x7fb46c0d69a0 msgr2=0x7fb46c062700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 shutdown_connections 2020-11-16T22:36:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.811+0000 7fb4738e6700 1 -- 172.21.15.181:0/3827839177 wait complete. 2020-11-16T22:36:50.801 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 1 to deep-scrub 2020-11-16T22:36:50.812 INFO:tasks.ceph:Scrubbing osd.2 2020-11-16T22:36:50.812 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 Processor -- start 2020-11-16T22:36:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 -- start start 2020-11-16T22:36:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8dc065fd0 0x7fe8dc066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc15b610 con 0x7fe8dc1611a0 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc066940 con 0x7fe8dc065fd0 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e3052700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc066b30 con 0x7fe8dc15ad40 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.942 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46170/0 (socket says 172.21.15.181:46170) 2020-11-16T22:36:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 -- 172.21.15.181:0/4112367496 learned_addr learned my addr 172.21.15.181:0/4112367496 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.943 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e184f700 1 --2- 172.21.15.181:0/4112367496 >> v2:172.21.15.175:3300/0 conn(0x7fe8dc065fd0 0x7fe8dc066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 -- 172.21.15.181:0/4112367496 >> v2:172.21.15.175:3300/0 conn(0x7fe8dc065fd0 msgr2=0x7fe8dc066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 --2- 172.21.15.181:0/4112367496 >> v2:172.21.15.175:3300/0 conn(0x7fe8dc065fd0 0x7fe8dc066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 -- 172.21.15.181:0/4112367496 >> v1:172.21.15.175:6789/0 conn(0x7fe8dc1611a0 legacy=0x7fe8dc161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 -- 172.21.15.181:0/4112367496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc066dd0 con 0x7fe8dc15ad40 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e2050700 1 --2- 172.21.15.181:0/4112367496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fe8d0006fd0 tx=0x7fe8d0006eb0).ready entity=mon.2 client_cookie=407d9655683c3c51 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e104e700 1 -- 172.21.15.181:0/4112367496 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0009810 con 0x7fe8dc15ad40 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e104e700 1 -- 172.21.15.181:0/4112367496 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0009970 con 0x7fe8dc15ad40 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.951+0000 7fe8e104e700 1 -- 172.21.15.181:0/4112367496 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0009cf0 con 0x7fe8dc15ad40 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- 172.21.15.181:0/4112367496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 msgr2=0x7fe8dc15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 --2- 172.21.15.181:0/4112367496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fe8d0006fd0 tx=0x7fe8d0006eb0).stop 2020-11-16T22:36:50.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- 172.21.15.181:0/4112367496 shutdown_connections 2020-11-16T22:36:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 --2- 172.21.15.181:0/4112367496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15ad40 0x7fe8dc15b120 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 --2- 172.21.15.181:0/4112367496 >> v2:172.21.15.175:3300/0 conn(0x7fe8dc065fd0 0x7fe8dc066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- 172.21.15.181:0/4112367496 >> 172.21.15.181:0/4112367496 conn(0x7fe8dc0d4460 msgr2=0x7fe8dc0d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- 172.21.15.181:0/4112367496 shutdown_connections 2020-11-16T22:36:50.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- 172.21.15.181:0/4112367496 wait complete. 2020-11-16T22:36:50.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 Processor -- start 2020-11-16T22:36:50.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.955+0000 7fe8e3052700 1 -- start start 2020-11-16T22:36:50.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 0x7fe8dc15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8dc15ad40 0x7fe8dc0b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 0x7fe8dc16e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc15bc90 con 0x7fe8dc15ad40 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b75f0 con 0x7fe8dc065fd0 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b7890 con 0x7fe8dc1611a0 2020-11-16T22:36:50.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2050700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 0x7fe8dc15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 0x7fe8dc16e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 0x7fe8dc16e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46178/0 (socket says 172.21.15.181:46178) 2020-11-16T22:36:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 -- 172.21.15.181:0/1668682261 learned_addr learned my addr 172.21.15.181:0/1668682261 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2050700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 0x7fe8dc15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38332/0 (socket says 172.21.15.181:38332) 2020-11-16T22:36:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e184f700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8dc15ad40 0x7fe8dc0b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 -- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 msgr2=0x7fe8dc15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 0x7fe8dc15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 -- 172.21.15.181:0/1668682261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8dc15ad40 msgr2=0x7fe8dc0b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8dc15ad40 0x7fe8dc0b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0b7b30 con 0x7fe8dc1611a0 2020-11-16T22:36:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2050700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc065fd0 0x7fe8dc15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e2851700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 0x7fe8dc16e430 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fe8d400c710 tx=0x7fe8d4007af0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4013410 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4013570 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0b7dd0 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4013910 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0b8ca0 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8dc0b83f0 con 0x7fe8dc1611a0 2020-11-16T22:36:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe8d4010030 con 0x7fe8dc1611a0 2020-11-16T22:36:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.959+0000 7fe8da7fc700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe8b4055060 0x7fe8b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe8d406da10 con 0x7fe8dc1611a0 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8e2050700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe8b4055060 0x7fe8b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8da7fc700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7fe8b405c5a0 0x7fe8b405e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8b405f010 con 0x7fe8b405c5a0 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d4072220 con 0x7fe8dc1611a0 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8e184f700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7fe8b405c5a0 0x7fe8b405e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8e2050700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe8b4055060 0x7fe8b4057520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fe8d000b1b0 tx=0x7fe8d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8e184f700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7fe8b405c5a0 0x7fe8b405e9c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:50.955 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.963+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== osd.2 v2:172.21.15.181:6800/15202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fe8b405f010 con 0x7fe8b405c5a0 2020-11-16T22:36:50.969 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 --> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe8dc044f60 con 0x7fe8b405c5a0 2020-11-16T22:36:50.970 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8da7fc700 1 -- 172.21.15.181:0/1668682261 <== osd.2 v2:172.21.15.181:6800/15202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe8dc044f60 con 0x7fe8b405c5a0 2020-11-16T22:36:50.970 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:50.970 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:50.970 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7fe8b405c5a0 msgr2=0x7fe8b405e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.971 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6800/15202,v1:172.21.15.181:6801/15202] conn(0x7fe8b405c5a0 0x7fe8b405e9c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe8b4055060 msgr2=0x7fe8b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe8b4055060 0x7fe8b4057520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fe8d000b1b0 tx=0x7fe8d0015000).stop 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 msgr2=0x7fe8dc16e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 --2- 172.21.15.181:0/1668682261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc1611a0 0x7fe8dc16e430 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fe8d400c710 tx=0x7fe8d4007af0).stop 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e2851700 1 -- 172.21.15.181:0/1668682261 reap_dead start 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 shutdown_connections 2020-11-16T22:36:50.972 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 >> 172.21.15.181:0/1668682261 conn(0x7fe8dc0d4460 msgr2=0x7fe8dc0d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:50.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 shutdown_connections 2020-11-16T22:36:50.973 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:50.979+0000 7fe8e3052700 1 -- 172.21.15.181:0/1668682261 wait complete. 2020-11-16T22:36:50.983 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2020-11-16T22:36:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 Processor -- start 2020-11-16T22:36:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 -- start start 2020-11-16T22:36:51.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 0x7f459c161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459c1621b0 con 0x7f459c0637b0 2020-11-16T22:36:51.110 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c065930 con 0x7f459c0633d0 2020-11-16T22:36:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a4346700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c064970 con 0x7f459c061810 2020-11-16T22:36:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a18e1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f459c0637b0 0x7f459c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52648/0 (socket says 172.21.15.181:52648) 2020-11-16T22:36:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a18e1700 1 -- 172.21.15.181:0/1918457466 learned_addr learned my addr 172.21.15.181:0/1918457466 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a28e3700 1 --2- 172.21.15.181:0/1918457466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a20e2700 1 --2- 172.21.15.181:0/1918457466 >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 0x7f459c161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.112 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a10e0700 1 -- 172.21.15.181:0/1918457466 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218214131 0 0) 0x7f459c1621b0 con 0x7f459c0637b0 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.119+0000 7f45a10e0700 1 -- 172.21.15.181:0/1918457466 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4584003240 con 0x7f459c0637b0 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a28e3700 1 -- 172.21.15.181:0/1918457466 >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 msgr2=0x7f459c161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a28e3700 1 --2- 172.21.15.181:0/1918457466 >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 0x7f459c161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a28e3700 1 -- 172.21.15.181:0/1918457466 >> v1:172.21.15.181:6789/0 conn(0x7f459c0637b0 legacy=0x7f459c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a28e3700 1 -- 172.21.15.181:0/1918457466 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c064c10 con 0x7f459c061810 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a20e2700 1 --2- 172.21.15.181:0/1918457466 >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 0x7f459c161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:51.113 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a28e3700 1 --2- 172.21.15.181:0/1918457466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c062330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f4594009c90 tx=0x7f45940091d0).ready entity=mon.2 client_cookie=6bc7fe04fecdb33f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a10e0700 1 -- 172.21.15.181:0/1918457466 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400b9c0 con 0x7f459c061810 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a10e0700 1 -- 172.21.15.181:0/1918457466 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400bb20 con 0x7f459c061810 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a10e0700 1 -- 172.21.15.181:0/1918457466 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400bea0 con 0x7f459c061810 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- 172.21.15.181:0/1918457466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 msgr2=0x7f459c062330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 --2- 172.21.15.181:0/1918457466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c062330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f4594009c90 tx=0x7f45940091d0).stop 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- 172.21.15.181:0/1918457466 shutdown_connections 2020-11-16T22:36:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 --2- 172.21.15.181:0/1918457466 >> v2:172.21.15.175:3300/0 conn(0x7f459c0633d0 0x7f459c161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 --2- 172.21.15.181:0/1918457466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c062330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- 172.21.15.181:0/1918457466 >> 172.21.15.181:0/1918457466 conn(0x7f459c0cf890 msgr2=0x7f459c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- 172.21.15.181:0/1918457466 shutdown_connections 2020-11-16T22:36:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- 172.21.15.181:0/1918457466 wait complete. 2020-11-16T22:36:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 Processor -- start 2020-11-16T22:36:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.123+0000 7f45a4346700 1 -- start start 2020-11-16T22:36:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 0x7f459c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 0x7f459c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0d0e70 con 0x7f459c0637b0 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0d1110 con 0x7f459c0633d0 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0d13b0 con 0x7f459c061810 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a18e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 0x7f459c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46190/0 (socket says 172.21.15.181:46190) 2020-11-16T22:36:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 -- 172.21.15.181:0/2487524342 learned_addr learned my addr 172.21.15.181:0/2487524342 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a28e3700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 0x7f459c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 -- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 msgr2=0x7f459c151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 0x7f459c151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 -- 172.21.15.181:0/2487524342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 msgr2=0x7f459c151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 0x7f459c151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.119 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 -- 172.21.15.181:0/2487524342 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0d1650 con 0x7f459c061810 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a28e3700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 0x7f459c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a18e1700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 0x7f459c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a20e2700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f459800ef50 tx=0x7f4598006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800cb30 con 0x7f459c061810 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459800cc90 con 0x7f459c061810 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c0d18f0 con 0x7f459c061810 2020-11-16T22:36:51.120 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598012600 con 0x7f459c061810 2020-11-16T22:36:51.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c15e870 con 0x7f459c061810 2020-11-16T22:36:51.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c069eb0 con 0x7f459c061810 2020-11-16T22:36:51.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.127+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4598012970 con 0x7f459c061810 2020-11-16T22:36:51.121 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.131+0000 7f458a7fc700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 0x7f4574057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.131+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f459806e360 con 0x7f459c061810 2020-11-16T22:36:51.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.131+0000 7f45a18e1700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 0x7f4574057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.122 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.131+0000 7f45a18e1700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 0x7f4574057550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f459c156160 tx=0x7f458c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.135+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f45980445c0 con 0x7f459c061810 2020-11-16T22:36:51.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.287+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f459c0d0cf0 con 0x7f4574055090 2020-11-16T22:36:51.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.287+0000 7f458a7fc700 1 -- 172.21.15.181:0/2487524342 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f458c013420 con 0x7f4574055090 2020-11-16T22:36:51.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 msgr2=0x7f4574057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.279 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 0x7f4574057550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f459c156160 tx=0x7f458c009040).stop 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 msgr2=0x7f459c15e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f459800ef50 tx=0x7f4598006df0).stop 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 shutdown_connections 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f4574055090 0x7f4574057550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f459c0637b0 0x7f459c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.280 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f459c061810 0x7f459c15e380 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 --2- 172.21.15.181:0/2487524342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f459c0633d0 0x7f459c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 >> 172.21.15.181:0/2487524342 conn(0x7f459c0cf890 msgr2=0x7f459c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 shutdown_connections 2020-11-16T22:36:51.281 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.291+0000 7f45a4346700 1 -- 172.21.15.181:0/2487524342 wait complete. 2020-11-16T22:36:51.281 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 2 to deep-scrub 2020-11-16T22:36:51.293 INFO:tasks.ceph:Scrubbing osd.3 2020-11-16T22:36:51.294 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:51.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.431+0000 7f263a0ba700 1 Processor -- start 2020-11-16T22:36:51.422 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.431+0000 7f263a0ba700 1 -- start start 2020-11-16T22:36:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2634145a60 0x7f2634145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 0x7f2634080fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26340cb950 con 0x7f2634061a30 2020-11-16T22:36:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26340cbbf0 con 0x7f2634145a60 2020-11-16T22:36:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340cbe90 con 0x7f26340615b0 2020-11-16T22:36:51.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f2632ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 0x7f2634080fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f2632ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 0x7f2634080fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46198/0 (socket says 172.21.15.181:46198) 2020-11-16T22:36:51.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f2632ffd700 1 -- 172.21.15.181:0/3550594679 learned_addr learned my addr 172.21.15.181:0/3550594679 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.425 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 --2- 172.21.15.181:0/3550594679 >> v2:172.21.15.175:3300/0 conn(0x7f2634145a60 0x7f2634145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 -- 172.21.15.181:0/3550594679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 msgr2=0x7f2634080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 --2- 172.21.15.181:0/3550594679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 0x7f2634080fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 -- 172.21.15.181:0/3550594679 >> v1:172.21.15.175:6789/0 conn(0x7f2634061a30 legacy=0x7f26340814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 -- 172.21.15.181:0/3550594679 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340cc130 con 0x7f2634145a60 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26337fe700 1 --2- 172.21.15.181:0/3550594679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634145a60 0x7f2634145e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f262400b9d0 tx=0x7f2624006df0).ready entity=mon.0 client_cookie=756e62bd6e7143fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26327fc700 1 -- 172.21.15.181:0/3550594679 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624009770 con 0x7f2634145a60 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26327fc700 1 -- 172.21.15.181:0/3550594679 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26240098d0 con 0x7f2634145a60 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f26327fc700 1 -- 172.21.15.181:0/3550594679 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624009c50 con 0x7f2634145a60 2020-11-16T22:36:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- 172.21.15.181:0/3550594679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634145a60 msgr2=0x7f2634145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 --2- 172.21.15.181:0/3550594679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634145a60 0x7f2634145e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f262400b9d0 tx=0x7f2624006df0).stop 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- 172.21.15.181:0/3550594679 shutdown_connections 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 --2- 172.21.15.181:0/3550594679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634145a60 0x7f2634145e40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 --2- 172.21.15.181:0/3550594679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26340615b0 0x7f2634080fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- 172.21.15.181:0/3550594679 >> 172.21.15.181:0/3550594679 conn(0x7f26340d68e0 msgr2=0x7f26340d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- 172.21.15.181:0/3550594679 shutdown_connections 2020-11-16T22:36:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.435+0000 7f263a0ba700 1 -- 172.21.15.181:0/3550594679 wait complete. 2020-11-16T22:36:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 Processor -- start 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 -- start start 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26340615b0 0x7f2634161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634061a30 0x7f26340d3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2634145a60 0x7f26340d3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340b99f0 con 0x7f2634061a30 2020-11-16T22:36:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340b9c90 con 0x7f26340615b0 2020-11-16T22:36:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340b9f30 con 0x7f2634145a60 2020-11-16T22:36:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26340615b0 0x7f2634161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2634145a60 0x7f26340d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2632ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634061a30 0x7f26340d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26340615b0 0x7f2634161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38364/0 (socket says 172.21.15.181:38364) 2020-11-16T22:36:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26337fe700 1 -- 172.21.15.181:0/1509414843 learned_addr learned my addr 172.21.15.181:0/1509414843 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 -- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26340615b0 msgr2=0x7f2634161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26340615b0 0x7f2634161cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 -- 172.21.15.181:0/1509414843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634061a30 msgr2=0x7f26340d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634061a30 0x7f26340d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340ba1d0 con 0x7f2634145a60 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2632ffd700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634061a30 0x7f26340d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f2633fff700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2634145a60 0x7f26340d3850 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f2634061ec0 tx=0x7f262c007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00ce00 con 0x7f2634145a60 2020-11-16T22:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c00a730 con 0x7f2634145a60 2020-11-16T22:36:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00aab0 con 0x7f2634145a60 2020-11-16T22:36:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.439+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26340ba470 con 0x7f2634145a60 2020-11-16T22:36:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341621e0 con 0x7f2634145a60 2020-11-16T22:36:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f262c019030 con 0x7f2634145a60 2020-11-16T22:36:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2634069eb0 con 0x7f2634145a60 2020-11-16T22:36:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f260c055060 0x7f260c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f262c06d610 con 0x7f2634145a60 2020-11-16T22:36:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26337fe700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f260c055060 0x7f260c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7f260c05c5a0 0x7f260c05e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f260c05f010 con 0x7f260c05c5a0 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f262c06fb70 con 0x7f2634145a60 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f26337fe700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f260c055060 0x7f260c057520 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f262400be90 tx=0x7f2624015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.443+0000 7f2632ffd700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7f260c05c5a0 0x7f260c05e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.447+0000 7f2632ffd700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7f260c05c5a0 0x7f260c05e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.447+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== osd.3 v2:172.21.15.181:6824/15205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f260c05f010 con 0x7f260c05c5a0 2020-11-16T22:36:51.454 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.463+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 --> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f26340bca10 con 0x7f260c05c5a0 2020-11-16T22:36:51.455 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f26137fe700 1 -- 172.21.15.181:0/1509414843 <== osd.3 v2:172.21.15.181:6824/15205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f26340bca10 con 0x7f260c05c5a0 2020-11-16T22:36:51.455 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:51.455 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7f260c05c5a0 msgr2=0x7f260c05e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6824/15205,v1:172.21.15.181:6825/15205] conn(0x7f260c05c5a0 0x7f260c05e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f260c055060 msgr2=0x7f260c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f260c055060 0x7f260c057520 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f262400be90 tx=0x7f2624015000).stop 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2634145a60 msgr2=0x7f26340d3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 --2- 172.21.15.181:0/1509414843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2634145a60 0x7f26340d3850 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f2634061ec0 tx=0x7f262c007e20).stop 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f2633fff700 1 -- 172.21.15.181:0/1509414843 reap_dead start 2020-11-16T22:36:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 shutdown_connections 2020-11-16T22:36:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 >> 172.21.15.181:0/1509414843 conn(0x7f26340d68e0 msgr2=0x7f2634144af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 shutdown_connections 2020-11-16T22:36:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.467+0000 7f263a0ba700 1 -- 172.21.15.181:0/1509414843 wait complete. 2020-11-16T22:36:51.470 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2020-11-16T22:36:51.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.603+0000 7ff674b4c700 1 Processor -- start 2020-11-16T22:36:51.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.603+0000 7ff674b4c700 1 -- start start 2020-11-16T22:36:51.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 0x7ff670145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff6700816e0 0x7ff6700cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff670061b50 con 0x7ff6701492c0 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6700618b0 con 0x7ff6700816e0 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700cb930 con 0x7ff670145a30 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff6700816e0 0x7ff6700cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff6701492c0 0x7ff670080fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52680/0 (socket says 172.21.15.181:52680) 2020-11-16T22:36:51.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66effd700 1 -- 172.21.15.181:0/877366386 learned_addr learned my addr 172.21.15.181:0/877366386 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:51.598 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66f7fe700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 0x7ff670145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 -- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 msgr2=0x7ff670145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 0x7ff670145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 -- 172.21.15.181:0/877366386 >> v1:172.21.15.181:6789/0 conn(0x7ff6701492c0 legacy=0x7ff670080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 -- 172.21.15.181:0/877366386 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700cbbd0 con 0x7ff6700816e0 2020-11-16T22:36:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66e7fc700 1 -- 172.21.15.181:0/877366386 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896266635 0 0) 0x7ff670061b50 con 0x7ff6701492c0 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66f7fe700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 0x7ff670145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66ffff700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff6700cc5b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7ff668009c90 tx=0x7ff6680091d0).ready entity=mon.1 client_cookie=e79c317f607e0b6b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66e7fc700 1 -- 172.21.15.181:0/877366386 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66800ba70 con 0x7ff6700816e0 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66e7fc700 1 -- 172.21.15.181:0/877366386 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66800bbd0 con 0x7ff6700816e0 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff66e7fc700 1 -- 172.21.15.181:0/877366386 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668011400 con 0x7ff6700816e0 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 msgr2=0x7ff6700cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff6700cc5b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7ff668009c90 tx=0x7ff6680091d0).stop 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- 172.21.15.181:0/877366386 shutdown_connections 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff6700cc5b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 --2- 172.21.15.181:0/877366386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670145a30 0x7ff670145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- 172.21.15.181:0/877366386 >> 172.21.15.181:0/877366386 conn(0x7ff6700d68c0 msgr2=0x7ff6700d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- 172.21.15.181:0/877366386 shutdown_connections 2020-11-16T22:36:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.607+0000 7ff674b4c700 1 -- 172.21.15.181:0/877366386 wait complete. 2020-11-16T22:36:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 Processor -- start 2020-11-16T22:36:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 -- start start 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff670145a30 0x7ff67013f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6701492c0 0x7ff67013fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670140200 con 0x7ff670145a30 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700d20d0 con 0x7ff6700816e0 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff674b4c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff670172f20 con 0x7ff6701492c0 2020-11-16T22:36:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6701492c0 0x7ff67013fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38380/0 (socket says 172.21.15.181:38380) 2020-11-16T22:36:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 -- 172.21.15.181:0/4243801339 learned_addr learned my addr 172.21.15.181:0/4243801339 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66effd700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff670145a30 0x7ff67013f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 -- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6701492c0 msgr2=0x7ff67013fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6701492c0 0x7ff67013fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 -- 172.21.15.181:0/4243801339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff670145a30 msgr2=0x7ff67013f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.611+0000 7ff66f7fe700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff670145a30 0x7ff67013f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff66f7fe700 1 -- 172.21.15.181:0/4243801339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6701731c0 con 0x7ff6700816e0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff66f7fe700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7ff65800ef50 tx=0x7ff658006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800b5c0 con 0x7ff6700816e0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65800ca80 con 0x7ff6700816e0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff670173460 con 0x7ff6700816e0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800ce00 con 0x7ff6700816e0 2020-11-16T22:36:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6701739a0 con 0x7ff6700816e0 2020-11-16T22:36:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670044f60 con 0x7ff6700816e0 2020-11-16T22:36:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff658012600 con 0x7ff6700816e0 2020-11-16T22:36:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 0x7ff64c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff66effd700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 0x7ff64c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff65806e360 con 0x7ff6700816e0 2020-11-16T22:36:51.608 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.615+0000 7ff66effd700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 0x7ff64c057520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7ff6700d0c00 tx=0x7ff6600079e0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.612 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.623+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff6580444b0 con 0x7ff6700816e0 2020-11-16T22:36:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.775+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6700d0c30 con 0x7ff64c055060 2020-11-16T22:36:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.775+0000 7ff6677fe700 1 -- 172.21.15.181:0/4243801339 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff6700d0c30 con 0x7ff64c055060 2020-11-16T22:36:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 msgr2=0x7ff64c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 0x7ff64c057520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7ff6700d0c00 tx=0x7ff6600079e0).stop 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 msgr2=0x7ff67015d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7ff65800ef50 tx=0x7ff658006df0).stop 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 shutdown_connections 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff64c055060 0x7ff64c057520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6701492c0 0x7ff67013fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6700816e0 0x7ff67015d1b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 --2- 172.21.15.181:0/4243801339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff670145a30 0x7ff67013f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 >> 172.21.15.181:0/4243801339 conn(0x7ff6700d68c0 msgr2=0x7ff670080fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 shutdown_connections 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.779+0000 7ff674b4c700 1 -- 172.21.15.181:0/4243801339 wait complete. 2020-11-16T22:36:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 3 to deep-scrub 2020-11-16T22:36:51.783 INFO:tasks.ceph:Scrubbing osd.4 2020-11-16T22:36:51.783 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:51.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.919+0000 7ff3608f2700 1 Processor -- start 2020-11-16T22:36:51.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.919+0000 7ff3608f2700 1 -- start start 2020-11-16T22:36:51.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff35c142ad0 0x7ff35c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c081a40 con 0x7ff35c0d1060 2020-11-16T22:36:51.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35c0d1800 con 0x7ff35c142ad0 2020-11-16T22:36:51.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0d1aa0 con 0x7ff35c081170 2020-11-16T22:36:51.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46228/0 (socket says 172.21.15.181:46228) 2020-11-16T22:36:51.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 -- 172.21.15.181:0/1566268 learned_addr learned my addr 172.21.15.181:0/1566268 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35b59e700 1 --2- 172.21.15.181:0/1566268 >> v2:172.21.15.175:3300/0 conn(0x7ff35c142ad0 0x7ff35c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 -- 172.21.15.181:0/1566268 >> v2:172.21.15.175:3300/0 conn(0x7ff35c142ad0 msgr2=0x7ff35c161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:51.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 --2- 172.21.15.181:0/1566268 >> v2:172.21.15.175:3300/0 conn(0x7ff35c142ad0 0x7ff35c161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 -- 172.21.15.181:0/1566268 >> v1:172.21.15.175:6789/0 conn(0x7ff35c0d1060 legacy=0x7ff35c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 -- 172.21.15.181:0/1566268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c0d1d40 con 0x7ff35c081170 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff35ad9d700 1 --2- 172.21.15.181:0/1566268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7ff354006fd0 tx=0x7ff354006eb0).ready entity=mon.2 client_cookie=53c704860ed93d44 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff359d9b700 1 -- 172.21.15.181:0/1566268 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354009810 con 0x7ff35c081170 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff359d9b700 1 -- 172.21.15.181:0/1566268 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff354009970 con 0x7ff35c081170 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff359d9b700 1 -- 172.21.15.181:0/1566268 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354009cf0 con 0x7ff35c081170 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- 172.21.15.181:0/1566268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 msgr2=0x7ff35c081550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1566268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7ff354006fd0 tx=0x7ff354006eb0).stop 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- 172.21.15.181:0/1566268 shutdown_connections 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1566268 >> v2:172.21.15.175:3300/0 conn(0x7ff35c142ad0 0x7ff35c161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1566268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c081170 0x7ff35c081550 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- 172.21.15.181:0/1566268 >> 172.21.15.181:0/1566268 conn(0x7ff35c0682f0 msgr2=0x7ff35c0d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- 172.21.15.181:0/1566268 shutdown_connections 2020-11-16T22:36:51.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.923+0000 7ff3608f2700 1 -- 172.21.15.181:0/1566268 wait complete. 2020-11-16T22:36:51.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 Processor -- start 2020-11-16T22:36:51.917 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 -- start start 2020-11-16T22:36:51.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff35c081170 0x7ff35c0b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.919 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35c0d1060 0x7ff35c170af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 0x7ff35c170fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0b9990 con 0x7ff35c081170 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c13d590 con 0x7ff35c0d1060 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.927+0000 7ff3608f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0bb870 con 0x7ff35c142ad0 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 0x7ff35c170fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35c0d1060 0x7ff35c170af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.920 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 0x7ff35c170fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46238/0 (socket says 172.21.15.181:46238) 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 -- 172.21.15.181:0/1939503816 learned_addr learned my addr 172.21.15.181:0/1939503816 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35ad9d700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff35c081170 0x7ff35c0b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 -- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35c0d1060 msgr2=0x7ff35c170af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35c0d1060 0x7ff35c170af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 -- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff35c081170 msgr2=0x7ff35c0b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff35c081170 0x7ff35c0b9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.921 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c0ba6b0 con 0x7ff35c142ad0 2020-11-16T22:36:51.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff35b59e700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 0x7ff35c170fe0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7ff34c00ef50 tx=0x7ff34c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.922 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c009690 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00cb40 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c0ba950 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0123f0 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c0bb130 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff35c044f60 con 0x7ff35c142ad0 2020-11-16T22:36:51.923 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff34c022070 con 0x7ff35c142ad0 2020-11-16T22:36:51.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.931+0000 7ff352ffd700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff32c055090 0x7ff32c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff34c06ea70 con 0x7ff35c142ad0 2020-11-16T22:36:51.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff35ad9d700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff32c055090 0x7ff32c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.924 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff352ffd700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7ff32c05c550 0x7ff32c05e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:51.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff32c05efc0 con 0x7ff32c05c550 2020-11-16T22:36:51.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff35a59c700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7ff32c05c550 0x7ff32c05e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:51.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff35ad9d700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff32c055090 0x7ff32c057550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7ff35400b1b0 tx=0x7ff354015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.925 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff35a59c700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7ff32c05c550 0x7ff32c05e970 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:51.926 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== osd.4 v2:172.21.15.175:6800/15329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7ff32c05efc0 con 0x7ff32c05c550 2020-11-16T22:36:51.927 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.935+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34c044cd0 con 0x7ff35c142ad0 2020-11-16T22:36:51.944 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 --> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff35c082560 con 0x7ff32c05c550 2020-11-16T22:36:51.945 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff352ffd700 1 -- 172.21.15.181:0/1939503816 <== osd.4 v2:172.21.15.175:6800/15329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff35c082560 con 0x7ff32c05c550 2020-11-16T22:36:51.945 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:51.945 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:51.945 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:51.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7ff32c05c550 msgr2=0x7ff32c05e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.175:6800/15329,v1:172.21.15.175:6801/15329] conn(0x7ff32c05c550 0x7ff32c05e970 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:51.946 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff32c055090 msgr2=0x7ff32c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7ff32c055090 0x7ff32c057550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7ff35400b1b0 tx=0x7ff354015000).stop 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 msgr2=0x7ff35c170fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 --2- 172.21.15.181:0/1939503816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff35c142ad0 0x7ff35c170fe0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7ff34c00ef50 tx=0x7ff34c0091d0).stop 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff35b59e700 1 -- 172.21.15.181:0/1939503816 reap_dead start 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 shutdown_connections 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 >> 172.21.15.181:0/1939503816 conn(0x7ff35c0682f0 msgr2=0x7ff35c0d5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:51.947 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 shutdown_connections 2020-11-16T22:36:51.948 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:51.955+0000 7ff3608f2700 1 -- 172.21.15.181:0/1939503816 wait complete. 2020-11-16T22:36:51.958 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2020-11-16T22:36:52.084 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 Processor -- start 2020-11-16T22:36:52.085 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 -- start start 2020-11-16T22:36:52.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f96d0145770 0x7f96d0145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.086 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d0081f20 con 0x7f96d0160450 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d0060470 con 0x7f96d0145770 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d6ede700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0060710 con 0x7f96d00d2a30 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d56db700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f96d0160450 0x7f96d00d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52708/0 (socket says 172.21.15.181:52708) 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d56db700 1 -- 172.21.15.181:0/3972874092 learned_addr learned my addr 172.21.15.181:0/3972874092 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:52.087 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 --2- 172.21.15.181:0/3972874092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46248/0 (socket says 172.21.15.181:46248) 2020-11-16T22:36:52.088 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d5edc700 1 --2- 172.21.15.181:0/3972874092 >> v2:172.21.15.175:3300/0 conn(0x7f96d0145770 0x7f96d0145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d4eda700 1 -- 172.21.15.181:0/3972874092 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055919116 0 0) 0x7f96d0081f20 con 0x7f96d0160450 2020-11-16T22:36:52.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 -- 172.21.15.181:0/3972874092 >> v2:172.21.15.175:3300/0 conn(0x7f96d0145770 msgr2=0x7f96d0145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:52.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 --2- 172.21.15.181:0/3972874092 >> v2:172.21.15.175:3300/0 conn(0x7f96d0145770 0x7f96d0145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 -- 172.21.15.181:0/3972874092 >> v1:172.21.15.181:6789/0 conn(0x7f96d0160450 legacy=0x7f96d00d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.089 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.095+0000 7f96d66dd700 1 -- 172.21.15.181:0/3972874092 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00609b0 con 0x7f96d00d2a30 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d66dd700 1 --2- 172.21.15.181:0/3972874092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f96c0009c90 tx=0x7f96c00091d0).ready entity=mon.2 client_cookie=7af743b813228190 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d4eda700 1 -- 172.21.15.181:0/3972874092 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000ba70 con 0x7f96d00d2a30 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d4eda700 1 -- 172.21.15.181:0/3972874092 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c000bbd0 con 0x7f96d00d2a30 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d4eda700 1 -- 172.21.15.181:0/3972874092 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0011400 con 0x7f96d00d2a30 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 -- 172.21.15.181:0/3972874092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 msgr2=0x7f96d01608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 --2- 172.21.15.181:0/3972874092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f96c0009c90 tx=0x7f96c00091d0).stop 2020-11-16T22:36:52.090 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 -- 172.21.15.181:0/3972874092 shutdown_connections 2020-11-16T22:36:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 --2- 172.21.15.181:0/3972874092 >> v2:172.21.15.175:3300/0 conn(0x7f96d0145770 0x7f96d0145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 --2- 172.21.15.181:0/3972874092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d00d2a30 0x7f96d01608b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 -- 172.21.15.181:0/3972874092 >> 172.21.15.181:0/3972874092 conn(0x7f96d00d6760 msgr2=0x7f96d00d6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 -- 172.21.15.181:0/3972874092 shutdown_connections 2020-11-16T22:36:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.099+0000 7f96d6ede700 1 -- 172.21.15.181:0/3972874092 wait complete. 2020-11-16T22:36:52.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 Processor -- start 2020-11-16T22:36:52.092 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- start start 2020-11-16T22:36:52.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96d00d2a30 0x7f96d015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 0x7f96d00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.093 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d016fb10 con 0x7f96d00d2a30 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00d0560 con 0x7f96d0145770 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d016f870 con 0x7f96d0160450 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d56db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 0x7f96d00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46254/0 (socket says 172.21.15.181:46254) 2020-11-16T22:36:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d56db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 0x7f96d00cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38410/0 (socket says 172.21.15.181:38410) 2020-11-16T22:36:52.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 -- 172.21.15.181:0/603695248 learned_addr learned my addr 172.21.15.181:0/603695248 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d5edc700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96d00d2a30 0x7f96d015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 -- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 msgr2=0x7f96d00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 0x7f96d00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 -- 172.21.15.181:0/603695248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96d00d2a30 msgr2=0x7f96d015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96d00d2a30 0x7f96d015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 -- 172.21.15.181:0/603695248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0172290 con 0x7f96d0160450 2020-11-16T22:36:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d66dd700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f96c0009210 tx=0x7f96c0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c001c6c0 con 0x7f96d0160450 2020-11-16T22:36:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c001c820 con 0x7f96d0160450 2020-11-16T22:36:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d0172530 con 0x7f96d0160450 2020-11-16T22:36:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c001cba0 con 0x7f96d0160450 2020-11-16T22:36:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.103+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0172a70 con 0x7f96d0160450 2020-11-16T22:36:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0172c30 con 0x7f96d0160450 2020-11-16T22:36:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f96c0024030 con 0x7f96d0160450 2020-11-16T22:36:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96cdffb700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 0x7f96ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96c006f6b0 con 0x7f96d0160450 2020-11-16T22:36:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96d5edc700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 0x7f96ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.107+0000 7f96d5edc700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 0x7f96ac057520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f96c8006fd0 tx=0x7f96c8006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.111+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f96c0045a40 con 0x7f96d0160450 2020-11-16T22:36:52.257 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.267+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f96d0044f60 con 0x7f96ac055060 2020-11-16T22:36:52.258 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.267+0000 7f96cdffb700 1 -- 172.21.15.181:0/603695248 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f96d0044f60 con 0x7f96ac055060 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 msgr2=0x7f96ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 0x7f96ac057520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f96c8006fd0 tx=0x7f96c8006df0).stop 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 msgr2=0x7f96d00d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f96c0009210 tx=0x7f96c0016000).stop 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 shutdown_connections 2020-11-16T22:36:52.260 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f96ac055060 0x7f96ac057520 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d0160450 0x7f96d00d0070 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96d00d2a30 0x7f96d015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 --2- 172.21.15.181:0/603695248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d0145770 0x7f96d00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 >> 172.21.15.181:0/603695248 conn(0x7f96d00d6760 msgr2=0x7f96d00b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 shutdown_connections 2020-11-16T22:36:52.261 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.271+0000 7f96d6ede700 1 -- 172.21.15.181:0/603695248 wait complete. 2020-11-16T22:36:52.262 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 4 to deep-scrub 2020-11-16T22:36:52.272 INFO:tasks.ceph:Scrubbing osd.5 2020-11-16T22:36:52.272 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:52.398 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.407+0000 7efbfb0b9700 1 Processor -- start 2020-11-16T22:36:52.399 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.407+0000 7efbfb0b9700 1 -- start start 2020-11-16T22:36:52.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efbf413f330 0x7efbf4066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efbf413f710 0x7efbf415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.400 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf40659d0 con 0x7efbf415e4b0 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf40b61c0 con 0x7efbf413f330 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf4065c70 con 0x7efbf413f710 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efbf413f710 0x7efbf415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efbf413f710 0x7efbf415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38416/0 (socket says 172.21.15.181:38416) 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 -- 172.21.15.181:0/3497734224 learned_addr learned my addr 172.21.15.181:0/3497734224 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.401 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf8e55700 1 --2- 172.21.15.181:0/3497734224 >> v2:172.21.15.175:3300/0 conn(0x7efbf413f330 0x7efbf4066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.402 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 -- 172.21.15.181:0/3497734224 >> v2:172.21.15.175:3300/0 conn(0x7efbf413f330 msgr2=0x7efbf4066b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 --2- 172.21.15.181:0/3497734224 >> v2:172.21.15.175:3300/0 conn(0x7efbf413f330 0x7efbf4066b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 -- 172.21.15.181:0/3497734224 >> v1:172.21.15.175:6789/0 conn(0x7efbf415e4b0 legacy=0x7efbf4062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 -- 172.21.15.181:0/3497734224 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4065f10 con 0x7efbf413f710 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf3fff700 1 --2- 172.21.15.181:0/3497734224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f710 0x7efbf415dfc0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7efbf4160020 tx=0x7efbdc006eb0).ready entity=mon.1 client_cookie=d1c80bb101d25bd6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf37fe700 1 -- 172.21.15.181:0/3497734224 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc009810 con 0x7efbf413f710 2020-11-16T22:36:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf37fe700 1 -- 172.21.15.181:0/3497734224 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbdc009970 con 0x7efbf413f710 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbf37fe700 1 -- 172.21.15.181:0/3497734224 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc009cf0 con 0x7efbf413f710 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- 172.21.15.181:0/3497734224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f710 msgr2=0x7efbf415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/3497734224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f710 0x7efbf415dfc0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7efbf4160020 tx=0x7efbdc006eb0).stop 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- 172.21.15.181:0/3497734224 shutdown_connections 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/3497734224 >> v2:172.21.15.175:3300/0 conn(0x7efbf413f330 0x7efbf4066b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/3497734224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f710 0x7efbf415dfc0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- 172.21.15.181:0/3497734224 >> 172.21.15.181:0/3497734224 conn(0x7efbf40d35d0 msgr2=0x7efbf40d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- 172.21.15.181:0/3497734224 shutdown_connections 2020-11-16T22:36:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.411+0000 7efbfb0b9700 1 -- 172.21.15.181:0/3497734224 wait complete. 2020-11-16T22:36:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 Processor -- start 2020-11-16T22:36:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 -- start start 2020-11-16T22:36:52.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 0x7efbf40b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efbf413f710 0x7efbf40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf415e4b0 0x7efbf41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4139a30 con 0x7efbf415e4b0 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf413ecb0 con 0x7efbf413f330 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbfb0b9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4173590 con 0x7efbf413f710 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 0x7efbf40b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 0x7efbf40b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38422/0 (socket says 172.21.15.181:38422) 2020-11-16T22:36:52.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 -- 172.21.15.181:0/24263468 learned_addr learned my addr 172.21.15.181:0/24263468 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf3fff700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efbf413f710 0x7efbf40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.408 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf9656700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf415e4b0 0x7efbf41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 -- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efbf413f710 msgr2=0x7efbf40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efbf413f710 0x7efbf40cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 -- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf415e4b0 msgr2=0x7efbf41596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.415+0000 7efbf8e55700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf415e4b0 0x7efbf41596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf8e55700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4139c20 con 0x7efbf413f330 2020-11-16T22:36:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf8e55700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 0x7efbf40b29b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7efbe400ef50 tx=0x7efbe40091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe400ccc0 con 0x7efbf413f330 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf4139ec0 con 0x7efbf413f330 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe400ce20 con 0x7efbf413f330 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4012600 con 0x7efbf413f330 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf413a400 con 0x7efbf413f330 2020-11-16T22:36:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efbf4069eb0 con 0x7efbf413f330 2020-11-16T22:36:52.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efbe4022070 con 0x7efbf413f330 2020-11-16T22:36:52.411 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd0055060 0x7efbd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf3fff700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd0055060 0x7efbd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.419+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efbe406ebc0 con 0x7efbf413f330 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf3fff700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd0055060 0x7efbd0057520 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7efbf4160020 tx=0x7efbdc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf0ff9700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7efbd005c5a0 0x7efbd005e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbd005f010 con 0x7efbd005c5a0 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe406efc0 con 0x7efbf413f330 2020-11-16T22:36:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf9656700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7efbd005c5a0 0x7efbd005e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf9656700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7efbd005c5a0 0x7efbd005e9c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.423+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== osd.5 v2:172.21.15.175:6823/15351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7efbd005f010 con 0x7efbd005c5a0 2020-11-16T22:36:52.433 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 --> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efbf4159e00 con 0x7efbd005c5a0 2020-11-16T22:36:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbf0ff9700 1 -- 172.21.15.181:0/24263468 <== osd.5 v2:172.21.15.175:6823/15351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efbf4159e00 con 0x7efbd005c5a0 2020-11-16T22:36:52.434 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:52.435 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:52.435 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7efbd005c5a0 msgr2=0x7efbd005e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.175:6823/15351,v1:172.21.15.175:6825/15351] conn(0x7efbd005c5a0 0x7efbd005e9c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd0055060 msgr2=0x7efbd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7efbd0055060 0x7efbd0057520 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7efbf4160020 tx=0x7efbdc015000).stop 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 msgr2=0x7efbf40b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbfb0b9700 1 --2- 172.21.15.181:0/24263468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf413f330 0x7efbf40b29b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7efbe400ef50 tx=0x7efbe40091d0).stop 2020-11-16T22:36:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.443+0000 7efbf9656700 1 -- 172.21.15.181:0/24263468 reap_dead start 2020-11-16T22:36:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.447+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 shutdown_connections 2020-11-16T22:36:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.447+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 >> 172.21.15.181:0/24263468 conn(0x7efbf40d35d0 msgr2=0x7efbf4064c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.447+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 shutdown_connections 2020-11-16T22:36:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.447+0000 7efbfb0b9700 1 -- 172.21.15.181:0/24263468 wait complete. 2020-11-16T22:36:52.453 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2020-11-16T22:36:52.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.591+0000 7fb8d772a700 1 Processor -- start 2020-11-16T22:36:52.583 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.591+0000 7fb8d772a700 1 -- start start 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d772a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8d0160430 0x7fb8d0161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d772a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb8d0160810 0x7fb8d00600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d772a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d0160ca0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d772a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00c8a40 con 0x7fb8d0160810 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d772a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00cf030 con 0x7fb8d0160430 2020-11-16T22:36:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8d0160430 0x7fb8d0161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8d0160430 0x7fb8d0161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38434/0 (socket says 172.21.15.181:38434) 2020-11-16T22:36:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 -- 172.21.15.181:0/3828887935 learned_addr learned my addr 172.21.15.181:0/3828887935 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d5cc7700 1 --2- 172.21.15.181:0/3828887935 >> v2:172.21.15.175:3300/0 conn(0x7fb8d0160810 0x7fb8d00600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 -- 172.21.15.181:0/3828887935 >> v2:172.21.15.175:3300/0 conn(0x7fb8d0160810 msgr2=0x7fb8d00600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 --2- 172.21.15.181:0/3828887935 >> v2:172.21.15.175:3300/0 conn(0x7fb8d0160810 0x7fb8d00600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 -- 172.21.15.181:0/3828887935 >> v1:172.21.15.175:6789/0 conn(0x7fb8d00c85e0 legacy=0x7fb8d0162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 -- 172.21.15.181:0/3828887935 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0161a10 con 0x7fb8d0160430 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8d54c6700 1 --2- 172.21.15.181:0/3828887935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0161460 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb8d00681b0 tx=0x7fb8cc006eb0).ready entity=mon.1 client_cookie=337178056f383a28 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8bffff700 1 -- 172.21.15.181:0/3828887935 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc009810 con 0x7fb8d0160430 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8bffff700 1 -- 172.21.15.181:0/3828887935 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc009970 con 0x7fb8d0160430 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.595+0000 7fb8bffff700 1 -- 172.21.15.181:0/3828887935 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc009cf0 con 0x7fb8d0160430 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- 172.21.15.181:0/3828887935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 msgr2=0x7fb8d0161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 --2- 172.21.15.181:0/3828887935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0161460 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb8d00681b0 tx=0x7fb8cc006eb0).stop 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- 172.21.15.181:0/3828887935 shutdown_connections 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 --2- 172.21.15.181:0/3828887935 >> v2:172.21.15.175:3300/0 conn(0x7fb8d0160810 0x7fb8d00600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 --2- 172.21.15.181:0/3828887935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0161460 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- 172.21.15.181:0/3828887935 >> 172.21.15.181:0/3828887935 conn(0x7fb8d00d3680 msgr2=0x7fb8d00d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- 172.21.15.181:0/3828887935 shutdown_connections 2020-11-16T22:36:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- 172.21.15.181:0/3828887935 wait complete. 2020-11-16T22:36:52.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 Processor -- start 2020-11-16T22:36:52.590 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.599+0000 7fb8d772a700 1 -- start start 2020-11-16T22:36:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 0x7fb8d01420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0170050 con 0x7fb8d0160810 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0157f90 con 0x7fb8d0160430 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0158230 con 0x7fb8d00c85e0 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46280/0 (socket says 172.21.15.181:46280) 2020-11-16T22:36:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 -- 172.21.15.181:0/1089921986 learned_addr learned my addr 172.21.15.181:0/1089921986 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d4cc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d5cc7700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 0x7fb8d01420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 -- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 msgr2=0x7fb8d0141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 -- 172.21.15.181:0/1089921986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 msgr2=0x7fb8d01420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 0x7fb8d01420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 -- 172.21.15.181:0/1089921986 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d01584d0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d5cc7700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 0x7fb8d01420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d54c6700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fb8d00b5840 tx=0x7fb8cc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc011dd0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc01e460 con 0x7fb8d00c85e0 2020-11-16T22:36:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc01e800 con 0x7fb8d00c85e0 2020-11-16T22:36:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d0158770 con 0x7fb8d00c85e0 2020-11-16T22:36:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0158cb0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb8cc025030 con 0x7fb8d00c85e0 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.603+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d0158e70 con 0x7fb8d00c85e0 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.607+0000 7fb8bd7fa700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 0x7fb8a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.607+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb8cc06f4a0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.607+0000 7fb8d4cc5700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 0x7fb8a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.607+0000 7fb8d4cc5700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 0x7fb8a8057520 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb8d00cdff0 tx=0x7fb8c0007b00).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.604 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.615+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb8cc0457b0 con 0x7fb8d00c85e0 2020-11-16T22:36:52.750 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.759+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d00c9940 con 0x7fb8a8055060 2020-11-16T22:36:52.751 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8bd7fa700 1 -- 172.21.15.181:0/1089921986 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb8d00c9940 con 0x7fb8a8055060 2020-11-16T22:36:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 msgr2=0x7fb8a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 0x7fb8a8057520 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb8d00cdff0 tx=0x7fb8c0007b00).stop 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 msgr2=0x7fb8d00c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fb8d00b5840 tx=0x7fb8cc015000).stop 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 shutdown_connections 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb8a8055060 0x7fb8a8057520 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d0160810 0x7fb8d01420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8d00c85e0 0x7fb8d00c93f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 --2- 172.21.15.181:0/1089921986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8d0160430 0x7fb8d0141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 >> 172.21.15.181:0/1089921986 conn(0x7fb8d00d3680 msgr2=0x7fb8d00d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 shutdown_connections 2020-11-16T22:36:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.763+0000 7fb8d772a700 1 -- 172.21.15.181:0/1089921986 wait complete. 2020-11-16T22:36:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 5 to deep-scrub 2020-11-16T22:36:52.771 INFO:tasks.ceph:Scrubbing osd.6 2020-11-16T22:36:52.771 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:52.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.911+0000 7f920bfff700 1 Processor -- start 2020-11-16T22:36:52.901 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.911+0000 7f920bfff700 1 -- start start 2020-11-16T22:36:52.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f920c081170 0x7f920c081550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.903 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f920c142ad0 0x7f920c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c081a40 con 0x7f920c0d1060 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c142f40 con 0x7f920c081170 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c0d1800 con 0x7f920c142ad0 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f920c142ad0 0x7f920c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f920c142ad0 0x7f920c161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38450/0 (socket says 172.21.15.181:38450) 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 -- 172.21.15.181:0/1715522154 learned_addr learned my addr 172.21.15.181:0/1715522154 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.904 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920affd700 1 --2- 172.21.15.181:0/1715522154 >> v2:172.21.15.175:3300/0 conn(0x7f920c081170 0x7f920c081550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 -- 172.21.15.181:0/1715522154 >> v2:172.21.15.175:3300/0 conn(0x7f920c081170 msgr2=0x7f920c081550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 --2- 172.21.15.181:0/1715522154 >> v2:172.21.15.175:3300/0 conn(0x7f920c081170 0x7f920c081550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 -- 172.21.15.181:0/1715522154 >> v1:172.21.15.175:6789/0 conn(0x7f920c0d1060 legacy=0x7f920c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 -- 172.21.15.181:0/1715522154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0d1aa0 con 0x7f920c142ad0 2020-11-16T22:36:52.906 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920b7fe700 1 --2- 172.21.15.181:0/1715522154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c142ad0 0x7f920c161630 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f9204009d50 tx=0x7f9204009290).ready entity=mon.1 client_cookie=7869afba637aff2b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f9209ffb700 1 -- 172.21.15.181:0/1715522154 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400bc40 con 0x7f920c142ad0 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f9209ffb700 1 -- 172.21.15.181:0/1715522154 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920400bda0 con 0x7f920c142ad0 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f9209ffb700 1 -- 172.21.15.181:0/1715522154 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204011610 con 0x7f920c142ad0 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- 172.21.15.181:0/1715522154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c142ad0 msgr2=0x7f920c161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 --2- 172.21.15.181:0/1715522154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c142ad0 0x7f920c161630 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f9204009d50 tx=0x7f9204009290).stop 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- 172.21.15.181:0/1715522154 shutdown_connections 2020-11-16T22:36:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 --2- 172.21.15.181:0/1715522154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c142ad0 0x7f920c161630 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 --2- 172.21.15.181:0/1715522154 >> v2:172.21.15.175:3300/0 conn(0x7f920c081170 0x7f920c081550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- 172.21.15.181:0/1715522154 >> 172.21.15.181:0/1715522154 conn(0x7f920c0682f0 msgr2=0x7f920c0d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- 172.21.15.181:0/1715522154 shutdown_connections 2020-11-16T22:36:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.915+0000 7f920bfff700 1 -- 172.21.15.181:0/1715522154 wait complete. 2020-11-16T22:36:52.909 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 Processor -- start 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- start start 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 0x7f920c0bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f920c0d1060 0x7f920c0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f920c142ad0 0x7f920c160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0bb4a0 con 0x7f920c0d1060 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0bb740 con 0x7f920c081170 2020-11-16T22:36:52.910 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0bb9e0 con 0x7f920c142ad0 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 0x7f920c0bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f920c142ad0 0x7f920c160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 0x7f920c0bc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38452/0 (socket says 172.21.15.181:38452) 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 -- 172.21.15.181:0/1324246133 learned_addr learned my addr 172.21.15.181:0/1324246133 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920a7fc700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f920c0d1060 0x7f920c0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 -- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f920c142ad0 msgr2=0x7f920c160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.911 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f920c142ad0 0x7f920c160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 -- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f920c0d1060 msgr2=0x7f920c0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f920c0d1060 0x7f920c0cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0bbc80 con 0x7f920c081170 2020-11-16T22:36:52.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920a7fc700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f920c0d1060 0x7f920c0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:52.912 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920affd700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 0x7f920c0bc3c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f920000db60 tx=0x7f9200008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.913 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200009790 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000098f0 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c0bbf20 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200009c70 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c0cc4c0 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.919+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f920c044f60 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9200013480 con 0x7f920c081170 2020-11-16T22:36:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91e8055060 0x7f91e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f920006e290 con 0x7f920c081170 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7f91e805c5a0 0x7f91e805e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f920a7fc700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91e8055060 0x7f91e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91e805f010 con 0x7f91e805c5a0 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92000707f0 con 0x7f920c081170 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f920b7fe700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7f91e805c5a0 0x7f91e805e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:52.915 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f920a7fc700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91e8055060 0x7f91e8057520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f91fc0010d0 tx=0x7f91fc009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.923+0000 7f920b7fe700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7f91e805c5a0 0x7f91e805e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:52.916 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.927+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== osd.6 v2:172.21.15.175:6808/15330 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f91e805f010 con 0x7f91e805c5a0 2020-11-16T22:36:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.939+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 --> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f920c082560 con 0x7f91e805c5a0 2020-11-16T22:36:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.939+0000 7f91f2ffd700 1 -- 172.21.15.181:0/1324246133 <== osd.6 v2:172.21.15.175:6808/15330 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f920c082560 con 0x7f91e805c5a0 2020-11-16T22:36:52.931 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:52.931 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:52.931 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7f91e805c5a0 msgr2=0x7f91e805e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.175:6808/15330,v1:172.21.15.175:6809/15330] conn(0x7f91e805c5a0 0x7f91e805e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91e8055060 msgr2=0x7f91e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f91e8055060 0x7f91e8057520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f91fc0010d0 tx=0x7f91fc009040).stop 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 msgr2=0x7f920c0bc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 --2- 172.21.15.181:0/1324246133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c081170 0x7f920c0bc3c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f920000db60 tx=0x7f9200008040).stop 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920b7fe700 1 -- 172.21.15.181:0/1324246133 reap_dead start 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 shutdown_connections 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 >> 172.21.15.181:0/1324246133 conn(0x7f920c0682f0 msgr2=0x7f920c164eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 shutdown_connections 2020-11-16T22:36:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:52.943+0000 7f920bfff700 1 -- 172.21.15.181:0/1324246133 wait complete. 2020-11-16T22:36:52.945 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2020-11-16T22:36:53.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.075+0000 7fe0d6b9e700 1 Processor -- start 2020-11-16T22:36:53.067 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- start start 2020-11-16T22:36:53.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d01621b0 con 0x7fe0d00633d0 2020-11-16T22:36:53.068 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d0064970 con 0x7fe0d00637b0 2020-11-16T22:36:53.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0065930 con 0x7fe0d0061810 2020-11-16T22:36:53.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46304/0 (socket says 172.21.15.181:46304) 2020-11-16T22:36:53.069 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 -- 172.21.15.181:0/1877515242 learned_addr learned my addr 172.21.15.181:0/1877515242 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 -- 172.21.15.181:0/1877515242 >> v1:172.21.15.181:6789/0 conn(0x7fe0d00637b0 legacy=0x7fe0d0062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 -- 172.21.15.181:0/1877515242 >> v1:172.21.15.175:6789/0 conn(0x7fe0d00633d0 legacy=0x7fe0d0061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 -- 172.21.15.181:0/1877515242 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d0064c10 con 0x7fe0d0061810 2020-11-16T22:36:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cffff700 1 --2- 172.21.15.181:0/1877515242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fe0c400ba20 tx=0x7fe0c4006eb0).ready entity=mon.2 client_cookie=aa7a08dab9d11ab7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cf7fe700 1 -- 172.21.15.181:0/1877515242 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40098d0 con 0x7fe0d0061810 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cf7fe700 1 -- 172.21.15.181:0/1877515242 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c4009a30 con 0x7fe0d0061810 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0cf7fe700 1 -- 172.21.15.181:0/1877515242 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4009db0 con 0x7fe0d0061810 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/1877515242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 msgr2=0x7fe0d0161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/1877515242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fe0c400ba20 tx=0x7fe0c4006eb0).stop 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/1877515242 shutdown_connections 2020-11-16T22:36:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/1877515242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d0061810 0x7fe0d0161cc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/1877515242 >> 172.21.15.181:0/1877515242 conn(0x7fe0d00cf890 msgr2=0x7fe0d00cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/1877515242 shutdown_connections 2020-11-16T22:36:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.079+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/1877515242 wait complete. 2020-11-16T22:36:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 Processor -- start 2020-11-16T22:36:53.073 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- start start 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d00633d0 0x7fe0d0158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d0170160 0x7fe0d01705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0170ad0 con 0x7fe0d00633d0 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d016efa0 con 0x7fe0d0170160 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d016f240 con 0x7fe0d00637b0 2020-11-16T22:36:53.074 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d513b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d0170160 0x7fe0d01705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46312/0 (socket says 172.21.15.181:46312) 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 -- 172.21.15.181:0/50285366 learned_addr learned my addr 172.21.15.181:0/50285366 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d493a700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d00633d0 0x7fe0d0158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 -- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d0170160 msgr2=0x7fe0d01705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d0170160 0x7fe0d01705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.075 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 -- 172.21.15.181:0/50285366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d00633d0 msgr2=0x7fe0d0158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d00633d0 0x7fe0d0158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 -- 172.21.15.181:0/50285366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d016f4e0 con 0x7fe0d00637b0 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0cffff700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fe0d00675a0 tx=0x7fe0c4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4011e00 con 0x7fe0d00637b0 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c401e460 con 0x7fe0d00637b0 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c401e800 con 0x7fe0d00637b0 2020-11-16T22:36:53.076 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d016f780 con 0x7fe0d00637b0 2020-11-16T22:36:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d016fcc0 con 0x7fe0d00637b0 2020-11-16T22:36:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.083+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d0069eb0 con 0x7fe0d00637b0 2020-11-16T22:36:53.077 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.087+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe0c4025030 con 0x7fe0d00637b0 2020-11-16T22:36:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.087+0000 7fe0ccff9700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 0x7fe0a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.087+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe0c406f500 con 0x7fe0d00637b0 2020-11-16T22:36:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.087+0000 7fe0d493a700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 0x7fe0a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.078 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.087+0000 7fe0d493a700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 0x7fe0a8057520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fe0c0007fa0 tx=0x7fe0c0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.080 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.091+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe0c4045810 con 0x7fe0d00637b0 2020-11-16T22:36:53.230 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.239+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0d005d4e0 con 0x7fe0a8055060 2020-11-16T22:36:53.231 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0ccff9700 1 -- 172.21.15.181:0/50285366 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe0d005d4e0 con 0x7fe0a8055060 2020-11-16T22:36:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 msgr2=0x7fe0a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 0x7fe0a8057520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fe0c0007fa0 tx=0x7fe0c0009040).stop 2020-11-16T22:36:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 msgr2=0x7fe0d0158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.233 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fe0d00675a0 tx=0x7fe0c4015000).stop 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 shutdown_connections 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fe0a8055060 0x7fe0a8057520 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d0170160 0x7fe0d01705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d00633d0 0x7fe0d0158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 --2- 172.21.15.181:0/50285366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00637b0 0x7fe0d0158e80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.234 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 >> 172.21.15.181:0/50285366 conn(0x7fe0d00cf890 msgr2=0x7fe0d0060aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 shutdown_connections 2020-11-16T22:36:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.243+0000 7fe0d6b9e700 1 -- 172.21.15.181:0/50285366 wait complete. 2020-11-16T22:36:53.235 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 6 to deep-scrub 2020-11-16T22:36:53.250 INFO:tasks.ceph:Scrubbing osd.7 2020-11-16T22:36:53.250 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:53.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.383+0000 7faa46468700 1 Processor -- start 2020-11-16T22:36:53.374 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.383+0000 7faa46468700 1 -- start start 2020-11-16T22:36:53.375 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa40145860 0x7faa40145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa40163180 con 0x7faa400c84d0 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400663e0 con 0x7faa40145860 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa40162c40 con 0x7faa40065b10 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa45466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa40145860 0x7faa40145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.376 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa40145860 0x7faa40145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38478/0 (socket says 172.21.15.181:38478) 2020-11-16T22:36:53.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa45466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46318/0 (socket says 172.21.15.181:46318) 2020-11-16T22:36:53.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa45c67700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faa400c84d0 0x7faa400c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52784/0 (socket says 172.21.15.181:52784) 2020-11-16T22:36:53.377 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 -- 172.21.15.181:0/2604875102 learned_addr learned my addr 172.21.15.181:0/2604875102 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 -- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 msgr2=0x7faa40065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:53.378 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 -- 172.21.15.181:0/2604875102 >> v1:172.21.15.181:6789/0 conn(0x7faa400c84d0 legacy=0x7faa400c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa37fff700 1 -- 172.21.15.181:0/2604875102 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957893826 0 0) 0x7faa40163180 con 0x7faa400c84d0 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 -- 172.21.15.181:0/2604875102 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40162ee0 con 0x7faa40145860 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa45466700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa44c65700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa40145cc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7faa30006fd0 tx=0x7faa30006eb0).ready entity=mon.1 client_cookie=1b9afabe491c5489 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa37fff700 1 -- 172.21.15.181:0/2604875102 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300098a0 con 0x7faa40145860 2020-11-16T22:36:53.379 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa37fff700 1 -- 172.21.15.181:0/2604875102 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30009a00 con 0x7faa40145860 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa37fff700 1 -- 172.21.15.181:0/2604875102 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30009d80 con 0x7faa40145860 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 msgr2=0x7faa40145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa40145cc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7faa30006fd0 tx=0x7faa30006eb0).stop 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- 172.21.15.181:0/2604875102 shutdown_connections 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa40065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 --2- 172.21.15.181:0/2604875102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa40145cc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.380 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- 172.21.15.181:0/2604875102 >> 172.21.15.181:0/2604875102 conn(0x7faa400d35c0 msgr2=0x7faa400cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- 172.21.15.181:0/2604875102 shutdown_connections 2020-11-16T22:36:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.387+0000 7faa46468700 1 -- 172.21.15.181:0/2604875102 wait complete. 2020-11-16T22:36:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 Processor -- start 2020-11-16T22:36:53.381 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 -- start start 2020-11-16T22:36:53.382 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa4013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa400c84d0 0x7faa400b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa400b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40164a70 con 0x7faa400c84d0 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40164d10 con 0x7faa40145860 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa46468700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa40164fb0 con 0x7faa40065b10 2020-11-16T22:36:53.383 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa4013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa400b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45466700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa4013fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46324/0 (socket says 172.21.15.181:46324) 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45466700 1 -- 172.21.15.181:0/2509284864 learned_addr learned my addr 172.21.15.181:0/2509284864 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa400b65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38486/0 (socket says 172.21.15.181:38486) 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa44c65700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa400c84d0 0x7faa400b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 -- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 msgr2=0x7faa4013fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.384 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa40065b10 0x7faa4013fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 -- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa400c84d0 msgr2=0x7faa400b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa400c84d0 0x7faa400b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40165250 con 0x7faa40145860 2020-11-16T22:36:53.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa44c65700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa400c84d0 0x7faa400b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:36:53.385 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.391+0000 7faa45c67700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa400b65d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7faa3800c860 tx=0x7faa38007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38005ac0 con 0x7faa40145860 2020-11-16T22:36:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38005c20 con 0x7faa40145860 2020-11-16T22:36:53.386 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa401654f0 con 0x7faa40145860 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38007110 con 0x7faa40145860 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa40140210 con 0x7faa40145860 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa40044f60 con 0x7faa40145860 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faa38010030 con 0x7faa40145860 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa18055090 0x7faa18057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.387 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa45466700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa18055090 0x7faa18057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa3806dba0 con 0x7faa40145860 2020-11-16T22:36:53.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7faa1805c5d0 0x7faa1805e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa1805f040 con 0x7faa1805c5d0 2020-11-16T22:36:53.388 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa38070210 con 0x7faa40145860 2020-11-16T22:36:53.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.395+0000 7faa45466700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa18055090 0x7faa18057550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7faa3c00ba80 tx=0x7faa3c006df0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.399+0000 7faa44c65700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7faa1805c5d0 0x7faa1805e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.399+0000 7faa44c65700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7faa1805c5d0 0x7faa1805e9f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.389 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.399+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== osd.7 v2:172.21.15.175:6816/15354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7faa1805f040 con 0x7faa1805c5d0 2020-11-16T22:36:53.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.415+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 --> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faa40173620 con 0x7faa1805c5d0 2020-11-16T22:36:53.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa357fa700 1 -- 172.21.15.181:0/2509284864 <== osd.7 v2:172.21.15.175:6816/15354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faa40173620 con 0x7faa1805c5d0 2020-11-16T22:36:53.408 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2020-11-16T22:36:53.408 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stdout:} 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7faa1805c5d0 msgr2=0x7faa1805e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.175:6816/15354,v1:172.21.15.175:6817/15354] conn(0x7faa1805c5d0 0x7faa1805e9f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa18055090 msgr2=0x7faa18057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7faa18055090 0x7faa18057550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7faa3c00ba80 tx=0x7faa3c006df0).stop 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 msgr2=0x7faa400b65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 --2- 172.21.15.181:0/2509284864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa40145860 0x7faa400b65d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7faa3800c860 tx=0x7faa38007c40).stop 2020-11-16T22:36:53.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa45c67700 1 -- 172.21.15.181:0/2509284864 reap_dead start 2020-11-16T22:36:53.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 shutdown_connections 2020-11-16T22:36:53.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 >> 172.21.15.181:0/2509284864 conn(0x7faa400d35c0 msgr2=0x7faa400b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 shutdown_connections 2020-11-16T22:36:53.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.419+0000 7faa46468700 1 -- 172.21.15.181:0/2509284864 wait complete. 2020-11-16T22:36:53.421 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2020-11-16T22:36:53.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 Processor -- start 2020-11-16T22:36:53.552 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 -- start start 2020-11-16T22:36:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8a4145a30 0x7fb8a4145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a40816e0 0x7fb8a40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a4061b50 con 0x7fb8a41492c0 2020-11-16T22:36:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40618b0 con 0x7fb8a4145a30 2020-11-16T22:36:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8aa4c3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40cb930 con 0x7fb8a40816e0 2020-11-16T22:36:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8a4145a30 0x7fb8a4145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8a4145a30 0x7fb8a4145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38492/0 (socket says 172.21.15.181:38492) 2020-11-16T22:36:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 -- 172.21.15.181:0/2984347831 learned_addr learned my addr 172.21.15.181:0/2984347831 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a9cc2700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a40816e0 0x7fb8a40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 -- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a40816e0 msgr2=0x7fb8a40cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:36:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a40816e0 0x7fb8a40cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 -- 172.21.15.181:0/2984347831 >> v1:172.21.15.181:6789/0 conn(0x7fb8a41492c0 legacy=0x7fb8a4080fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 -- 172.21.15.181:0/2984347831 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40cbbd0 con 0x7fb8a4145a30 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb8a94c1700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a4145e10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb8a0009d50 tx=0x7fb8a0009290).ready entity=mon.1 client_cookie=513d8abcf9ca2b90 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb89bfff700 1 -- 172.21.15.181:0/2984347831 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000bbc0 con 0x7fb8a4145a30 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb89bfff700 1 -- 172.21.15.181:0/2984347831 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000bd20 con 0x7fb8a4145a30 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.563+0000 7fb89bfff700 1 -- 172.21.15.181:0/2984347831 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0011610 con 0x7fb8a4145a30 2020-11-16T22:36:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 msgr2=0x7fb8a4145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a4145e10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb8a0009d50 tx=0x7fb8a0009290).stop 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/2984347831 shutdown_connections 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a4145e10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/2984347831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a40816e0 0x7fb8a40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/2984347831 >> 172.21.15.181:0/2984347831 conn(0x7fb8a40d68c0 msgr2=0x7fb8a40d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/2984347831 shutdown_connections 2020-11-16T22:36:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/2984347831 wait complete. 2020-11-16T22:36:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 Processor -- start 2020-11-16T22:36:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- start start 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a40816e0 0x7fb8a413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 0x7fb8a40cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40d4b70 con 0x7fb8a40816e0 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40d4e10 con 0x7fb8a4145a30 2020-11-16T22:36:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8aa4c3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40d50b0 con 0x7fb8a41492c0 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8a8cc0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8a8cc0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38500/0 (socket says 172.21.15.181:38500) 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8a8cc0700 1 -- 172.21.15.181:0/253376452 learned_addr learned my addr 172.21.15.181:0/253376452 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.567+0000 7fb8a9cc2700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 0x7fb8a40cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a94c1700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a40816e0 0x7fb8a413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 -- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 msgr2=0x7fb8a40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.561 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 0x7fb8a40cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 -- 172.21.15.181:0/253376452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a40816e0 msgr2=0x7fb8a413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a40816e0 0x7fb8a413cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 -- 172.21.15.181:0/253376452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40d5350 con 0x7fb8a4145a30 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a9cc2700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 0x7fb8a40cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a8cc0700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fb89400c6e0 tx=0x7fb894007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894007020 con 0x7fb8a4145a30 2020-11-16T22:36:53.562 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894007180 con 0x7fb8a4145a30 2020-11-16T22:36:53.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940064f0 con 0x7fb8a4145a30 2020-11-16T22:36:53.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a40d55f0 con 0x7fb8a4145a30 2020-11-16T22:36:53.563 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a40d5b30 con 0x7fb8a4145a30 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a4044f60 con 0x7fb8a4145a30 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb894010070 con 0x7fb8a4145a30 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 0x7fb87c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb89406d050 con 0x7fb8a4145a30 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.571+0000 7fb8a94c1700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 0x7fb87c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.564 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.575+0000 7fb8a94c1700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 0x7fb87c057520 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb8a0009fd0 tx=0x7fb8a0016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.579+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb8940432b0 con 0x7fb8a4145a30 2020-11-16T22:36:53.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a4068e30 con 0x7fb87c055060 2020-11-16T22:36:53.723 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8997fa700 1 -- 172.21.15.181:0/253376452 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb8a4068e30 con 0x7fb87c055060 2020-11-16T22:36:53.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 msgr2=0x7fb87c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 0x7fb87c057520 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb8a0009fd0 tx=0x7fb8a0016000).stop 2020-11-16T22:36:53.725 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 msgr2=0x7fb8a413d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fb89400c6e0 tx=0x7fb894007c50).stop 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 shutdown_connections 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fb87c055060 0x7fb87c057520 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8a41492c0 0x7fb8a40cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a40816e0 0x7fb8a413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 --2- 172.21.15.181:0/253376452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a4145a30 0x7fb8a413d080 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.726 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 >> 172.21.15.181:0/253376452 conn(0x7fb8a40d68c0 msgr2=0x7fb8a4165ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 shutdown_connections 2020-11-16T22:36:53.727 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.735+0000 7fb8aa4c3700 1 -- 172.21.15.181:0/253376452 wait complete. 2020-11-16T22:36:53.727 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 7 to deep-scrub 2020-11-16T22:36:53.737 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.863+0000 7f7928d25700 1 Processor -- start 2020-11-16T22:36:53.854 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.863+0000 7f7928d25700 1 -- start start 2020-11-16T22:36:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 0x7f7924061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.855 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7924061b70 con 0x7f792415fa10 2020-11-16T22:36:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792415cda0 con 0x7f79240ce750 2020-11-16T22:36:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792415d040 con 0x7f79240612a0 2020-11-16T22:36:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7921d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f79240ce750 0x7f79240cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52810/0 (socket says 172.21.15.181:52810) 2020-11-16T22:36:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7921d9b700 1 -- 172.21.15.181:0/1325650625 learned_addr learned my addr 172.21.15.181:0/1325650625 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:36:53.856 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792259c700 1 --2- 172.21.15.181:0/1325650625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 0x7f7924061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225916429 0 0) 0x7f792415cda0 con 0x7f79240ce750 2020-11-16T22:36:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7908003240 con 0x7f79240ce750 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 <== mon.0 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49212167 0 0) 0x7f7924061b70 con 0x7f792415fa10 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792259c700 1 -- 172.21.15.181:0/1325650625 >> v1:172.21.15.181:6789/0 conn(0x7f79240ce750 legacy=0x7f79240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792259c700 1 -- 172.21.15.181:0/1325650625 >> v1:172.21.15.175:6789/0 conn(0x7f792415fa10 legacy=0x7f792415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792259c700 1 -- 172.21.15.181:0/1325650625 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792415d2e0 con 0x7f79240612a0 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792259c700 1 --2- 172.21.15.181:0/1325650625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 0x7f7924061680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f791400ba00 tx=0x7f7914006df0).ready entity=mon.2 client_cookie=859761d794d5e35e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914009770 con 0x7f79240612a0 2020-11-16T22:36:53.858 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79140098d0 con 0x7f79240612a0 2020-11-16T22:36:53.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f792159a700 1 -- 172.21.15.181:0/1325650625 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914009c50 con 0x7f79240612a0 2020-11-16T22:36:53.859 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- 172.21.15.181:0/1325650625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 msgr2=0x7f7924061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 --2- 172.21.15.181:0/1325650625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 0x7f7924061680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f791400ba00 tx=0x7f7914006df0).stop 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- 172.21.15.181:0/1325650625 shutdown_connections 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 --2- 172.21.15.181:0/1325650625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240612a0 0x7f7924061680 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- 172.21.15.181:0/1325650625 >> 172.21.15.181:0/1325650625 conn(0x7f79240cf890 msgr2=0x7f79240cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- 172.21.15.181:0/1325650625 shutdown_connections 2020-11-16T22:36:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.867+0000 7f7928d25700 1 -- 172.21.15.181:0/1325650625 wait complete. 2020-11-16T22:36:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 Processor -- start 2020-11-16T22:36:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 -- start start 2020-11-16T22:36:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79240612a0 0x7f792416f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f792415fa10 0x7f79241654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924170230 con 0x7f79240612a0 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792416fcf0 con 0x7f79240ce750 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7928d25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792416ff90 con 0x7f792415fa10 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38514/0 (socket says 172.21.15.181:38514) 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7922d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f792415fa10 0x7f79241654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 -- 172.21.15.181:0/2335474880 learned_addr learned my addr 172.21.15.181:0/2335474880 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:36:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f792259c700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79240612a0 0x7f792416f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 -- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f792415fa10 msgr2=0x7f79241654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f792415fa10 0x7f79241654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 -- 172.21.15.181:0/2335474880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79240612a0 msgr2=0x7f792416f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79240612a0 0x7f792416f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.871+0000 7f7921d9b700 1 -- 172.21.15.181:0/2335474880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240b6700 con 0x7f79240ce750 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f7921d9b700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f791800c6e0 tx=0x7f7918007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180072e0 con 0x7f79240ce750 2020-11-16T22:36:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918007440 con 0x7f79240ce750 2020-11-16T22:36:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79240b69a0 con 0x7f79240ce750 2020-11-16T22:36:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918005820 con 0x7f79240ce750 2020-11-16T22:36:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792416f7d0 con 0x7f79240ce750 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7924069eb0 con 0x7f79240ce750 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7918010070 con 0x7f79240ce750 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 0x7f78fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f792259c700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 0x7f78fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f791806d410 con 0x7f79240ce750 2020-11-16T22:36:53.867 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.875+0000 7f792259c700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 0x7f78fc057520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f791400b150 tx=0x7f7914015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:36:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:53.883+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7918043560 con 0x7f79240ce750 2020-11-16T22:36:54.022 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.031+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7924066280 con 0x7f78fc055060 2020-11-16T22:36:54.029 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.039+0000 7f79127fc700 1 -- 172.21.15.181:0/2335474880 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+104455 (secure 0 0 0) 0x7f7924066280 con 0x7f78fc055060 2020-11-16T22:36:54.031 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:36:54.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 msgr2=0x7f78fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:54.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 0x7f78fc057520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f791400b150 tx=0x7f7914015000).stop 2020-11-16T22:36:54.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 msgr2=0x7f7924164ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:36:54.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f791800c6e0 tx=0x7f7918007a00).stop 2020-11-16T22:36:54.032 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 shutdown_connections 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f78fc055060 0x7f78fc057520 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f792415fa10 0x7f79241654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79240612a0 0x7f792416f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 --2- 172.21.15.181:0/2335474880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79240ce750 0x7f7924164ff0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 >> 172.21.15.181:0/2335474880 conn(0x7f79240cf890 msgr2=0x7f79240d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:36:54.033 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 shutdown_connections 2020-11-16T22:36:54.034 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:36:54.043+0000 7f7928d25700 1 -- 172.21.15.181:0/2335474880 wait complete. 2020-11-16T22:36:54.034 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:36:54.045 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2020-11-16T22:36:52.863036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117916,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":259,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":63096,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":51968,"kb_avail":754911624,"statfs":{"total":773094113280,"available":773029502976,"internally_reserved":0,"allocated":11329536,"data_stored":5273368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53215232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":157,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003386"},"pg_stats":[{"pgid":"16.7","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.595364+0000","last_change":"2020-11-16T22:35:22.593356+0000","last_active":"2020-11-16T22:35:22.595364+0000","last_peered":"2020-11-16T22:35:22.595364+0000","last_clean":"2020-11-16T22:35:22.595364+0000","last_became_active":"2020-11-16T22:35:22.592949+0000","last_became_peered":"2020-11-16T22:35:22.592949+0000","last_unstale":"2020-11-16T22:35:22.595364+0000","last_undegraded":"2020-11-16T22:35:22.595364+0000","last_fullsized":"2020-11-16T22:35:22.595364+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.593947+0000","last_change":"2020-11-16T22:35:22.590884+0000","last_active":"2020-11-16T22:35:22.593947+0000","last_peered":"2020-11-16T22:35:22.593947+0000","last_clean":"2020-11-16T22:35:22.593947+0000","last_became_active":"2020-11-16T22:35:22.590435+0000","last_became_peered":"2020-11-16T22:35:22.590435+0000","last_unstale":"2020-11-16T22:35:22.593947+0000","last_undegraded":"2020-11-16T22:35:22.593947+0000","last_fullsized":"2020-11-16T22:35:22.593947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.590491+0000","last_change":"2020-11-16T22:35:22.587484+0000","last_active":"2020-11-16T22:35:22.590491+0000","last_peered":"2020-11-16T22:35:22.590491+0000","last_clean":"2020-11-16T22:35:22.590491+0000","last_became_active":"2020-11-16T22:35:22.586786+0000","last_became_peered":"2020-11-16T22:35:22.586786+0000","last_unstale":"2020-11-16T22:35:22.590491+0000","last_undegraded":"2020-11-16T22:35:22.590491+0000","last_fullsized":"2020-11-16T22:35:22.590491+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.591364+0000","last_change":"2020-11-16T22:35:22.588060+0000","last_active":"2020-11-16T22:35:22.591364+0000","last_peered":"2020-11-16T22:35:22.591364+0000","last_clean":"2020-11-16T22:35:22.591364+0000","last_became_active":"2020-11-16T22:35:22.586336+0000","last_became_peered":"2020-11-16T22:35:22.586336+0000","last_unstale":"2020-11-16T22:35:22.591364+0000","last_undegraded":"2020-11-16T22:35:22.591364+0000","last_fullsized":"2020-11-16T22:35:22.591364+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.607868+0000","last_change":"2020-11-16T22:35:22.603425+0000","last_active":"2020-11-16T22:35:22.607868+0000","last_peered":"2020-11-16T22:35:22.607868+0000","last_clean":"2020-11-16T22:35:22.607868+0000","last_became_active":"2020-11-16T22:35:22.602895+0000","last_became_peered":"2020-11-16T22:35:22.602895+0000","last_unstale":"2020-11-16T22:35:22.607868+0000","last_undegraded":"2020-11-16T22:35:22.607868+0000","last_fullsized":"2020-11-16T22:35:22.607868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.599330+0000","last_change":"2020-11-16T22:35:22.593614+0000","last_active":"2020-11-16T22:35:22.599330+0000","last_peered":"2020-11-16T22:35:22.599330+0000","last_clean":"2020-11-16T22:35:22.599330+0000","last_became_active":"2020-11-16T22:35:22.593172+0000","last_became_peered":"2020-11-16T22:35:22.593172+0000","last_unstale":"2020-11-16T22:35:22.599330+0000","last_undegraded":"2020-11-16T22:35:22.599330+0000","last_fullsized":"2020-11-16T22:35:22.599330+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.589327+0000","last_change":"2020-11-16T22:35:22.588442+0000","last_active":"2020-11-16T22:35:22.589327+0000","last_peered":"2020-11-16T22:35:22.589327+0000","last_clean":"2020-11-16T22:35:22.589327+0000","last_became_active":"2020-11-16T22:35:22.588015+0000","last_became_peered":"2020-11-16T22:35:22.588015+0000","last_unstale":"2020-11-16T22:35:22.589327+0000","last_undegraded":"2020-11-16T22:35:22.589327+0000","last_fullsized":"2020-11-16T22:35:22.589327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.602537+0000","last_change":"2020-11-16T22:35:22.601873+0000","last_active":"2020-11-16T22:35:22.602537+0000","last_peered":"2020-11-16T22:35:22.602537+0000","last_clean":"2020-11-16T22:35:22.602537+0000","last_became_active":"2020-11-16T22:35:22.601487+0000","last_became_peered":"2020-11-16T22:35:22.601487+0000","last_unstale":"2020-11-16T22:35:22.602537+0000","last_undegraded":"2020-11-16T22:35:22.602537+0000","last_fullsized":"2020-11-16T22:35:22.602537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"111'29","reported_seq":"40","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.153498+0000","last_change":"2020-11-16T22:35:22.192135+0000","last_active":"2020-11-16T22:36:38.153498+0000","last_peered":"2020-11-16T22:36:38.153498+0000","last_clean":"2020-11-16T22:36:38.153498+0000","last_became_active":"2020-11-16T22:35:22.191742+0000","last_became_peered":"2020-11-16T22:35:22.191742+0000","last_unstale":"2020-11-16T22:36:38.153498+0000","last_undegraded":"2020-11-16T22:36:38.153498+0000","last_fullsized":"2020-11-16T22:36:38.153498+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.6","version":"111'14","reported_seq":"25","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:01.326219+0000","last_change":"2020-11-16T22:35:22.196234+0000","last_active":"2020-11-16T22:36:01.326219+0000","last_peered":"2020-11-16T22:36:01.326219+0000","last_clean":"2020-11-16T22:36:01.326219+0000","last_became_active":"2020-11-16T22:35:22.195870+0000","last_became_peered":"2020-11-16T22:35:22.195870+0000","last_unstale":"2020-11-16T22:36:01.326219+0000","last_undegraded":"2020-11-16T22:36:01.326219+0000","last_fullsized":"2020-11-16T22:36:01.326219+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"111'19","reported_seq":"31","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:12.379446+0000","last_change":"2020-11-16T22:35:21.258469+0000","last_active":"2020-11-16T22:36:12.379446+0000","last_peered":"2020-11-16T22:36:12.379446+0000","last_clean":"2020-11-16T22:36:12.379446+0000","last_became_active":"2020-11-16T22:35:21.257651+0000","last_became_peered":"2020-11-16T22:35:21.257651+0000","last_unstale":"2020-11-16T22:36:12.379446+0000","last_undegraded":"2020-11-16T22:36:12.379446+0000","last_fullsized":"2020-11-16T22:36:12.379446+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"111'55","reported_seq":"68","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.154153+0000","last_change":"2020-11-16T22:35:21.130500+0000","last_active":"2020-11-16T22:36:38.154153+0000","last_peered":"2020-11-16T22:36:38.154153+0000","last_clean":"2020-11-16T22:36:38.154153+0000","last_became_active":"2020-11-16T22:35:21.130104+0000","last_became_peered":"2020-11-16T22:35:21.130104+0000","last_unstale":"2020-11-16T22:36:38.154153+0000","last_undegraded":"2020-11-16T22:36:38.154153+0000","last_fullsized":"2020-11-16T22:36:38.154153+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":81610,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"111'26","reported_seq":"38","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:04.895494+0000","last_change":"2020-11-16T22:35:21.178478+0000","last_active":"2020-11-16T22:36:04.895494+0000","last_peered":"2020-11-16T22:36:04.895494+0000","last_clean":"2020-11-16T22:36:04.895494+0000","last_became_active":"2020-11-16T22:35:21.177888+0000","last_became_peered":"2020-11-16T22:35:21.177888+0000","last_unstale":"2020-11-16T22:36:04.895494+0000","last_undegraded":"2020-11-16T22:36:04.895494+0000","last_fullsized":"2020-11-16T22:36:04.895494+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20641,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"111'19","reported_seq":"30","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:12.136875+0000","last_change":"2020-11-16T22:35:22.196350+0000","last_active":"2020-11-16T22:36:12.136875+0000","last_peered":"2020-11-16T22:36:12.136875+0000","last_clean":"2020-11-16T22:36:12.136875+0000","last_became_active":"2020-11-16T22:35:22.195927+0000","last_became_peered":"2020-11-16T22:35:22.195927+0000","last_unstale":"2020-11-16T22:36:12.136875+0000","last_undegraded":"2020-11-16T22:36:12.136875+0000","last_fullsized":"2020-11-16T22:36:12.136875+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1","version":"111'24","reported_seq":"37","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:38.154106+0000","last_change":"2020-11-16T22:35:21.179123+0000","last_active":"2020-11-16T22:36:38.154106+0000","last_peered":"2020-11-16T22:36:38.154106+0000","last_clean":"2020-11-16T22:36:38.154106+0000","last_became_active":"2020-11-16T22:35:21.178511+0000","last_became_peered":"2020-11-16T22:35:21.178511+0000","last_unstale":"2020-11-16T22:36:38.154106+0000","last_undegraded":"2020-11-16T22:36:38.154106+0000","last_fullsized":"2020-11-16T22:36:38.154106+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14153,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.0","version":"111'23","reported_seq":"34","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:01.330524+0000","last_change":"2020-11-16T22:35:22.266939+0000","last_active":"2020-11-16T22:36:01.330524+0000","last_peered":"2020-11-16T22:36:01.330524+0000","last_clean":"2020-11-16T22:36:01.330524+0000","last_became_active":"2020-11-16T22:35:22.266066+0000","last_became_peered":"2020-11-16T22:35:22.266066+0000","last_unstale":"2020-11-16T22:36:01.330524+0000","last_undegraded":"2020-11-16T22:36:01.330524+0000","last_fullsized":"2020-11-16T22:36:01.330524+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:20.960416+0000","last_clean_scrub_stamp":"2020-11-16T22:35:20.960416+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"74","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.581875+0000","last_change":"2020-11-16T22:26:48.228531+0000","last_active":"2020-11-16T22:35:22.581875+0000","last_peered":"2020-11-16T22:35:22.581875+0000","last_clean":"2020-11-16T22:35:22.581875+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:35:22.581875+0000","last_undegraded":"2020-11-16T22:35:22.581875+0000","last_fullsized":"2020-11-16T22:35:22.581875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.587008+0000","last_change":"2020-11-16T22:26:48.226107+0000","last_active":"2020-11-16T22:35:22.587008+0000","last_peered":"2020-11-16T22:35:22.587008+0000","last_clean":"2020-11-16T22:35:22.587008+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:35:22.587008+0000","last_undegraded":"2020-11-16T22:35:22.587008+0000","last_fullsized":"2020-11-16T22:35:22.587008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"77","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.575716+0000","last_change":"2020-11-16T22:26:48.145112+0000","last_active":"2020-11-16T22:35:22.575716+0000","last_peered":"2020-11-16T22:35:22.575716+0000","last_clean":"2020-11-16T22:35:22.575716+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:35:22.575716+0000","last_undegraded":"2020-11-16T22:35:22.575716+0000","last_fullsized":"2020-11-16T22:35:22.575716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.586204+0000","last_change":"2020-11-16T22:26:48.223877+0000","last_active":"2020-11-16T22:35:22.586204+0000","last_peered":"2020-11-16T22:35:22.586204+0000","last_clean":"2020-11-16T22:35:22.586204+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:35:22.586204+0000","last_undegraded":"2020-11-16T22:35:22.586204+0000","last_fullsized":"2020-11-16T22:35:22.586204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"96","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.578850+0000","last_change":"2020-11-16T22:26:44.683337+0000","last_active":"2020-11-16T22:35:22.578850+0000","last_peered":"2020-11-16T22:35:22.578850+0000","last_clean":"2020-11-16T22:35:22.578850+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:35:22.578850+0000","last_undegraded":"2020-11-16T22:35:22.578850+0000","last_fullsized":"2020-11-16T22:35:22.578850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:39.662675+0000","last_clean_scrub_stamp":"2020-11-16T22:26:39.662675+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.586201+0000","last_change":"2020-11-16T22:26:48.143290+0000","last_active":"2020-11-16T22:35:22.586201+0000","last_peered":"2020-11-16T22:35:22.586201+0000","last_clean":"2020-11-16T22:35:22.586201+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:35:22.586201+0000","last_undegraded":"2020-11-16T22:35:22.586201+0000","last_fullsized":"2020-11-16T22:35:22.586201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.588135+0000","last_change":"2020-11-16T22:26:48.141598+0000","last_active":"2020-11-16T22:35:22.588135+0000","last_peered":"2020-11-16T22:35:22.588135+0000","last_clean":"2020-11-16T22:35:22.588135+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:35:22.588135+0000","last_undegraded":"2020-11-16T22:35:22.588135+0000","last_fullsized":"2020-11-16T22:35:22.588135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"88","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.580628+0000","last_change":"2020-11-16T22:26:48.228444+0000","last_active":"2020-11-16T22:35:22.580628+0000","last_peered":"2020-11-16T22:35:22.580628+0000","last_clean":"2020-11-16T22:35:22.580628+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:35:22.580628+0000","last_undegraded":"2020-11-16T22:35:22.580628+0000","last_fullsized":"2020-11-16T22:35:22.580628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"85","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.588195+0000","last_change":"2020-11-16T22:26:48.225986+0000","last_active":"2020-11-16T22:35:22.588195+0000","last_peered":"2020-11-16T22:35:22.588195+0000","last_clean":"2020-11-16T22:35:22.588195+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:35:22.588195+0000","last_undegraded":"2020-11-16T22:35:22.588195+0000","last_fullsized":"2020-11-16T22:35:22.588195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:26:47.111928+0000","last_clean_scrub_stamp":"2020-11-16T22:26:47.111928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":8,"stat_sum":{"num_bytes":117916,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":257,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":235832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":16,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542269,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7744,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364096,"statfs":{"total":96636764160,"available":96628834304,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.762,"15min":1.766},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.65100000000000002},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.514},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.774,"15min":1.784},"min":{"1min":1.2070000000000001,"5min":1.014,"15min":0.68899999999999995},"max":{"1min":2.702,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":1.3009999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":2.105,"15min":2.0209999999999999},"min":{"1min":1.2849999999999999,"5min":0.93000000000000005,"15min":0.92500000000000004},"max":{"1min":2.7269999999999999,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":1.738},{"interface":"front","average":{"1min":1.871,"5min":1.8029999999999999,"15min":1.76},"min":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":0.73099999999999998},"max":{"1min":2.5830000000000002,"5min":2.7749999999999999,"15min":2.883},"last":2.1389999999999998}]},{"osd":2,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.3279999999999998,"5min":2.1520000000000001,"15min":2.0259999999999998},"min":{"1min":1.669,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":3.0529999999999999,"5min":3.0790000000000002,"15min":3.4780000000000002},"last":1.9410000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.2810000000000001,"15min":2.29},"min":{"1min":1.794,"5min":1.3,"15min":1.105},"max":{"1min":3.129,"5min":3.98,"15min":3.98},"last":2.2810000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.7789999999999999,"5min":2.5840000000000001,"15min":2.5609999999999999},"min":{"1min":1.546,"5min":1.4910000000000001,"15min":1.2190000000000001},"max":{"1min":3.6850000000000001,"5min":3.6850000000000001,"15min":3.9969999999999999},"last":2.4369999999999998},{"interface":"front","average":{"1min":2.9430000000000001,"5min":2.8460000000000001,"15min":2.823},"min":{"1min":2.2250000000000001,"5min":1.6020000000000001,"15min":1.2689999999999999},"max":{"1min":3.8420000000000001,"5min":4.266,"15min":4.266},"last":2.6619999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.5329999999999999,"5min":2.3069999999999999,"15min":2.2290000000000001},"min":{"1min":1.718,"5min":1.5169999999999999,"15min":1.018},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.5110000000000001},{"interface":"front","average":{"1min":2.996,"5min":2.7789999999999999,"15min":2.7690000000000001},"min":{"1min":2.3029999999999999,"5min":1.748,"15min":1.1479999999999999},"max":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9940000000000002},"last":2.2370000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.9910000000000001,"5min":2.9100000000000001,"15min":2.944},"min":{"1min":2.2480000000000002,"5min":1.4810000000000001,"15min":1.3540000000000001},"max":{"1min":4.3339999999999996,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":3.0179999999999998},{"interface":"front","average":{"1min":2.7690000000000001,"5min":2.746,"15min":2.7170000000000001},"min":{"1min":1.899,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":3.5259999999999998,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":2.875}]},{"osd":6,"last update":"Mon Nov 16 22:36:03 2020","interfaces":[{"interface":"back","average":{"1min":2.9830000000000001,"5min":2.8700000000000001,"15min":2.8650000000000002},"min":{"1min":2.1190000000000002,"5min":1.7569999999999999,"15min":1.2989999999999999},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":3.1829999999999998},{"interface":"front","average":{"1min":3.3100000000000001,"5min":3.2040000000000002,"15min":3.1699999999999999},"min":{"1min":2.6640000000000001,"5min":2.1629999999999998,"15min":1.222},"max":{"1min":4.5510000000000002,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":2.6190000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542268,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7508,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94364332,"statfs":{"total":96636764160,"available":96629075968,"internally_reserved":0,"allocated":1388544,"data_stored":630469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.696,"15min":1.6499999999999999},"min":{"1min":1.335,"5min":0.98699999999999999,"15min":0.97499999999999998},"max":{"1min":3.206,"5min":3.206,"15min":3.206},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.698,"5min":1.768,"15min":1.7490000000000001},"min":{"1min":1.083,"5min":1.083,"15min":0.94099999999999995},"max":{"1min":2.617,"5min":3.1469999999999998,"15min":3.1579999999999999},"last":1.722}]},{"osd":1,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":1.7669999999999999,"5min":1.768,"15min":1.6850000000000001},"min":{"1min":1.4450000000000001,"5min":1.1779999999999999,"15min":1.0469999999999999},"max":{"1min":2.8330000000000002,"5min":3.423,"15min":3.423},"last":1.6339999999999999},{"interface":"front","average":{"1min":2.347,"5min":2.1899999999999999,"15min":2.1709999999999998},"min":{"1min":1.2909999999999999,"5min":1.01,"15min":1.01},"max":{"1min":3.5270000000000001,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":2.9580000000000002}]},{"osd":2,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":2.2759999999999998,"15min":2.2170000000000001},"min":{"1min":1.764,"5min":1.282,"15min":1.131},"max":{"1min":3.3849999999999998,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.4769999999999999},{"interface":"front","average":{"1min":2.1560000000000001,"5min":2.101,"15min":2.069},"min":{"1min":1.4019999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":2.6640000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":2.4940000000000002,"15min":2.4449999999999998},"min":{"1min":1.6479999999999999,"5min":1.6419999999999999,"15min":1.1850000000000001},"max":{"1min":3.0019999999999998,"5min":3.2650000000000001,"15min":3.3340000000000001},"last":2.3010000000000002},{"interface":"front","average":{"1min":2.766,"5min":2.6739999999999999,"15min":2.5630000000000002},"min":{"1min":1.869,"5min":1.377,"15min":1.377},"max":{"1min":3.7130000000000001,"5min":3.7130000000000001,"15min":3.7130000000000001},"last":2.754}]},{"osd":4,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.4119999999999999,"5min":2.3759999999999999,"15min":2.3639999999999999},"min":{"1min":1.905,"5min":1.3380000000000001,"15min":1.214},"max":{"1min":3.335,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":2.222},{"interface":"front","average":{"1min":3.0219999999999998,"5min":2.8210000000000002,"15min":2.8069999999999999},"min":{"1min":2.246,"5min":1.9390000000000001,"15min":1.6879999999999999},"max":{"1min":3.915,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":3.3650000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9369999999999998,"5min":2.8250000000000002,"15min":2.8300000000000001},"min":{"1min":2.2410000000000001,"5min":1.8109999999999999,"15min":1.6379999999999999},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":3.2120000000000002},{"interface":"front","average":{"1min":3.3159999999999998,"5min":3.1560000000000001,"15min":3.1379999999999999},"min":{"1min":2.504,"5min":2.278,"15min":1.8680000000000001},"max":{"1min":4.6920000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":3.7320000000000002}]},{"osd":7,"last update":"Mon Nov 16 22:36:05 2020","interfaces":[{"interface":"back","average":{"1min":2.9580000000000002,"5min":2.875,"15min":2.887},"min":{"1min":2.2559999999999998,"5min":1.913,"15min":1.6000000000000001},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6469999999999998},"last":3.411},{"interface":"front","average":{"1min":3.3250000000000002,"5min":3.2450000000000001,"15min":3.1970000000000001},"min":{"1min":2.8170000000000002,"5min":2.1749999999999998,"15min":1.913},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":3.5739999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7716,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94364124,"statfs":{"total":96636764160,"available":96628862976,"internally_reserved":0,"allocated":1404928,"data_stored":644510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.581,"15min":1.611},"min":{"1min":1.24,"5min":0.95799999999999996,"15min":0.94899999999999995},"max":{"1min":2.871,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.6180000000000001,"15min":1.639},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":3.492}]},{"osd":1,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":1.925,"15min":1.853},"min":{"1min":1.2390000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":2.524},{"interface":"front","average":{"1min":2.379,"5min":2.4820000000000002,"15min":2.5089999999999999},"min":{"1min":1.417,"5min":1.417,"15min":1.417},"max":{"1min":3.4790000000000001,"5min":3.875,"15min":3.875},"last":2.4159999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.1720000000000002,"15min":2.2109999999999999},"min":{"1min":1.5,"5min":1.171,"15min":1.1259999999999999},"max":{"1min":2.8399999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":3.0680000000000001},{"interface":"front","average":{"1min":2.3559999999999999,"5min":2.327,"15min":2.3439999999999999},"min":{"1min":1.7330000000000001,"5min":1.583,"15min":1.258},"max":{"1min":3.0289999999999999,"5min":3.1739999999999999,"15min":3.4860000000000002},"last":2.883}]},{"osd":3,"last update":"Mon Nov 16 22:35:54 2020","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":2.226,"15min":2.2570000000000001},"min":{"1min":1.403,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":2.73,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":2.7749999999999999},{"interface":"front","average":{"1min":2.823,"5min":2.665,"15min":2.7410000000000001},"min":{"1min":2.0139999999999998,"5min":1.6559999999999999,"15min":1.5009999999999999},"max":{"1min":3.5070000000000001,"5min":3.7320000000000002,"15min":4.0830000000000002},"last":3.2890000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.5369999999999999,"5min":2.6080000000000001,"15min":2.6429999999999998},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.1509999999999998,"5min":5.6399999999999997,"15min":5.6399999999999997},"last":2},{"interface":"front","average":{"1min":2.79,"5min":2.6680000000000001,"15min":2.8010000000000002},"min":{"1min":1.9890000000000001,"5min":1.171,"15min":1.171},"max":{"1min":3.6619999999999999,"5min":4.766,"15min":4.766},"last":2.2330000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:35:53 2020","interfaces":[{"interface":"back","average":{"1min":2.6869999999999998,"5min":2.5230000000000001,"15min":2.5249999999999999},"min":{"1min":1.821,"5min":1.5129999999999999,"15min":1.5129999999999999},"max":{"1min":3.4089999999999998,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":2.1160000000000001},{"interface":"front","average":{"1min":3.0259999999999998,"5min":3.0179999999999998,"15min":3.0800000000000001},"min":{"1min":2.2959999999999998,"5min":1.516,"15min":1.516},"max":{"1min":3.8079999999999998,"5min":6.0380000000000003,"15min":6.0380000000000003},"last":3.121}]},{"osd":7,"last update":"Mon Nov 16 22:35:54 2020","interfaces":[{"interface":"back","average":{"1min":3.1240000000000001,"5min":3.073,"15min":3.21},"min":{"1min":2.5529999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"max":{"1min":3.7050000000000001,"5min":5.1379999999999999,"15min":5.3220000000000001},"last":2.681},{"interface":"front","average":{"1min":2.984,"5min":3.0750000000000002,"15min":3.077},"min":{"1min":2.0910000000000002,"5min":2.0459999999999998,"15min":2.0459999999999998},"max":{"1min":3.7549999999999999,"5min":5.2569999999999997,"15min":5.2569999999999997},"last":3.242}]}]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8384,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94363456,"statfs":{"total":96636764160,"available":96628178944,"internally_reserved":0,"allocated":1499136,"data_stored":731943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.7410000000000001,"15min":1.8160000000000001},"min":{"1min":1.2410000000000001,"5min":0.96799999999999997,"15min":0.65200000000000002},"max":{"1min":3.0139999999999998,"5min":3.02,"15min":3.02},"last":1.298},{"interface":"front","average":{"1min":2.2949999999999999,"5min":2.1720000000000002,"15min":2.1829999999999998},"min":{"1min":1.472,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.956,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":2.016}]},{"osd":1,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.9239999999999999,"5min":1.8029999999999999,"15min":1.7430000000000001},"min":{"1min":1.294,"5min":1.0580000000000001,"15min":0.74199999999999999},"max":{"1min":3.2360000000000002,"5min":3.302,"15min":3.302},"last":1.506},{"interface":"front","average":{"1min":2.0059999999999998,"5min":2.0259999999999998,"15min":1.8979999999999999},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.0660000000000001},"max":{"1min":3.0030000000000001,"5min":3.1869999999999998,"15min":3.3620000000000001},"last":1.6879999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.9410000000000001,"15min":1.9370000000000001},"min":{"1min":1.5069999999999999,"5min":1.29,"15min":1.054},"max":{"1min":2.931,"5min":4.117,"15min":4.117},"last":1.5780000000000001},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.2309999999999999,"15min":2.113},"min":{"1min":1.3759999999999999,"5min":1.198,"15min":1.0129999999999999},"max":{"1min":3.0939999999999999,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.2959999999999998}]},{"osd":3,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":2.3759999999999999,"15min":2.2610000000000001},"min":{"1min":1.2370000000000001,"5min":1.206,"15min":0.94599999999999995},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":2.0939999999999999},{"interface":"front","average":{"1min":2.577,"5min":2.7290000000000001,"15min":2.75},"min":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.2719999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":2.4409999999999998}]},{"osd":5,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4489999999999998},"min":{"1min":1.8779999999999999,"5min":1.361,"15min":1.075},"max":{"1min":3.448,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.925},{"interface":"front","average":{"1min":2.9809999999999999,"5min":2.9430000000000001,"15min":2.9060000000000001},"min":{"1min":2.258,"5min":1.6890000000000001,"15min":1.1160000000000001},"max":{"1min":4.0819999999999999,"5min":4.9859999999999998,"15min":4.9859999999999998},"last":2.8599999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.8999999999999999,"5min":2.851,"15min":2.8370000000000002},"min":{"1min":2.2080000000000002,"5min":1.78,"15min":1.5529999999999999},"max":{"1min":3.6850000000000001,"5min":4.3739999999999997,"15min":4.5220000000000002},"last":2.4910000000000001},{"interface":"front","average":{"1min":2.9700000000000002,"5min":2.8660000000000001,"15min":2.8180000000000001},"min":{"1min":2.1899999999999999,"5min":1.8959999999999999,"15min":1.6359999999999999},"max":{"1min":3.6429999999999998,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":2.6459999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.1259999999999999,"5min":3.1539999999999999,"15min":3.1110000000000002},"min":{"1min":2.3460000000000001,"5min":2.242,"15min":1.857},"max":{"1min":3.9449999999999998,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":2.254},{"interface":"front","average":{"1min":3.069,"5min":2.9980000000000002,"15min":2.9670000000000001},"min":{"1min":2.544,"5min":1.8440000000000001,"15min":1.496},"max":{"1min":3.7890000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":2.5649999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542268,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7824,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94364016,"statfs":{"total":96636764160,"available":96628752384,"internally_reserved":0,"allocated":1384448,"data_stored":630427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.677,"15min":1.6339999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3199999999999998,"5min":2.6469999999999998,"15min":2.8159999999999998},"last":2.0990000000000002},{"interface":"front","average":{"1min":1.665,"5min":1.72,"15min":1.694},"min":{"1min":1.1279999999999999,"5min":1.0449999999999999,"15min":1.02},"max":{"1min":2.0720000000000001,"5min":2.653,"15min":2.653},"last":1.9279999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6799999999999999,"15min":1.649},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.998,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":2.3039999999999998},{"interface":"front","average":{"1min":2.1259999999999999,"5min":2.1520000000000001,"15min":2.089},"min":{"1min":1.0149999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.641,"5min":2.9119999999999999,"15min":3.4119999999999999},"last":1.446}]},{"osd":3,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1280000000000001,"5min":2.1619999999999999,"15min":2.0409999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.7160000000000002,"5min":3.1509999999999998,"15min":3.2559999999999998},"last":1.5960000000000001},{"interface":"front","average":{"1min":2.3450000000000002,"5min":2.379,"15min":2.29},"min":{"1min":1.056,"5min":1.056,"15min":0.99399999999999999},"max":{"1min":3.2469999999999999,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":2.714}]},{"osd":4,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.5379999999999998,"5min":2.5649999999999999,"15min":2.6920000000000002},"min":{"1min":1.603,"5min":1.5129999999999999,"15min":1.5029999999999999},"max":{"1min":3.2210000000000001,"5min":3.8559999999999999,"15min":4.8479999999999999},"last":2.6389999999999998},{"interface":"front","average":{"1min":2.3889999999999998,"5min":2.4279999999999999,"15min":2.4169999999999998},"min":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.1930000000000001},"max":{"1min":3.319,"5min":3.956,"15min":4.0380000000000003},"last":1.6519999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8650000000000002,"5min":2.923,"15min":2.9540000000000002},"min":{"1min":2.2530000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"max":{"1min":3.5449999999999999,"5min":3.8700000000000001,"15min":5.0659999999999998},"last":2.855},{"interface":"front","average":{"1min":2.6629999999999998,"5min":2.7530000000000001,"15min":2.661},"min":{"1min":1.8,"5min":1.6779999999999999,"15min":1.6180000000000001},"max":{"1min":3.3239999999999998,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":2.1760000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8679999999999999,"5min":2.8809999999999998,"15min":2.7989999999999999},"min":{"1min":1.772,"5min":1.772,"15min":1.772},"max":{"1min":3.3410000000000002,"5min":4.2599999999999998,"15min":4.2599999999999998},"last":3.0990000000000002},{"interface":"front","average":{"1min":3.125,"5min":3.1629999999999998,"15min":3.2010000000000001},"min":{"1min":2.2210000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"max":{"1min":3.9060000000000001,"5min":4.3490000000000002,"15min":5.2329999999999997},"last":3.0579999999999998}]},{"osd":7,"last update":"Mon Nov 16 22:36:06 2020","interfaces":[{"interface":"back","average":{"1min":3.3340000000000001,"5min":3.3370000000000002,"15min":3.3530000000000002},"min":{"1min":2.3999999999999999,"5min":2.1320000000000001,"15min":2.0489999999999999},"max":{"1min":4.2779999999999996,"5min":4.5910000000000002,"15min":5.6379999999999999},"last":3.2069999999999999},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.004,"15min":2.9399999999999999},"min":{"1min":2.036,"5min":2.036,"15min":1.895},"max":{"1min":3.5150000000000001,"5min":4.6059999999999999,"15min":4.6059999999999999},"last":2.7890000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7732,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94364108,"statfs":{"total":96636764160,"available":96628846592,"internally_reserved":0,"allocated":1486848,"data_stored":725455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.667,"15min":1.6339999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.70199999999999996},"max":{"1min":2.8450000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.79,"5min":1.7330000000000001,"15min":1.6859999999999999},"min":{"1min":1.3819999999999999,"5min":1.119,"15min":0.83699999999999997},"max":{"1min":2.6819999999999999,"5min":2.6829999999999998,"15min":2.718},"last":1.5509999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":1.6619999999999999,"5min":1.8029999999999999,"15min":1.716},"min":{"1min":1.3440000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.2789999999999999,"5min":10.023,"15min":10.023},"last":1.728},{"interface":"front","average":{"1min":2.1960000000000002,"5min":2.2440000000000002,"15min":2.097},"min":{"1min":1.345,"5min":0.95499999999999996,"15min":0.80400000000000005},"max":{"1min":3.2530000000000001,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":2.4260000000000002}]},{"osd":3,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.0059999999999998,"5min":2.0609999999999999,"15min":2.1320000000000001},"min":{"1min":1.339,"5min":1.1319999999999999,"15min":0.81000000000000005},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":3.3319999999999999},"last":2.149},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2519999999999998,"15min":2.2130000000000001},"min":{"1min":1.387,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.214,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":2.2999999999999998}]},{"osd":4,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.226,"5min":2.383,"15min":2.3410000000000002},"min":{"1min":1.7410000000000001,"5min":1.008,"15min":1.008},"max":{"1min":3.0379999999999998,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":2.6299999999999999},{"interface":"front","average":{"1min":2.581,"5min":2.6739999999999999,"15min":2.6760000000000002},"min":{"1min":1.7969999999999999,"5min":1.022,"15min":1.022},"max":{"1min":3.379,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":2.956}]},{"osd":5,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":2.9750000000000001,"15min":3.0179999999999998},"min":{"1min":1.581,"5min":1.276,"15min":1.276},"max":{"1min":3.7639999999999998,"5min":5.351,"15min":5.351},"last":3.411},{"interface":"front","average":{"1min":2.778,"5min":2.7320000000000002,"15min":2.8050000000000002},"min":{"1min":2.153,"5min":1.4279999999999999,"15min":1.3839999999999999},"max":{"1min":3.4809999999999999,"5min":3.7349999999999999,"15min":3.863},"last":2.774}]},{"osd":6,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.016,"5min":3.0009999999999999,"15min":3},"min":{"1min":2.2480000000000002,"5min":1.5269999999999999,"15min":1.5269999999999999},"max":{"1min":3.931,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":3.0369999999999999},{"interface":"front","average":{"1min":2.8130000000000002,"5min":2.9239999999999999,"15min":2.8690000000000002},"min":{"1min":2.3180000000000001,"5min":1.8149999999999999,"15min":1.4670000000000001},"max":{"1min":3.6930000000000001,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":3.081}]},{"osd":7,"last update":"Mon Nov 16 22:35:57 2020","interfaces":[{"interface":"back","average":{"1min":3.1139999999999999,"5min":3.1230000000000002,"15min":3.0979999999999999},"min":{"1min":2.2000000000000002,"5min":2.008,"15min":1.7010000000000001},"max":{"1min":4.1239999999999997,"5min":4.1239999999999997,"15min":4.7430000000000003},"last":3.6299999999999999},{"interface":"front","average":{"1min":3.2610000000000001,"5min":3.1800000000000002,"15min":3.2120000000000002},"min":{"1min":2.6619999999999999,"5min":1.712,"15min":1.712},"max":{"1min":4.3899999999999997,"5min":4.3899999999999997,"15min":4.4029999999999996},"last":3.355}]}]},{"osd":2,"up_from":14,"seq":60129542269,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7936,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94363904,"statfs":{"total":96636764160,"available":96628637696,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.671,"15min":1.6519999999999999},"min":{"1min":0.84699999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":3.0710000000000002},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.845,"5min":1.776,"15min":1.7929999999999999},"min":{"1min":1.1950000000000001,"5min":1.032,"15min":0.81899999999999995},"max":{"1min":2.484,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":1.5069999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.946,"5min":1.897,"15min":1.837},"min":{"1min":1.1240000000000001,"5min":1.0069999999999999,"15min":0.91500000000000004},"max":{"1min":2.6400000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.853},{"interface":"front","average":{"1min":2.0089999999999999,"5min":1.986,"15min":1.944},"min":{"1min":1.2929999999999999,"5min":0.81499999999999995,"15min":0.80500000000000005},"max":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":2.3490000000000002}]},{"osd":3,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":1.911,"5min":1.8859999999999999,"15min":1.8620000000000001},"min":{"1min":1.2989999999999999,"5min":1.0840000000000001,"15min":0.92900000000000005},"max":{"1min":2.4929999999999999,"5min":3.0840000000000001,"15min":3.9660000000000002},"last":1.669},{"interface":"front","average":{"1min":2.544,"5min":2.581,"15min":2.464},"min":{"1min":1.5860000000000001,"5min":1.1830000000000001,"15min":1.1499999999999999},"max":{"1min":3.238,"5min":3.944,"15min":3.944},"last":2.7090000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":2.5979999999999999,"15min":2.512},"min":{"1min":1.782,"5min":1.5800000000000001,"15min":1.498},"max":{"1min":3.423,"5min":5.0209999999999999,"15min":5.0209999999999999},"last":3.137},{"interface":"front","average":{"1min":2.4249999999999998,"5min":2.5470000000000002,"15min":2.569},"min":{"1min":1.452,"5min":1.452,"15min":1.3260000000000001},"max":{"1min":3.1520000000000001,"5min":3.7400000000000002,"15min":3.8010000000000002},"last":2.847}]},{"osd":5,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.6930000000000001,"5min":2.7909999999999999,"15min":2.8010000000000002},"min":{"1min":2.0219999999999998,"5min":1.6080000000000001,"15min":1.6080000000000001},"max":{"1min":3.5310000000000001,"5min":3.738,"15min":4.0259999999999998},"last":3.5},{"interface":"front","average":{"1min":2.2440000000000002,"5min":2.4100000000000001,"15min":2.4540000000000002},"min":{"1min":1.639,"5min":1.639,"15min":1.363},"max":{"1min":2.9140000000000001,"5min":3.7970000000000002,"15min":4.3070000000000004},"last":2.0640000000000001}]},{"osd":6,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":2.589,"5min":2.6800000000000002,"15min":2.681},"min":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":3.6699999999999999,"5min":4.1059999999999999,"15min":4.165},"last":2.427},{"interface":"front","average":{"1min":2.907,"5min":3.0430000000000001,"15min":2.9470000000000001},"min":{"1min":2.2360000000000002,"5min":2.0550000000000002,"15min":1.9930000000000001},"max":{"1min":3.5369999999999999,"5min":4.4450000000000003,"15min":4.4450000000000003},"last":3.3479999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:36:04 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":3.0800000000000001,"15min":3.0790000000000002},"min":{"1min":2.4260000000000002,"5min":1.903,"15min":1.8879999999999999},"max":{"1min":3.8410000000000002,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":3.0529999999999999},{"interface":"front","average":{"1min":2.7850000000000001,"5min":2.9710000000000001,"15min":3.0089999999999999},"min":{"1min":1.488,"5min":1.488,"15min":1.488},"max":{"1min":3.6120000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":2.794}]}]},{"osd":3,"up_from":14,"seq":60129542267,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8252,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":6848,"kb_avail":94363588,"statfs":{"total":96636764160,"available":96628314112,"internally_reserved":0,"allocated":1429504,"data_stored":651180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7012352},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.677,"15min":1.6359999999999999},"min":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":0.73599999999999999},"max":{"1min":2.621,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.8169999999999999,"5min":1.7729999999999999,"15min":1.774},"min":{"1min":1.3660000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":3.6739999999999999},"last":1.387}]},{"osd":1,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.6579999999999999,"15min":1.6339999999999999},"min":{"1min":1.1579999999999999,"5min":0.872,"15min":0.81100000000000005},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.77},"last":2.1110000000000002},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.1779999999999999,"15min":2.173},"min":{"1min":1.5800000000000001,"5min":1.25,"15min":1.099},"max":{"1min":5.258,"5min":5.258,"15min":5.258},"last":1.7649999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.1499999999999999,"5min":2.2010000000000001,"15min":2.1800000000000002},"min":{"1min":1.4750000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":3.1030000000000002,"5min":3.214,"15min":3.597},"last":2.0590000000000002},{"interface":"front","average":{"1min":2.488,"5min":2.3999999999999999,"15min":2.3759999999999999},"min":{"1min":1.776,"5min":1.3700000000000001,"15min":1.075},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0990000000000002},"last":1.9359999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.48,"5min":2.5779999999999998,"15min":2.5329999999999999},"min":{"1min":1.782,"5min":1.298,"15min":1.2150000000000001},"max":{"1min":3.5070000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":2.573},{"interface":"front","average":{"1min":2.371,"5min":2.4009999999999998,"15min":2.403},"min":{"1min":1.758,"5min":1.403,"15min":1.1699999999999999},"max":{"1min":3.3210000000000002,"5min":3.681,"15min":3.6829999999999998},"last":1.8899999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7879999999999998,"15min":2.7530000000000001},"min":{"1min":1.9750000000000001,"5min":1.5600000000000001,"15min":1.2709999999999999},"max":{"1min":3.5950000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":2.319},{"interface":"front","average":{"1min":3.0529999999999999,"5min":2.9620000000000002,"15min":3.0009999999999999},"min":{"1min":2.1120000000000001,"5min":1.857,"15min":1.8360000000000001},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":2.6349999999999998}]},{"osd":6,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.1680000000000001,"5min":3.0600000000000001,"15min":3.1080000000000001},"min":{"1min":2.4159999999999999,"5min":1.615,"15min":1.615},"max":{"1min":4.3970000000000002,"5min":4.3970000000000002,"15min":4.5359999999999996},"last":3.0350000000000001},{"interface":"front","average":{"1min":3.032,"5min":3.0609999999999999,"15min":3.093},"min":{"1min":2.1890000000000001,"5min":2.04,"15min":1.7310000000000001},"max":{"1min":3.895,"5min":4.1959999999999997,"15min":4.5590000000000002},"last":2.9489999999999998}]},{"osd":7,"last update":"Mon Nov 16 22:36:00 2020","interfaces":[{"interface":"back","average":{"1min":3.194,"5min":3.2309999999999999,"15min":3.1669999999999998},"min":{"1min":2.4380000000000002,"5min":2.0259999999999998,"15min":1.8240000000000001},"max":{"1min":4.343,"5min":4.343,"15min":4.6470000000000002},"last":2.7730000000000001},{"interface":"front","average":{"1min":2.9159999999999999,"5min":2.9649999999999999,"15min":2.9409999999999998},"min":{"1min":2.0390000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"max":{"1min":3.831,"5min":3.98,"15min":3.98},"last":2.5249999999999999}]}]}]}} 2020-11-16T22:36:54.050 INFO:tasks.ceph:pgid 16.7 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.050 INFO:tasks.ceph:pgid 16.6 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.051 INFO:tasks.ceph:pgid 16.5 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.051 INFO:tasks.ceph:pgid 16.4 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.051 INFO:tasks.ceph:pgid 16.3 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.051 INFO:tasks.ceph:pgid 16.2 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.052 INFO:tasks.ceph:pgid 16.1 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.052 INFO:tasks.ceph:pgid 16.0 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.052 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.052 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.053 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.053 INFO:tasks.ceph:pgid 15.4 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.053 INFO:tasks.ceph:pgid 15.3 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.053 INFO:tasks.ceph:pgid 15.2 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.053 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.054 INFO:tasks.ceph:pgid 15.0 last_scrub_stamp 2020-11-16T22:35:20.960416+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=20, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.054 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.054 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.054 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.055 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.055 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-11-16T22:26:39.662675+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=39, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.055 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.055 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.055 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.056 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-11-16T22:26:47.111928+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=26, tm_sec=47, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:54.056 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-16T22:37:04.853 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:37:04.860 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:37:04.862 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:37:14.057 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:37:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 Processor -- start 2020-11-16T22:37:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- start start 2020-11-16T22:37:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7684161480 0x7f7684164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:37:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76840c8db0 con 0x7f7684164e10 2020-11-16T22:37:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76840c8330 con 0x7f7684161860 2020-11-16T22:37:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76840c85d0 con 0x7f7684161480 2020-11-16T22:37:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7684161480 0x7f7684164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:37:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7682d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7684161860 0x7f76840644e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52828/0 (socket says 172.21.15.181:52828) 2020-11-16T22:37:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7682d9d700 1 -- 172.21.15.181:0/399341756 learned_addr learned my addr 172.21.15.181:0/399341756 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:37:14.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768259c700 1 -- 172.21.15.181:0/399341756 >> v1:172.21.15.181:6789/0 conn(0x7f7684161860 legacy=0x7f76840644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7681d9b700 1 -- 172.21.15.181:0/399341756 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934468489 0 0) 0x7f76840c8330 con 0x7f7684161860 2020-11-16T22:37:14.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768259c700 1 -- 172.21.15.181:0/399341756 >> v1:172.21.15.175:6789/0 conn(0x7f7684164e10 legacy=0x7f7684062250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.195 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768259c700 1 -- 172.21.15.181:0/399341756 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840c8870 con 0x7f7684161480 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768259c700 1 --2- 172.21.15.181:0/399341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161480 0x7f7684164920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f767400b9a0 tx=0x7f7674006df0).ready entity=mon.1 client_cookie=86e3c10b09a0c416 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7681d9b700 1 -- 172.21.15.181:0/399341756 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76740096b0 con 0x7f7684161480 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7681d9b700 1 -- 172.21.15.181:0/399341756 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674009810 con 0x7f7684161480 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f7681d9b700 1 -- 172.21.15.181:0/399341756 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674009b90 con 0x7f7684161480 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- 172.21.15.181:0/399341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161480 msgr2=0x7f7684164920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 --2- 172.21.15.181:0/399341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161480 0x7f7684164920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f767400b9a0 tx=0x7f7674006df0).stop 2020-11-16T22:37:14.196 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- 172.21.15.181:0/399341756 shutdown_connections 2020-11-16T22:37:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 --2- 172.21.15.181:0/399341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161480 0x7f7684164920 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- 172.21.15.181:0/399341756 >> 172.21.15.181:0/399341756 conn(0x7f76840d2dc0 msgr2=0x7f76840ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:37:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- 172.21.15.181:0/399341756 shutdown_connections 2020-11-16T22:37:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.203+0000 7f768359e700 1 -- 172.21.15.181:0/399341756 wait complete. 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 Processor -- start 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- start start 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 0x7f7684148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7684173850 0x7f76840c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:37:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684174ca0 con 0x7f7684173850 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684174760 con 0x7f7684161860 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7684173ce0 con 0x7f7684161480 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 0x7f7684148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 0x7f7684148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46368/0 (socket says 172.21.15.181:46368) 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38528/0 (socket says 172.21.15.181:38528) 2020-11-16T22:37:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768259c700 1 -- 172.21.15.181:0/3500045814 learned_addr learned my addr 172.21.15.181:0/3500045814 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7682d9d700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7684173850 0x7f76840c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 -- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 msgr2=0x7f7684148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 0x7f7684148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 -- 172.21.15.181:0/3500045814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7684173850 msgr2=0x7f76840c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7684173850 0x7f76840c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 -- 172.21.15.181:0/3500045814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684173f80 con 0x7f7684161860 2020-11-16T22:37:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f7679d9b700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f76840c91e0 tx=0x7f766c007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:37:14.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00e560 con 0x7f7684161860 2020-11-16T22:37:14.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00e700 con 0x7f7684161860 2020-11-16T22:37:14.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7684174220 con 0x7f7684161860 2020-11-16T22:37:14.406 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00eac0 con 0x7f7684161860 2020-11-16T22:37:14.407 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.207+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7684165310 con 0x7f7684161860 2020-11-16T22:37:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76841654d0 con 0x7f7684161860 2020-11-16T22:37:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f766c010030 con 0x7f7684161860 2020-11-16T22:37:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f767affd700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 0x7f7660057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:37:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f768259c700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 0x7f7660057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:37:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f766c06daa0 con 0x7f7684161860 2020-11-16T22:37:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.211+0000 7f768259c700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 0x7f7660057520 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f767400a010 tx=0x7f7674015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:37:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.219+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f766c043d00 con 0x7f7684161860 2020-11-16T22:37:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.371+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7684044f60 con 0x7f7660055060 2020-11-16T22:37:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f767affd700 1 -- 172.21.15.181:0/3500045814 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+104312 (secure 0 0 0) 0x7f7684044f60 con 0x7f7660055060 2020-11-16T22:37:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 msgr2=0x7f7660057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 0x7f7660057520 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f767400a010 tx=0x7f7674015000).stop 2020-11-16T22:37:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 msgr2=0x7f7684176440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f76840c91e0 tx=0x7f766c007b00).stop 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 shutdown_connections 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f7660055060 0x7f7660057520 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7684173850 0x7f76840c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7684161480 0x7f7684148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 --2- 172.21.15.181:0/3500045814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7684161860 0x7f7684176440 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:37:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 >> 172.21.15.181:0/3500045814 conn(0x7f76840d2dc0 msgr2=0x7f7684141c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:37:14.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 shutdown_connections 2020-11-16T22:37:14.616 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:37:14.379+0000 7f768359e700 1 -- 172.21.15.181:0/3500045814 wait complete. 2020-11-16T22:37:14.616 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:37:14.616 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:37:14.616 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2020-11-16T22:37:12.869580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117916,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":259,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19808,"num_omap_keys":43,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":63480,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":52352,"kb_avail":754911240,"statfs":{"total":773094113280,"available":773029109760,"internally_reserved":0,"allocated":11329536,"data_stored":5273368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53608448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18157,"num_omap_keys":40,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003745"},"pg_stats":[{"pgid":"16.7","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:06.868904+0000","last_change":"2020-11-16T22:37:06.869075+0000","last_active":"2020-11-16T22:37:06.868904+0000","last_peered":"2020-11-16T22:37:06.868904+0000","last_clean":"2020-11-16T22:37:06.868904+0000","last_became_active":"2020-11-16T22:35:22.592949+0000","last_became_peered":"2020-11-16T22:35:22.592949+0000","last_unstale":"2020-11-16T22:37:06.868904+0000","last_undegraded":"2020-11-16T22:37:06.868904+0000","last_fullsized":"2020-11-16T22:37:06.868904+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:06.868894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:06.868894+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.868894+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:07.089435+0000","last_change":"2020-11-16T22:37:07.089634+0000","last_active":"2020-11-16T22:37:07.089435+0000","last_peered":"2020-11-16T22:37:07.089435+0000","last_clean":"2020-11-16T22:37:07.089435+0000","last_became_active":"2020-11-16T22:35:22.590435+0000","last_became_peered":"2020-11-16T22:35:22.590435+0000","last_unstale":"2020-11-16T22:37:07.089435+0000","last_undegraded":"2020-11-16T22:37:07.089435+0000","last_fullsized":"2020-11-16T22:37:07.089435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:07.089425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:07.089425+0000","last_clean_scrub_stamp":"2020-11-16T22:37:07.089425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:05.882323+0000","last_change":"2020-11-16T22:37:05.882556+0000","last_active":"2020-11-16T22:37:05.882323+0000","last_peered":"2020-11-16T22:37:05.882323+0000","last_clean":"2020-11-16T22:37:05.882323+0000","last_became_active":"2020-11-16T22:35:22.586786+0000","last_became_peered":"2020-11-16T22:35:22.586786+0000","last_unstale":"2020-11-16T22:37:05.882323+0000","last_undegraded":"2020-11-16T22:37:05.882323+0000","last_fullsized":"2020-11-16T22:37:05.882323+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:05.882309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:05.882309+0000","last_clean_scrub_stamp":"2020-11-16T22:37:05.882309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:02.933270+0000","last_change":"2020-11-16T22:37:02.933532+0000","last_active":"2020-11-16T22:37:02.933270+0000","last_peered":"2020-11-16T22:37:02.933270+0000","last_clean":"2020-11-16T22:37:02.933270+0000","last_became_active":"2020-11-16T22:35:22.586336+0000","last_became_peered":"2020-11-16T22:35:22.586336+0000","last_unstale":"2020-11-16T22:37:02.933270+0000","last_undegraded":"2020-11-16T22:37:02.933270+0000","last_fullsized":"2020-11-16T22:37:02.933270+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:02.933260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:02.933260+0000","last_clean_scrub_stamp":"2020-11-16T22:37:02.933260+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:04.166552+0000","last_change":"2020-11-16T22:37:04.166741+0000","last_active":"2020-11-16T22:37:04.166552+0000","last_peered":"2020-11-16T22:37:04.166552+0000","last_clean":"2020-11-16T22:37:04.166552+0000","last_became_active":"2020-11-16T22:35:22.602895+0000","last_became_peered":"2020-11-16T22:35:22.602895+0000","last_unstale":"2020-11-16T22:37:04.166552+0000","last_undegraded":"2020-11-16T22:37:04.166552+0000","last_fullsized":"2020-11-16T22:37:04.166552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:04.166535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:04.166535+0000","last_clean_scrub_stamp":"2020-11-16T22:37:04.166535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.642505+0000","last_change":"2020-11-16T22:36:53.642671+0000","last_active":"2020-11-16T22:36:53.642505+0000","last_peered":"2020-11-16T22:36:53.642505+0000","last_clean":"2020-11-16T22:36:53.642505+0000","last_became_active":"2020-11-16T22:35:22.593172+0000","last_became_peered":"2020-11-16T22:35:22.593172+0000","last_unstale":"2020-11-16T22:36:53.642505+0000","last_undegraded":"2020-11-16T22:36:53.642505+0000","last_fullsized":"2020-11-16T22:36:53.642505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:53.642496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:53.642496+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.642496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:55.933703+0000","last_change":"2020-11-16T22:36:55.933898+0000","last_active":"2020-11-16T22:36:55.933703+0000","last_peered":"2020-11-16T22:36:55.933703+0000","last_clean":"2020-11-16T22:36:55.933703+0000","last_became_active":"2020-11-16T22:35:22.588015+0000","last_became_peered":"2020-11-16T22:35:22.588015+0000","last_unstale":"2020-11-16T22:36:55.933703+0000","last_undegraded":"2020-11-16T22:36:55.933703+0000","last_fullsized":"2020-11-16T22:36:55.933703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:55.933690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:55.933690+0000","last_clean_scrub_stamp":"2020-11-16T22:36:55.933690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":"9","reported_epoch":"110","state":"active+clean","last_fresh":"2020-11-16T22:35:22.602537+0000","last_change":"2020-11-16T22:35:22.601873+0000","last_active":"2020-11-16T22:35:22.602537+0000","last_peered":"2020-11-16T22:35:22.602537+0000","last_clean":"2020-11-16T22:35:22.602537+0000","last_became_active":"2020-11-16T22:35:22.601487+0000","last_became_peered":"2020-11-16T22:35:22.601487+0000","last_unstale":"2020-11-16T22:35:22.602537+0000","last_undegraded":"2020-11-16T22:35:22.602537+0000","last_fullsized":"2020-11-16T22:35:22.602537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:35:22.171935+0000","last_clean_scrub_stamp":"2020-11-16T22:35:22.171935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"111'38","reported_seq":"52","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:06.957094+0000","last_change":"2020-11-16T22:37:06.957207+0000","last_active":"2020-11-16T22:37:06.957094+0000","last_peered":"2020-11-16T22:37:06.957094+0000","last_clean":"2020-11-16T22:37:06.957094+0000","last_became_active":"2020-11-16T22:35:22.191742+0000","last_became_peered":"2020-11-16T22:35:22.191742+0000","last_unstale":"2020-11-16T22:37:06.957094+0000","last_undegraded":"2020-11-16T22:37:06.957094+0000","last_fullsized":"2020-11-16T22:37:06.957094+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'38","last_scrub_stamp":"2020-11-16T22:37:06.957085+0000","last_deep_scrub":"111'38","last_deep_scrub_stamp":"2020-11-16T22:37:06.957085+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.957085+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":942,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.6","version":"111'17","reported_seq":"31","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:03.862617+0000","last_change":"2020-11-16T22:37:03.862803+0000","last_active":"2020-11-16T22:37:03.862617+0000","last_peered":"2020-11-16T22:37:03.862617+0000","last_clean":"2020-11-16T22:37:03.862617+0000","last_became_active":"2020-11-16T22:35:22.195870+0000","last_became_peered":"2020-11-16T22:35:22.195870+0000","last_unstale":"2020-11-16T22:37:03.862617+0000","last_undegraded":"2020-11-16T22:37:03.862617+0000","last_fullsized":"2020-11-16T22:37:03.862617+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'17","last_scrub_stamp":"2020-11-16T22:37:03.862600+0000","last_deep_scrub":"111'17","last_deep_scrub_stamp":"2020-11-16T22:37:03.862600+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.862600+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"111'29","reported_seq":"44","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:03.213006+0000","last_change":"2020-11-16T22:37:03.213129+0000","last_active":"2020-11-16T22:37:03.213006+0000","last_peered":"2020-11-16T22:37:03.213006+0000","last_clean":"2020-11-16T22:37:03.213006+0000","last_became_active":"2020-11-16T22:35:21.257651+0000","last_became_peered":"2020-11-16T22:35:21.257651+0000","last_unstale":"2020-11-16T22:37:03.213006+0000","last_undegraded":"2020-11-16T22:37:03.213006+0000","last_fullsized":"2020-11-16T22:37:03.213006+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'29","last_scrub_stamp":"2020-11-16T22:37:03.212996+0000","last_deep_scrub":"111'29","last_deep_scrub_stamp":"2020-11-16T22:37:03.212996+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.212996+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5652,"num_omap_keys":12,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"111'61","reported_seq":"77","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.903709+0000","last_change":"2020-11-16T22:36:56.903915+0000","last_active":"2020-11-16T22:36:56.903709+0000","last_peered":"2020-11-16T22:36:56.903709+0000","last_clean":"2020-11-16T22:36:56.903709+0000","last_became_active":"2020-11-16T22:35:21.130104+0000","last_became_peered":"2020-11-16T22:35:21.130104+0000","last_unstale":"2020-11-16T22:36:56.903709+0000","last_undegraded":"2020-11-16T22:36:56.903709+0000","last_fullsized":"2020-11-16T22:36:56.903709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'61","last_scrub_stamp":"2020-11-16T22:36:56.903691+0000","last_deep_scrub":"111'61","last_deep_scrub_stamp":"2020-11-16T22:36:56.903691+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.903691+0000","log_size":61,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":81610,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"111'36","reported_seq":"51","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:01.232357+0000","last_change":"2020-11-16T22:37:01.232533+0000","last_active":"2020-11-16T22:37:01.232357+0000","last_peered":"2020-11-16T22:37:01.232357+0000","last_clean":"2020-11-16T22:37:01.232357+0000","last_became_active":"2020-11-16T22:35:21.177888+0000","last_became_peered":"2020-11-16T22:35:21.177888+0000","last_unstale":"2020-11-16T22:37:01.232357+0000","last_undegraded":"2020-11-16T22:37:01.232357+0000","last_fullsized":"2020-11-16T22:37:01.232357+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'36","last_scrub_stamp":"2020-11-16T22:37:01.232343+0000","last_deep_scrub":"111'36","last_deep_scrub_stamp":"2020-11-16T22:37:01.232343+0000","last_clean_scrub_stamp":"2020-11-16T22:37:01.232343+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20641,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9928,"num_omap_keys":22,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"111'23","reported_seq":"37","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:00.782034+0000","last_change":"2020-11-16T22:37:00.782199+0000","last_active":"2020-11-16T22:37:00.782034+0000","last_peered":"2020-11-16T22:37:00.782034+0000","last_clean":"2020-11-16T22:37:00.782034+0000","last_became_active":"2020-11-16T22:35:22.195927+0000","last_became_peered":"2020-11-16T22:35:22.195927+0000","last_unstale":"2020-11-16T22:37:00.782034+0000","last_undegraded":"2020-11-16T22:37:00.782034+0000","last_fullsized":"2020-11-16T22:37:00.782034+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'23","last_scrub_stamp":"2020-11-16T22:37:00.782020+0000","last_deep_scrub":"111'23","last_deep_scrub_stamp":"2020-11-16T22:37:00.782020+0000","last_clean_scrub_stamp":"2020-11-16T22:37:00.782020+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":942,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1","version":"111'30","reported_seq":"46","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:59.606089+0000","last_change":"2020-11-16T22:36:59.606291+0000","last_active":"2020-11-16T22:36:59.606089+0000","last_peered":"2020-11-16T22:36:59.606089+0000","last_clean":"2020-11-16T22:36:59.606089+0000","last_became_active":"2020-11-16T22:35:21.178511+0000","last_became_peered":"2020-11-16T22:35:21.178511+0000","last_unstale":"2020-11-16T22:36:59.606089+0000","last_undegraded":"2020-11-16T22:36:59.606089+0000","last_fullsized":"2020-11-16T22:36:59.606089+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'30","last_scrub_stamp":"2020-11-16T22:36:59.606072+0000","last_deep_scrub":"111'30","last_deep_scrub_stamp":"2020-11-16T22:36:59.606072+0000","last_clean_scrub_stamp":"2020-11-16T22:36:59.606072+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14153,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":693,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.0","version":"111'34","reported_seq":"48","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:57.784324+0000","last_change":"2020-11-16T22:36:57.784479+0000","last_active":"2020-11-16T22:36:57.784324+0000","last_peered":"2020-11-16T22:36:57.784324+0000","last_clean":"2020-11-16T22:36:57.784324+0000","last_became_active":"2020-11-16T22:35:22.266066+0000","last_became_peered":"2020-11-16T22:35:22.266066+0000","last_unstale":"2020-11-16T22:36:57.784324+0000","last_undegraded":"2020-11-16T22:36:57.784324+0000","last_fullsized":"2020-11-16T22:36:57.784324+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'34","last_scrub_stamp":"2020-11-16T22:36:57.784312+0000","last_deep_scrub":"111'34","last_deep_scrub_stamp":"2020-11-16T22:36:57.784312+0000","last_clean_scrub_stamp":"2020-11-16T22:36:57.784312+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":471,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"77","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.725413+0000","last_change":"2020-11-16T22:36:56.725629+0000","last_active":"2020-11-16T22:36:56.725413+0000","last_peered":"2020-11-16T22:36:56.725413+0000","last_clean":"2020-11-16T22:36:56.725413+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:36:56.725413+0000","last_undegraded":"2020-11-16T22:36:56.725413+0000","last_fullsized":"2020-11-16T22:36:56.725413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:56.725392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:56.725392+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.725392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:54.187347+0000","last_change":"2020-11-16T22:36:54.187547+0000","last_active":"2020-11-16T22:36:54.187347+0000","last_peered":"2020-11-16T22:36:54.187347+0000","last_clean":"2020-11-16T22:36:54.187347+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:36:54.187347+0000","last_undegraded":"2020-11-16T22:36:54.187347+0000","last_fullsized":"2020-11-16T22:36:54.187347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:54.187336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:54.187336+0000","last_clean_scrub_stamp":"2020-11-16T22:36:54.187336+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"80","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:57.588001+0000","last_change":"2020-11-16T22:36:57.588200+0000","last_active":"2020-11-16T22:36:57.588001+0000","last_peered":"2020-11-16T22:36:57.588001+0000","last_clean":"2020-11-16T22:36:57.588001+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:36:57.588001+0000","last_undegraded":"2020-11-16T22:36:57.588001+0000","last_fullsized":"2020-11-16T22:36:57.588001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:57.587987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:57.587987+0000","last_clean_scrub_stamp":"2020-11-16T22:36:57.587987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.172068+0000","last_change":"2020-11-16T22:36:56.172274+0000","last_active":"2020-11-16T22:36:56.172068+0000","last_peered":"2020-11-16T22:36:56.172068+0000","last_clean":"2020-11-16T22:36:56.172068+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:36:56.172068+0000","last_undegraded":"2020-11-16T22:36:56.172068+0000","last_fullsized":"2020-11-16T22:36:56.172068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:56.172044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:56.172044+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.172044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"111'4","reported_seq":"101","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.732418+0000","last_change":"2020-11-16T22:36:53.732578+0000","last_active":"2020-11-16T22:36:53.732418+0000","last_peered":"2020-11-16T22:36:53.732418+0000","last_clean":"2020-11-16T22:36:53.732418+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:36:53.732418+0000","last_undegraded":"2020-11-16T22:36:53.732418+0000","last_fullsized":"2020-11-16T22:36:53.732418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'4","last_scrub_stamp":"2020-11-16T22:36:53.732406+0000","last_deep_scrub":"111'4","last_deep_scrub_stamp":"2020-11-16T22:36:53.732406+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.732406+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.191036+0000","last_change":"2020-11-16T22:36:53.191223+0000","last_active":"2020-11-16T22:36:53.191036+0000","last_peered":"2020-11-16T22:36:53.191036+0000","last_clean":"2020-11-16T22:36:53.191036+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:36:53.191036+0000","last_undegraded":"2020-11-16T22:36:53.191036+0000","last_fullsized":"2020-11-16T22:36:53.191036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:53.191027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:53.191027+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.191027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:59.172773+0000","last_change":"2020-11-16T22:36:59.172950+0000","last_active":"2020-11-16T22:36:59.172773+0000","last_peered":"2020-11-16T22:36:59.172773+0000","last_clean":"2020-11-16T22:36:59.172773+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:36:59.172773+0000","last_undegraded":"2020-11-16T22:36:59.172773+0000","last_fullsized":"2020-11-16T22:36:59.172773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:59.172759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:59.172759+0000","last_clean_scrub_stamp":"2020-11-16T22:36:59.172759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"91","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:52.810822+0000","last_change":"2020-11-16T22:36:52.811013+0000","last_active":"2020-11-16T22:36:52.810822+0000","last_peered":"2020-11-16T22:36:52.810822+0000","last_clean":"2020-11-16T22:36:52.810822+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:36:52.810822+0000","last_undegraded":"2020-11-16T22:36:52.810822+0000","last_fullsized":"2020-11-16T22:36:52.810822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:52.810808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:52.810808+0000","last_clean_scrub_stamp":"2020-11-16T22:36:52.810808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:00.202485+0000","last_change":"2020-11-16T22:37:00.202679+0000","last_active":"2020-11-16T22:37:00.202485+0000","last_peered":"2020-11-16T22:37:00.202485+0000","last_clean":"2020-11-16T22:37:00.202485+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:37:00.202485+0000","last_undegraded":"2020-11-16T22:37:00.202485+0000","last_fullsized":"2020-11-16T22:37:00.202485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:00.202471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:00.202471+0000","last_clean_scrub_stamp":"2020-11-16T22:37:00.202471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":8,"stat_sum":{"num_bytes":117916,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":257,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18628,"num_omap_keys":41,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":235832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":3},{"poolid":16,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7744,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364096,"statfs":{"total":96636764160,"available":96628834304,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.718,"15min":1.7569999999999999},"min":{"1min":1.115,"5min":0.98599999999999999,"15min":0.65100000000000002},"max":{"1min":2.7770000000000001,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.915,"5min":1.8009999999999999,"15min":1.792},"min":{"1min":1.262,"5min":1.1279999999999999,"15min":0.68899999999999995},"max":{"1min":3.0800000000000001,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":2.6859999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":2.0870000000000002,"15min":2.028},"min":{"1min":1.306,"5min":0.93000000000000005,"15min":0.92500000000000004},"max":{"1min":2.6909999999999998,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":2.298},{"interface":"front","average":{"1min":1.748,"5min":1.8029999999999999,"15min":1.7609999999999999},"min":{"1min":1.4279999999999999,"5min":1.1060000000000001,"15min":0.73099999999999998},"max":{"1min":2.4020000000000001,"5min":2.7749999999999999,"15min":2.883},"last":1.4279999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2400000000000002,"5min":2.1869999999999998,"15min":2.0470000000000002},"min":{"1min":1.5920000000000001,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":3.0129999999999999,"5min":3.0790000000000002,"15min":3.4780000000000002},"last":2.069},{"interface":"front","average":{"1min":2.2570000000000001,"5min":2.274,"15min":2.2890000000000001},"min":{"1min":1.3140000000000001,"5min":1.3,"15min":1.105},"max":{"1min":2.9950000000000001,"5min":3.98,"15min":3.98},"last":1.919}]},{"osd":3,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.6240000000000001,"15min":2.5710000000000002},"min":{"1min":2.0430000000000001,"5min":1.4910000000000001,"15min":1.2190000000000001},"max":{"1min":3.6059999999999999,"5min":3.6850000000000001,"15min":3.9969999999999999},"last":2.9580000000000002},{"interface":"front","average":{"1min":2.859,"5min":2.8290000000000002,"15min":2.8260000000000001},"min":{"1min":2.2509999999999999,"5min":1.6020000000000001,"15min":1.2689999999999999},"max":{"1min":3.3370000000000002,"5min":4.266,"15min":4.266},"last":2.907}]},{"osd":4,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.194,"5min":2.3250000000000002,"15min":2.23},"min":{"1min":1.673,"5min":1.631,"15min":1.018},"max":{"1min":3.46,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.788},{"interface":"front","average":{"1min":2.8849999999999998,"5min":2.802,"15min":2.778},"min":{"1min":2.0529999999999999,"5min":1.748,"15min":1.1479999999999999},"max":{"1min":3.5920000000000001,"5min":3.9729999999999999,"15min":3.9940000000000002},"last":2.5619999999999998}]},{"osd":5,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":2.9500000000000002,"15min":2.9489999999999998},"min":{"1min":2.5,"5min":1.9750000000000001,"15min":1.3540000000000001},"max":{"1min":3.7120000000000002,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":3.0960000000000001},{"interface":"front","average":{"1min":2.7850000000000001,"5min":2.8119999999999998,"15min":2.7240000000000002},"min":{"1min":2.2080000000000002,"5min":1.899,"15min":1.1299999999999999},"max":{"1min":3.9390000000000001,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":2.2080000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":2.907,"15min":2.871},"min":{"1min":2.3519999999999999,"5min":2.0409999999999999,"15min":1.2989999999999999},"max":{"1min":3.7250000000000001,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":2.6389999999999998},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.226,"15min":3.1739999999999999},"min":{"1min":2.5299999999999998,"5min":2.1629999999999998,"15min":1.222},"max":{"1min":3.9449999999999998,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":3.3500000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542272,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7572,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":6208,"kb_avail":94364268,"statfs":{"total":96636764160,"available":96629010432,"internally_reserved":0,"allocated":1388544,"data_stored":630469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6356992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.7250000000000001,"15min":1.6539999999999999},"min":{"1min":1.343,"5min":0.98699999999999999,"15min":0.97499999999999998},"max":{"1min":2.2330000000000001,"5min":3.206,"15min":3.206},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.774,"15min":1.75},"min":{"1min":1.1060000000000001,"5min":1.083,"15min":0.94099999999999995},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.1579999999999999},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.956,"5min":1.806,"15min":1.7070000000000001},"min":{"1min":1.431,"5min":1.1779999999999999,"15min":1.0469999999999999},"max":{"1min":2.8679999999999999,"5min":3.423,"15min":3.423},"last":1.7849999999999999},{"interface":"front","average":{"1min":2.3610000000000002,"5min":2.2549999999999999,"15min":2.1819999999999999},"min":{"1min":1.712,"5min":1.181,"15min":1.01},"max":{"1min":2.9580000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":2.028}]},{"osd":2,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":2.2869999999999999,"15min":2.23},"min":{"1min":1.7769999999999999,"5min":1.282,"15min":1.131},"max":{"1min":3.0720000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.6099999999999999},{"interface":"front","average":{"1min":2.165,"5min":2.117,"15min":2.0790000000000002},"min":{"1min":1.361,"5min":1.133,"15min":1.0700000000000001},"max":{"1min":2.8130000000000002,"5min":2.948,"15min":2.948},"last":1.361}]},{"osd":3,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":2.512,"15min":2.448},"min":{"1min":1.615,"5min":1.615,"15min":1.1850000000000001},"max":{"1min":3.1629999999999998,"5min":3.2650000000000001,"15min":3.3340000000000001},"last":1.615},{"interface":"front","average":{"1min":2.847,"5min":2.6880000000000002,"15min":2.5880000000000001},"min":{"1min":2.2930000000000001,"5min":1.671,"15min":1.377},"max":{"1min":3.5209999999999999,"5min":3.7130000000000001,"15min":3.7130000000000001},"last":2.375}]},{"osd":4,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.234,"5min":2.323,"15min":2.355},"min":{"1min":1.4750000000000001,"5min":1.3380000000000001,"15min":1.214},"max":{"1min":2.665,"5min":3.343,"15min":3.8570000000000002},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.7869999999999999,"5min":2.8199999999999998,"15min":2.8069999999999999},"min":{"1min":1.415,"5min":1.415,"15min":1.415},"max":{"1min":3.3650000000000002,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":1.415}]},{"osd":6,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.9390000000000001,"5min":2.8740000000000001,"15min":2.835},"min":{"1min":1.907,"5min":1.8109999999999999,"15min":1.6379999999999999},"max":{"1min":3.8820000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":2.669},{"interface":"front","average":{"1min":3.1230000000000002,"5min":3.181,"15min":3.137},"min":{"1min":2.319,"5min":2.319,"15min":1.8680000000000001},"max":{"1min":3.9119999999999999,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":2.319}]},{"osd":7,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8130000000000002,"5min":2.867,"15min":2.8809999999999998},"min":{"1min":1.206,"5min":1.206,"15min":1.206},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":1.206},{"interface":"front","average":{"1min":3.2770000000000001,"5min":3.2410000000000001,"15min":3.2029999999999998},"min":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":1.913},"max":{"1min":3.9700000000000002,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":2.0800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7780,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364060,"statfs":{"total":96636764160,"available":96628797440,"internally_reserved":0,"allocated":1404928,"data_stored":644510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.556,"15min":1.591},"min":{"1min":1.1259999999999999,"5min":0.95799999999999996,"15min":0.94899999999999995},"max":{"1min":1.792,"5min":2.871,"15min":2.9350000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.621,"15min":1.6419999999999999},"min":{"1min":1.1060000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.325}]},{"osd":1,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":1.97,"5min":1.9259999999999999,"15min":1.861},"min":{"1min":1.3799999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.6749999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.496},{"interface":"front","average":{"1min":2.472,"5min":2.4710000000000001,"15min":2.5049999999999999},"min":{"1min":2.0739999999999998,"5min":1.417,"15min":1.417},"max":{"1min":2.9369999999999998,"5min":3.875,"15min":3.875},"last":2.585}]},{"osd":2,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":2.1299999999999999,"15min":2.2029999999999998},"min":{"1min":1.3600000000000001,"5min":1.171,"15min":1.1259999999999999},"max":{"1min":3.0680000000000001,"5min":3.1309999999999998,"15min":3.4380000000000002},"last":2.7109999999999999},{"interface":"front","average":{"1min":2.2410000000000001,"5min":2.3050000000000002,"15min":2.3370000000000002},"min":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.258},"max":{"1min":2.883,"5min":3.1739999999999999,"15min":3.4860000000000002},"last":2.0230000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.1989999999999998,"5min":2.1909999999999998,"15min":2.2490000000000001},"min":{"1min":1.718,"5min":1.2949999999999999,"15min":1.2949999999999999},"max":{"1min":2.8820000000000001,"5min":3.2170000000000001,"15min":5.3449999999999998},"last":2.1749999999999998},{"interface":"front","average":{"1min":2.593,"5min":2.6349999999999998,"15min":2.726},"min":{"1min":1.7849999999999999,"5min":1.728,"15min":1.5009999999999999},"max":{"1min":3.4529999999999998,"5min":3.7320000000000002,"15min":4.0830000000000002},"last":3.0139999999999998}]},{"osd":4,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.6899999999999999,"5min":2.5939999999999999,"15min":2.6440000000000001},"min":{"1min":2,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.3849999999999998,"5min":3.782,"15min":5.6399999999999997},"last":2.9580000000000002},{"interface":"front","average":{"1min":2.714,"5min":2.6619999999999999,"15min":2.7869999999999999},"min":{"1min":2.2330000000000001,"5min":1.171,"15min":1.171},"max":{"1min":3.117,"5min":3.6629999999999998,"15min":4.766},"last":3.4279999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":2.4710000000000001,"5min":2.5009999999999999,"15min":2.5190000000000001},"min":{"1min":1.8899999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"max":{"1min":3.234,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":2.4089999999999998},{"interface":"front","average":{"1min":3.0600000000000001,"5min":2.9950000000000001,"15min":3.0750000000000002},"min":{"1min":2.548,"5min":1.516,"15min":1.516},"max":{"1min":3.4020000000000001,"5min":4.4729999999999999,"15min":6.0380000000000003},"last":3.1619999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:36:55 2020","interfaces":[{"interface":"back","average":{"1min":3.1160000000000001,"5min":3.0649999999999999,"15min":3.1949999999999998},"min":{"1min":2.4910000000000001,"5min":1.6379999999999999,"15min":1.6379999999999999},"max":{"1min":3.6560000000000001,"5min":4.0700000000000003,"15min":5.3220000000000001},"last":3.2389999999999999},{"interface":"front","average":{"1min":3.0099999999999998,"5min":3.0369999999999999,"15min":3.0699999999999998},"min":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"max":{"1min":3.4609999999999999,"5min":4.806,"15min":5.2569999999999997},"last":3.069}]}]},{"osd":4,"up_from":13,"seq":55834574977,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8448,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":6976,"kb_avail":94363392,"statfs":{"total":96636764160,"available":96628113408,"internally_reserved":0,"allocated":1499136,"data_stored":731943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7143424},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":1.7390000000000001,"15min":1.8129999999999999},"min":{"1min":1.224,"5min":0.96799999999999997,"15min":0.65200000000000002},"max":{"1min":2.5539999999999998,"5min":3.02,"15min":3.02},"last":2.5609999999999999},{"interface":"front","average":{"1min":2.1880000000000002,"5min":2.1669999999999998,"15min":2.1819999999999999},"min":{"1min":1.462,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.8109999999999999,"5min":2.956,"15min":3.7170000000000001},"last":2.359}]},{"osd":1,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.851,"5min":1.8089999999999999,"15min":1.7509999999999999},"min":{"1min":1.3360000000000001,"5min":1.1970000000000001,"15min":0.74199999999999999},"max":{"1min":2.8959999999999999,"5min":3.302,"15min":3.302},"last":2.0219999999999998},{"interface":"front","average":{"1min":1.9790000000000001,"5min":2.0190000000000001,"15min":1.9119999999999999},"min":{"1min":1.3620000000000001,"5min":1.1180000000000001,"15min":1.0660000000000001},"max":{"1min":2.8359999999999999,"5min":3.1869999999999998,"15min":3.3620000000000001},"last":1.4079999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":1.9019999999999999,"15min":1.919},"min":{"1min":1.345,"5min":1.294,"15min":1.054},"max":{"1min":3.5739999999999998,"5min":4.117,"15min":4.117},"last":1.55},{"interface":"front","average":{"1min":2.1099999999999999,"5min":2.2029999999999998,"15min":2.1190000000000002},"min":{"1min":1.222,"5min":1.198,"15min":1.0129999999999999},"max":{"1min":3.3130000000000002,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.7770000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":2.3780000000000001,"15min":2.278},"min":{"1min":1.282,"5min":1.206,"15min":0.94599999999999995},"max":{"1min":3.1520000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":2.1899999999999999},{"interface":"front","average":{"1min":2.7360000000000002,"5min":2.7189999999999999,"15min":2.746},"min":{"1min":1.903,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.2200000000000002,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":3.0430000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5009999999999999,"5min":2.4180000000000001,"15min":2.448},"min":{"1min":1.613,"5min":1.361,"15min":1.075},"max":{"1min":3.3159999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":2.423},{"interface":"front","average":{"1min":2.895,"5min":2.9199999999999999,"15min":2.9079999999999999},"min":{"1min":2.4540000000000002,"5min":1.6890000000000001,"15min":1.1160000000000001},"max":{"1min":3.4079999999999999,"5min":4.9859999999999998,"15min":4.9859999999999998},"last":3.2829999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.8149999999999999,"5min":2.8330000000000002,"15min":2.8370000000000002},"min":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.5529999999999999},"max":{"1min":3.859,"5min":4.3739999999999997,"15min":4.5220000000000002},"last":2.9300000000000002},{"interface":"front","average":{"1min":2.8570000000000002,"5min":2.8439999999999999,"15min":2.8199999999999998},"min":{"1min":1.899,"5min":1.8959999999999999,"15min":1.6359999999999999},"max":{"1min":4.181,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":2.6360000000000001}]},{"osd":7,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0600000000000001,"5min":3.1200000000000001,"15min":3.1059999999999999},"min":{"1min":2.254,"5min":2.242,"15min":1.857},"max":{"1min":3.931,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":2.8420000000000001},{"interface":"front","average":{"1min":2.9830000000000001,"5min":2.9809999999999999,"15min":2.9670000000000001},"min":{"1min":2.3980000000000001,"5min":1.8440000000000001,"15min":1.496},"max":{"1min":3.7970000000000002,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":3.1499999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542272,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7824,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94364016,"statfs":{"total":96636764160,"available":96628752384,"internally_reserved":0,"allocated":1384448,"data_stored":630427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.655,"15min":1.6359999999999999},"min":{"1min":1.347,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3260000000000001,"5min":2.6469999999999998,"15min":2.8159999999999998},"last":1.468},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7190000000000001,"15min":1.698},"min":{"1min":1.2070000000000001,"5min":1.0449999999999999,"15min":1.02},"max":{"1min":2.3940000000000001,"5min":2.653,"15min":2.653},"last":1.577}]},{"osd":2,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.671,"15min":1.657},"min":{"1min":1.2609999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.496,"5min":2.7069999999999999,"15min":2.7869999999999999},"last":1.528},{"interface":"front","average":{"1min":1.9950000000000001,"5min":2.0950000000000002,"15min":2.0899999999999999},"min":{"1min":1.262,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.653,"5min":2.9119999999999999,"15min":3.4119999999999999},"last":2.1800000000000002}]},{"osd":3,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.0230000000000001,"5min":2.125,"15min":2.0449999999999999},"min":{"1min":1.5,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2559999999999998},"last":3.2229999999999999},{"interface":"front","average":{"1min":2.2709999999999999,"5min":2.3450000000000002,"15min":2.2919999999999998},"min":{"1min":1.411,"5min":1.056,"15min":0.99399999999999999},"max":{"1min":3.044,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":3.044}]},{"osd":4,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.452,"5min":2.54,"15min":2.669},"min":{"1min":1.897,"5min":1.5129999999999999,"15min":1.5029999999999999},"max":{"1min":2.8690000000000002,"5min":3.8559999999999999,"15min":4.8479999999999999},"last":2.3959999999999999},{"interface":"front","average":{"1min":2.2530000000000001,"5min":2.391,"15min":2.4089999999999998},"min":{"1min":1.6519999999999999,"5min":1.5049999999999999,"15min":1.1930000000000001},"max":{"1min":3.419,"5min":3.956,"15min":4.0380000000000003},"last":1.9379999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.798,"5min":2.891,"15min":2.9430000000000001},"min":{"1min":2.0710000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"max":{"1min":3.3490000000000002,"5min":3.8700000000000001,"15min":5.0659999999999998},"last":2.8050000000000002},{"interface":"front","average":{"1min":2.706,"5min":2.7080000000000002,"15min":2.669},"min":{"1min":2.133,"5min":1.6779999999999999,"15min":1.6180000000000001},"max":{"1min":3.5129999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":2.133}]},{"osd":6,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.7610000000000001,"5min":2.8610000000000002,"15min":2.8029999999999999},"min":{"1min":1.9510000000000001,"5min":1.772,"15min":1.772},"max":{"1min":3.4630000000000001,"5min":4.2599999999999998,"15min":4.2599999999999998},"last":2.6949999999999998},{"interface":"front","average":{"1min":3.0129999999999999,"5min":3.1240000000000001,"15min":3.1859999999999999},"min":{"1min":2.2930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"max":{"1min":3.6739999999999999,"5min":4.3490000000000002,"15min":5.2329999999999997},"last":3.1429999999999998}]},{"osd":7,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":3.1110000000000002,"5min":3.2799999999999998,"15min":3.3380000000000001},"min":{"1min":2.5009999999999999,"5min":2.1320000000000001,"15min":2.0489999999999999},"max":{"1min":3.9369999999999998,"5min":4.5910000000000002,"15min":5.6379999999999999},"last":3.4220000000000002},{"interface":"front","average":{"1min":2.9710000000000001,"5min":2.9870000000000001,"15min":2.9449999999999998},"min":{"1min":2.4500000000000002,"5min":2.036,"15min":1.895},"max":{"1min":3.4249999999999998,"5min":4.6059999999999999,"15min":4.6059999999999999},"last":2.7679999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542272,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7796,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94364044,"statfs":{"total":96636764160,"available":96628781056,"internally_reserved":0,"allocated":1486848,"data_stored":725455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.665,"15min":1.6359999999999999},"min":{"1min":1.4059999999999999,"5min":0.97299999999999998,"15min":0.70199999999999996},"max":{"1min":2.0550000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.611},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.736,"15min":1.6890000000000001},"min":{"1min":1.3799999999999999,"5min":1.119,"15min":0.83699999999999997},"max":{"1min":2.5680000000000001,"5min":2.6829999999999998,"15min":2.718},"last":2.1680000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.8600000000000001,"5min":1.8160000000000001,"15min":1.7290000000000001},"min":{"1min":1.524,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.6659999999999999,"5min":10.023,"15min":10.023},"last":1.7370000000000001},{"interface":"front","average":{"1min":2.1230000000000002,"5min":2.2269999999999999,"15min":2.1040000000000001},"min":{"1min":1.633,"5min":0.95499999999999996,"15min":0.80400000000000005},"max":{"1min":2.492,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":2.0790000000000002}]},{"osd":3,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.1789999999999998,"5min":2.069,"15min":2.1309999999999998},"min":{"1min":1.605,"5min":1.218,"15min":0.81000000000000005},"max":{"1min":2.6539999999999999,"5min":2.9159999999999999,"15min":3.3319999999999999},"last":1.665},{"interface":"front","average":{"1min":2.274,"5min":2.2719999999999998,"15min":2.2160000000000002},"min":{"1min":1.671,"5min":1.1990000000000001,"15min":0.85199999999999998},"max":{"1min":3.0720000000000001,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":2.2269999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.3599999999999999,"5min":2.4060000000000001,"15min":2.343},"min":{"1min":1.8220000000000001,"5min":1.516,"15min":1.008},"max":{"1min":3.2189999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":2.516},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.6960000000000002,"15min":2.6859999999999999},"min":{"1min":2.0680000000000001,"5min":1.022,"15min":1.022},"max":{"1min":3.847,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":3.1000000000000001}]},{"osd":5,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.9820000000000002,"5min":2.9710000000000001,"15min":3.0110000000000001},"min":{"1min":2.274,"5min":1.276,"15min":1.276},"max":{"1min":3.6989999999999998,"5min":5.351,"15min":5.351},"last":3.5219999999999998},{"interface":"front","average":{"1min":2.8620000000000001,"5min":2.758,"15min":2.802},"min":{"1min":2.1389999999999998,"5min":1.4279999999999999,"15min":1.3839999999999999},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":3.1800000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3.101,"5min":3.0169999999999999,"15min":3.0070000000000001},"min":{"1min":2.3919999999999999,"5min":1.8480000000000001,"15min":1.5269999999999999},"max":{"1min":4.3559999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":2.7890000000000001},{"interface":"front","average":{"1min":2.9769999999999999,"5min":2.9399999999999999,"15min":2.875},"min":{"1min":2.3149999999999999,"5min":1.861,"15min":1.4670000000000001},"max":{"1min":3.6299999999999999,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":2.847}]},{"osd":7,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3.1720000000000002,"5min":3.121,"15min":3.1040000000000001},"min":{"1min":2.5249999999999999,"5min":2.008,"15min":1.7010000000000001},"max":{"1min":3.9049999999999998,"5min":4.1239999999999997,"15min":4.7430000000000003},"last":2.7160000000000002},{"interface":"front","average":{"1min":3.319,"5min":3.2000000000000002,"15min":3.2170000000000001},"min":{"1min":2.3140000000000001,"5min":2.302,"15min":1.712},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":3.6160000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542273,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6739999999999999,"15min":1.653},"min":{"1min":1.012,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.202,"5min":2.6179999999999999,"15min":3.0710000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.752,"15min":1.7849999999999999},"min":{"1min":1.0700000000000001,"5min":1.032,"15min":0.81899999999999995},"max":{"1min":2.246,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":1.5549999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.925,"5min":1.905,"15min":1.849},"min":{"1min":1.26,"5min":1.0069999999999999,"15min":0.91500000000000004},"max":{"1min":2.4889999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.778},{"interface":"front","average":{"1min":1.9830000000000001,"5min":1.9870000000000001,"15min":1.95},"min":{"1min":1.032,"5min":0.81499999999999995,"15min":0.80500000000000005},"max":{"1min":3.0110000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":2.641}]},{"osd":3,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.909,"15min":1.867},"min":{"1min":1.1080000000000001,"5min":1.0840000000000001,"15min":0.92900000000000005},"max":{"1min":2.887,"5min":3.0840000000000001,"15min":3.9660000000000002},"last":2.23},{"interface":"front","average":{"1min":2.6379999999999999,"5min":2.6000000000000001,"15min":2.4830000000000001},"min":{"1min":2.2320000000000002,"5min":1.5860000000000001,"15min":1.1499999999999999},"max":{"1min":3.3090000000000002,"5min":3.944,"15min":3.944},"last":2.2730000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":2.6219999999999999,"15min":2.5230000000000001},"min":{"1min":1.962,"5min":1.5800000000000001,"15min":1.498},"max":{"1min":3.464,"5min":5.0209999999999999,"15min":5.0209999999999999},"last":1.962},{"interface":"front","average":{"1min":2.569,"5min":2.552,"15min":2.5710000000000002},"min":{"1min":1.4810000000000001,"5min":1.452,"15min":1.3260000000000001},"max":{"1min":4.0830000000000002,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":2.1339999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.8180000000000001,"5min":2.7959999999999998,"15min":2.8039999999999998},"min":{"1min":1.7390000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"max":{"1min":3.5,"5min":3.738,"15min":4.0259999999999998},"last":2.4209999999999998},{"interface":"front","average":{"1min":2.444,"5min":2.4249999999999998,"15min":2.4529999999999998},"min":{"1min":2.032,"5min":1.639,"15min":1.363},"max":{"1min":3.5640000000000001,"5min":3.7970000000000002,"15min":4.3070000000000004},"last":2.6949999999999998}]},{"osd":6,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":2.6760000000000002,"15min":2.681},"min":{"1min":1.7769999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.165},"last":2.8969999999999998},{"interface":"front","average":{"1min":3.1120000000000001,"5min":3.0489999999999999,"15min":2.9660000000000002},"min":{"1min":2.4700000000000002,"5min":2.0550000000000002,"15min":1.9930000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":2.556}]},{"osd":7,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":3.085,"5min":3.069,"15min":3.081},"min":{"1min":2.1619999999999999,"5min":1.903,"15min":1.8879999999999999},"max":{"1min":3.8940000000000001,"5min":4.0599999999999996,"15min":4.1210000000000004},"last":2.8540000000000001},{"interface":"front","average":{"1min":2.847,"5min":2.9359999999999999,"15min":2.996},"min":{"1min":1.919,"5min":1.488,"15min":1.488},"max":{"1min":3.7549999999999999,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":2.484}]}]},{"osd":3,"up_from":14,"seq":60129542271,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8316,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94363524,"statfs":{"total":96636764160,"available":96628248576,"internally_reserved":0,"allocated":1429504,"data_stored":651180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6799999999999999,"15min":1.639},"min":{"1min":1.133,"5min":1.0820000000000001,"15min":0.73599999999999999},"max":{"1min":2.1549999999999998,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":2.4620000000000002},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.786,"15min":1.7729999999999999},"min":{"1min":0.79700000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2989999999999999,"5min":2.8780000000000001,"15min":3.6739999999999999},"last":1.972}]},{"osd":1,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6759999999999999,"15min":1.6419999999999999},"min":{"1min":1.2849999999999999,"5min":0.872,"15min":0.81100000000000005},"max":{"1min":2.2229999999999999,"5min":2.6880000000000002,"15min":2.77},"last":2.0249999999999999},{"interface":"front","average":{"1min":2.0129999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":1.3400000000000001,"5min":1.25,"15min":1.099},"max":{"1min":2.645,"5min":5.258,"15min":5.258},"last":2.0790000000000002}]},{"osd":2,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":2.1829999999999998,"15min":2.1749999999999998},"min":{"1min":1.401,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.6099999999999999,"5min":3.214,"15min":3.597},"last":2.5739999999999998},{"interface":"front","average":{"1min":2.3100000000000001,"5min":2.403,"15min":2.3759999999999999},"min":{"1min":1.5029999999999999,"5min":1.3700000000000001,"15min":1.075},"max":{"1min":2.907,"5min":4.0289999999999999,"15min":4.0990000000000002},"last":2.9830000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.5739999999999998,"15min":2.5379999999999998},"min":{"1min":1.589,"5min":1.298,"15min":1.2150000000000001},"max":{"1min":3.419,"5min":3.738,"15min":3.7450000000000001},"last":2.948},{"interface":"front","average":{"1min":2.3780000000000001,"5min":2.4020000000000001,"15min":2.4009999999999998},"min":{"1min":1.496,"5min":1.403,"15min":1.1699999999999999},"max":{"1min":3.2170000000000001,"5min":3.681,"15min":3.6829999999999998},"last":3.177}]},{"osd":5,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.6560000000000001,"5min":2.7519999999999998,"15min":2.7469999999999999},"min":{"1min":1.6559999999999999,"5min":1.5600000000000001,"15min":1.2709999999999999},"max":{"1min":3.5059999999999998,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":3.6629999999999998},{"interface":"front","average":{"1min":2.9260000000000002,"5min":2.968,"15min":2.9950000000000001},"min":{"1min":2.2040000000000002,"5min":1.857,"15min":1.8360000000000001},"max":{"1min":3.8319999999999999,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":3.891}]},{"osd":6,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":3.077,"15min":3.097},"min":{"1min":1.8799999999999999,"5min":1.615,"15min":1.615},"max":{"1min":3.6360000000000001,"5min":4.3970000000000002,"15min":4.5359999999999996},"last":4.4189999999999996},{"interface":"front","average":{"1min":2.891,"5min":3.0590000000000002,"15min":3.0750000000000002},"min":{"1min":2.1850000000000001,"5min":2.04,"15min":1.7310000000000001},"max":{"1min":3.5449999999999999,"5min":4.1959999999999997,"15min":4.5590000000000002},"last":3.1379999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3,"5min":3.2109999999999999,"15min":3.1600000000000001},"min":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.8240000000000001},"max":{"1min":3.722,"5min":4.343,"15min":4.6470000000000002},"last":3.3220000000000001},{"interface":"front","average":{"1min":2.7509999999999999,"5min":2.9239999999999999,"15min":2.9289999999999998},"min":{"1min":1.633,"5min":1.633,"15min":1.633},"max":{"1min":3.444,"5min":3.98,"15min":3.98},"last":3.4409999999999998}]}]}]}} 2020-11-16T22:37:19.926 INFO:tasks.ceph:pgid 16.0 last_scrub_stamp 2020-11-16T22:35:22.171935+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=35, tm_sec=22, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=48, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:37:19.926 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-16T22:37:35.009 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:02.963 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:38:02.970 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:02.973 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:03.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 Processor -- start 2020-11-16T22:38:03.094 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 -- start start 2020-11-16T22:38:03.095 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 0x7f1eb00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1eb00cf6b0 0x7f1eb015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 0x7f1eb00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb00c8910 con 0x7f1eb00b2900 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb00b3460 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.101+0000 7f1eaf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b31d0 con 0x7f1eb015d2e0 2020-11-16T22:38:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 0x7f1eb00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1eb00cf6b0 0x7f1eb015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaed9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 0x7f1eb00c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46378/0 (socket says 172.21.15.181:46378) 2020-11-16T22:38:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaed9d700 1 -- 172.21.15.181:0/1885722007 learned_addr learned my addr 172.21.15.181:0/1885722007 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:38:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eae59c700 1 --2- 172.21.15.181:0/1885722007 >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 0x7f1eb00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 -- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 msgr2=0x7f1eb00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:38:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 0x7f1eb00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 -- 172.21.15.181:0/1885722007 >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 msgr2=0x7f1eb00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/1885722007 >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 0x7f1eb00b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 -- 172.21.15.181:0/1885722007 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00c8130 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eae59c700 1 --2- 172.21.15.181:0/1885722007 >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 0x7f1eb00b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb015cdf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f1eb0061220 tx=0x7f1e98006df0).ready entity=mon.1 client_cookie=9bd1b00557b21d77 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1ead59a700 1 -- 172.21.15.181:0/1885722007 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980097b0 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1ead59a700 1 -- 172.21.15.181:0/1885722007 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98009910 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1ead59a700 1 -- 172.21.15.181:0/1885722007 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98009c90 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 -- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 msgr2=0x7f1eb015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb015cdf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f1eb0061220 tx=0x7f1e98006df0).stop 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 -- 172.21.15.181:0/1885722007 shutdown_connections 2020-11-16T22:38:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb015d2e0 0x7f1eb00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/1885722007 >> v2:172.21.15.175:3300/0 conn(0x7f1eb00b2900 0x7f1eb00b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/1885722007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb015cdf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 -- 172.21.15.181:0/1885722007 >> 172.21.15.181:0/1885722007 conn(0x7f1eb00d2dc0 msgr2=0x7f1eb00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:38:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 -- 172.21.15.181:0/1885722007 shutdown_connections 2020-11-16T22:38:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.105+0000 7f1eaf59e700 1 -- 172.21.15.181:0/1885722007 wait complete. 2020-11-16T22:38:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 Processor -- start 2020-11-16T22:38:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- start start 2020-11-16T22:38:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb015d2e0 0x7f1eb0062df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b5280 con 0x7f1eb015d2e0 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00619f0 con 0x7f1eb00cf6b0 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0061c30 con 0x7f1eb00b2900 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46382/0 (socket says 172.21.15.181:46382) 2020-11-16T22:38:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 -- 172.21.15.181:0/816607388 learned_addr learned my addr 172.21.15.181:0/816607388 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaed9d700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb015d2e0 0x7f1eb0062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 -- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 msgr2=0x7f1eb014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 -- 172.21.15.181:0/816607388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb015d2e0 msgr2=0x7f1eb0062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb015d2e0 0x7f1eb0062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 -- 172.21.15.181:0/816607388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0061ed0 con 0x7f1eb00b2900 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eae59c700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f1ea000db90 tx=0x7f1ea0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0009820 con 0x7f1eb00b2900 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea0009980 con 0x7f1eb00b2900 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0009d40 con 0x7f1eb00b2900 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb0062170 con 0x7f1eb00b2900 2020-11-16T22:38:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.109+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb00b54b0 con 0x7f1eb00b2900 2020-11-16T22:38:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb0044f60 con 0x7f1eb00b2900 2020-11-16T22:38:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1ea0013690 con 0x7f1eb00b2900 2020-11-16T22:38:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1ea67fc700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 0x7f1e8c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ea006dd90 con 0x7f1eb00b2900 2020-11-16T22:38:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 0x7f1e8c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.113+0000 7f1eadd9b700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 0x7f1e8c057520 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e98006e30 tx=0x7f1e98015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.117+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1ea0043e80 con 0x7f1eb00b2900 2020-11-16T22:38:03.262 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.269+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 --> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1eb014f8a0 con 0x7f1e8c055060 2020-11-16T22:38:03.272 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1ea67fc700 1 -- 172.21.15.181:0/816607388 <== mgr.4111 v2:172.21.15.181:6832/15040 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+104319 (secure 0 0 0) 0x7f1eb014f8a0 con 0x7f1e8c055060 2020-11-16T22:38:03.273 INFO:teuthology.orchestra.run.smithi181.stdout: 2020-11-16T22:38:03.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 msgr2=0x7f1e8c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:03.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 0x7f1e8c057520 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e98006e30 tx=0x7f1e98015000).stop 2020-11-16T22:38:03.274 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 msgr2=0x7f1eb0061500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f1ea000db90 tx=0x7f1ea0008040).stop 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 shutdown_connections 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7f1e8c055060 0x7f1e8c057520 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb015d2e0 0x7f1eb0062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1eb00b2900 0x7f1eb0061500 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 --2- 172.21.15.181:0/816607388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb00cf6b0 0x7f1eb014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:03.275 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 >> 172.21.15.181:0/816607388 conn(0x7f1eb00d2dc0 msgr2=0x7f1eb00b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:38:03.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 shutdown_connections 2020-11-16T22:38:03.276 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:03.281+0000 7f1eaf59e700 1 -- 172.21.15.181:0/816607388 wait complete. 2020-11-16T22:38:03.276 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2020-11-16T22:38:03.289 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2020-11-16T22:38:02.885893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117916,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":259,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19808,"num_omap_keys":43,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":63480,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":52352,"kb_avail":754911240,"statfs":{"total":773094113280,"available":773029109760,"internally_reserved":0,"allocated":11329536,"data_stored":5273368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53608448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004056"},"pg_stats":[{"pgid":"16.7","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:06.868904+0000","last_change":"2020-11-16T22:37:06.869075+0000","last_active":"2020-11-16T22:37:06.868904+0000","last_peered":"2020-11-16T22:37:06.868904+0000","last_clean":"2020-11-16T22:37:06.868904+0000","last_became_active":"2020-11-16T22:35:22.592949+0000","last_became_peered":"2020-11-16T22:35:22.592949+0000","last_unstale":"2020-11-16T22:37:06.868904+0000","last_undegraded":"2020-11-16T22:37:06.868904+0000","last_fullsized":"2020-11-16T22:37:06.868904+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:06.868894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:06.868894+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.868894+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:07.089435+0000","last_change":"2020-11-16T22:37:07.089634+0000","last_active":"2020-11-16T22:37:07.089435+0000","last_peered":"2020-11-16T22:37:07.089435+0000","last_clean":"2020-11-16T22:37:07.089435+0000","last_became_active":"2020-11-16T22:35:22.590435+0000","last_became_peered":"2020-11-16T22:35:22.590435+0000","last_unstale":"2020-11-16T22:37:07.089435+0000","last_undegraded":"2020-11-16T22:37:07.089435+0000","last_fullsized":"2020-11-16T22:37:07.089435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:07.089425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:07.089425+0000","last_clean_scrub_stamp":"2020-11-16T22:37:07.089425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:05.882323+0000","last_change":"2020-11-16T22:37:05.882556+0000","last_active":"2020-11-16T22:37:05.882323+0000","last_peered":"2020-11-16T22:37:05.882323+0000","last_clean":"2020-11-16T22:37:05.882323+0000","last_became_active":"2020-11-16T22:35:22.586786+0000","last_became_peered":"2020-11-16T22:35:22.586786+0000","last_unstale":"2020-11-16T22:37:05.882323+0000","last_undegraded":"2020-11-16T22:37:05.882323+0000","last_fullsized":"2020-11-16T22:37:05.882323+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:05.882309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:05.882309+0000","last_clean_scrub_stamp":"2020-11-16T22:37:05.882309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:02.933270+0000","last_change":"2020-11-16T22:37:02.933532+0000","last_active":"2020-11-16T22:37:02.933270+0000","last_peered":"2020-11-16T22:37:02.933270+0000","last_clean":"2020-11-16T22:37:02.933270+0000","last_became_active":"2020-11-16T22:35:22.586336+0000","last_became_peered":"2020-11-16T22:35:22.586336+0000","last_unstale":"2020-11-16T22:37:02.933270+0000","last_undegraded":"2020-11-16T22:37:02.933270+0000","last_fullsized":"2020-11-16T22:37:02.933270+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:02.933260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:02.933260+0000","last_clean_scrub_stamp":"2020-11-16T22:37:02.933260+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:04.166552+0000","last_change":"2020-11-16T22:37:04.166741+0000","last_active":"2020-11-16T22:37:04.166552+0000","last_peered":"2020-11-16T22:37:04.166552+0000","last_clean":"2020-11-16T22:37:04.166552+0000","last_became_active":"2020-11-16T22:35:22.602895+0000","last_became_peered":"2020-11-16T22:35:22.602895+0000","last_unstale":"2020-11-16T22:37:04.166552+0000","last_undegraded":"2020-11-16T22:37:04.166552+0000","last_fullsized":"2020-11-16T22:37:04.166552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:04.166535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:04.166535+0000","last_clean_scrub_stamp":"2020-11-16T22:37:04.166535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.642505+0000","last_change":"2020-11-16T22:36:53.642671+0000","last_active":"2020-11-16T22:36:53.642505+0000","last_peered":"2020-11-16T22:36:53.642505+0000","last_clean":"2020-11-16T22:36:53.642505+0000","last_became_active":"2020-11-16T22:35:22.593172+0000","last_became_peered":"2020-11-16T22:35:22.593172+0000","last_unstale":"2020-11-16T22:36:53.642505+0000","last_undegraded":"2020-11-16T22:36:53.642505+0000","last_fullsized":"2020-11-16T22:36:53.642505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:53.642496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:53.642496+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.642496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:55.933703+0000","last_change":"2020-11-16T22:36:55.933898+0000","last_active":"2020-11-16T22:36:55.933703+0000","last_peered":"2020-11-16T22:36:55.933703+0000","last_clean":"2020-11-16T22:36:55.933703+0000","last_became_active":"2020-11-16T22:35:22.588015+0000","last_became_peered":"2020-11-16T22:35:22.588015+0000","last_unstale":"2020-11-16T22:36:55.933703+0000","last_undegraded":"2020-11-16T22:36:55.933703+0000","last_fullsized":"2020-11-16T22:36:55.933703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:55.933690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:55.933690+0000","last_clean_scrub_stamp":"2020-11-16T22:36:55.933690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":"12","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:14.915826+0000","last_change":"2020-11-16T22:37:14.916049+0000","last_active":"2020-11-16T22:37:14.915826+0000","last_peered":"2020-11-16T22:37:14.915826+0000","last_clean":"2020-11-16T22:37:14.915826+0000","last_became_active":"2020-11-16T22:35:22.601487+0000","last_became_peered":"2020-11-16T22:35:22.601487+0000","last_unstale":"2020-11-16T22:37:14.915826+0000","last_undegraded":"2020-11-16T22:37:14.915826+0000","last_fullsized":"2020-11-16T22:37:14.915826+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:14.915812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:14.915812+0000","last_clean_scrub_stamp":"2020-11-16T22:37:14.915812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"111'38","reported_seq":"52","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:06.957094+0000","last_change":"2020-11-16T22:37:06.957207+0000","last_active":"2020-11-16T22:37:06.957094+0000","last_peered":"2020-11-16T22:37:06.957094+0000","last_clean":"2020-11-16T22:37:06.957094+0000","last_became_active":"2020-11-16T22:35:22.191742+0000","last_became_peered":"2020-11-16T22:35:22.191742+0000","last_unstale":"2020-11-16T22:37:06.957094+0000","last_undegraded":"2020-11-16T22:37:06.957094+0000","last_fullsized":"2020-11-16T22:37:06.957094+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'38","last_scrub_stamp":"2020-11-16T22:37:06.957085+0000","last_deep_scrub":"111'38","last_deep_scrub_stamp":"2020-11-16T22:37:06.957085+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.957085+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":942,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.6","version":"111'17","reported_seq":"31","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:03.862617+0000","last_change":"2020-11-16T22:37:03.862803+0000","last_active":"2020-11-16T22:37:03.862617+0000","last_peered":"2020-11-16T22:37:03.862617+0000","last_clean":"2020-11-16T22:37:03.862617+0000","last_became_active":"2020-11-16T22:35:22.195870+0000","last_became_peered":"2020-11-16T22:35:22.195870+0000","last_unstale":"2020-11-16T22:37:03.862617+0000","last_undegraded":"2020-11-16T22:37:03.862617+0000","last_fullsized":"2020-11-16T22:37:03.862617+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'17","last_scrub_stamp":"2020-11-16T22:37:03.862600+0000","last_deep_scrub":"111'17","last_deep_scrub_stamp":"2020-11-16T22:37:03.862600+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.862600+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"111'29","reported_seq":"44","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:03.213006+0000","last_change":"2020-11-16T22:37:03.213129+0000","last_active":"2020-11-16T22:37:03.213006+0000","last_peered":"2020-11-16T22:37:03.213006+0000","last_clean":"2020-11-16T22:37:03.213006+0000","last_became_active":"2020-11-16T22:35:21.257651+0000","last_became_peered":"2020-11-16T22:35:21.257651+0000","last_unstale":"2020-11-16T22:37:03.213006+0000","last_undegraded":"2020-11-16T22:37:03.213006+0000","last_fullsized":"2020-11-16T22:37:03.213006+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'29","last_scrub_stamp":"2020-11-16T22:37:03.212996+0000","last_deep_scrub":"111'29","last_deep_scrub_stamp":"2020-11-16T22:37:03.212996+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.212996+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5652,"num_omap_keys":12,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"111'61","reported_seq":"77","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.903709+0000","last_change":"2020-11-16T22:36:56.903915+0000","last_active":"2020-11-16T22:36:56.903709+0000","last_peered":"2020-11-16T22:36:56.903709+0000","last_clean":"2020-11-16T22:36:56.903709+0000","last_became_active":"2020-11-16T22:35:21.130104+0000","last_became_peered":"2020-11-16T22:35:21.130104+0000","last_unstale":"2020-11-16T22:36:56.903709+0000","last_undegraded":"2020-11-16T22:36:56.903709+0000","last_fullsized":"2020-11-16T22:36:56.903709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'61","last_scrub_stamp":"2020-11-16T22:36:56.903691+0000","last_deep_scrub":"111'61","last_deep_scrub_stamp":"2020-11-16T22:36:56.903691+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.903691+0000","log_size":61,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":81610,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"111'36","reported_seq":"51","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:01.232357+0000","last_change":"2020-11-16T22:37:01.232533+0000","last_active":"2020-11-16T22:37:01.232357+0000","last_peered":"2020-11-16T22:37:01.232357+0000","last_clean":"2020-11-16T22:37:01.232357+0000","last_became_active":"2020-11-16T22:35:21.177888+0000","last_became_peered":"2020-11-16T22:35:21.177888+0000","last_unstale":"2020-11-16T22:37:01.232357+0000","last_undegraded":"2020-11-16T22:37:01.232357+0000","last_fullsized":"2020-11-16T22:37:01.232357+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'36","last_scrub_stamp":"2020-11-16T22:37:01.232343+0000","last_deep_scrub":"111'36","last_deep_scrub_stamp":"2020-11-16T22:37:01.232343+0000","last_clean_scrub_stamp":"2020-11-16T22:37:01.232343+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20641,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9928,"num_omap_keys":22,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"111'23","reported_seq":"37","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:00.782034+0000","last_change":"2020-11-16T22:37:00.782199+0000","last_active":"2020-11-16T22:37:00.782034+0000","last_peered":"2020-11-16T22:37:00.782034+0000","last_clean":"2020-11-16T22:37:00.782034+0000","last_became_active":"2020-11-16T22:35:22.195927+0000","last_became_peered":"2020-11-16T22:35:22.195927+0000","last_unstale":"2020-11-16T22:37:00.782034+0000","last_undegraded":"2020-11-16T22:37:00.782034+0000","last_fullsized":"2020-11-16T22:37:00.782034+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'23","last_scrub_stamp":"2020-11-16T22:37:00.782020+0000","last_deep_scrub":"111'23","last_deep_scrub_stamp":"2020-11-16T22:37:00.782020+0000","last_clean_scrub_stamp":"2020-11-16T22:37:00.782020+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":942,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1","version":"111'30","reported_seq":"46","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:59.606089+0000","last_change":"2020-11-16T22:36:59.606291+0000","last_active":"2020-11-16T22:36:59.606089+0000","last_peered":"2020-11-16T22:36:59.606089+0000","last_clean":"2020-11-16T22:36:59.606089+0000","last_became_active":"2020-11-16T22:35:21.178511+0000","last_became_peered":"2020-11-16T22:35:21.178511+0000","last_unstale":"2020-11-16T22:36:59.606089+0000","last_undegraded":"2020-11-16T22:36:59.606089+0000","last_fullsized":"2020-11-16T22:36:59.606089+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'30","last_scrub_stamp":"2020-11-16T22:36:59.606072+0000","last_deep_scrub":"111'30","last_deep_scrub_stamp":"2020-11-16T22:36:59.606072+0000","last_clean_scrub_stamp":"2020-11-16T22:36:59.606072+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14153,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":693,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.0","version":"111'34","reported_seq":"48","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:57.784324+0000","last_change":"2020-11-16T22:36:57.784479+0000","last_active":"2020-11-16T22:36:57.784324+0000","last_peered":"2020-11-16T22:36:57.784324+0000","last_clean":"2020-11-16T22:36:57.784324+0000","last_became_active":"2020-11-16T22:35:22.266066+0000","last_became_peered":"2020-11-16T22:35:22.266066+0000","last_unstale":"2020-11-16T22:36:57.784324+0000","last_undegraded":"2020-11-16T22:36:57.784324+0000","last_fullsized":"2020-11-16T22:36:57.784324+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'34","last_scrub_stamp":"2020-11-16T22:36:57.784312+0000","last_deep_scrub":"111'34","last_deep_scrub_stamp":"2020-11-16T22:36:57.784312+0000","last_clean_scrub_stamp":"2020-11-16T22:36:57.784312+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":471,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"77","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.725413+0000","last_change":"2020-11-16T22:36:56.725629+0000","last_active":"2020-11-16T22:36:56.725413+0000","last_peered":"2020-11-16T22:36:56.725413+0000","last_clean":"2020-11-16T22:36:56.725413+0000","last_became_active":"2020-11-16T22:26:48.228131+0000","last_became_peered":"2020-11-16T22:26:48.228131+0000","last_unstale":"2020-11-16T22:36:56.725413+0000","last_undegraded":"2020-11-16T22:36:56.725413+0000","last_fullsized":"2020-11-16T22:36:56.725413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:56.725392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:56.725392+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.725392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:54.187347+0000","last_change":"2020-11-16T22:36:54.187547+0000","last_active":"2020-11-16T22:36:54.187347+0000","last_peered":"2020-11-16T22:36:54.187347+0000","last_clean":"2020-11-16T22:36:54.187347+0000","last_became_active":"2020-11-16T22:26:48.225632+0000","last_became_peered":"2020-11-16T22:26:48.225632+0000","last_unstale":"2020-11-16T22:36:54.187347+0000","last_undegraded":"2020-11-16T22:36:54.187347+0000","last_fullsized":"2020-11-16T22:36:54.187347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:54.187336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:54.187336+0000","last_clean_scrub_stamp":"2020-11-16T22:36:54.187336+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"80","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:57.588001+0000","last_change":"2020-11-16T22:36:57.588200+0000","last_active":"2020-11-16T22:36:57.588001+0000","last_peered":"2020-11-16T22:36:57.588001+0000","last_clean":"2020-11-16T22:36:57.588001+0000","last_became_active":"2020-11-16T22:26:48.144543+0000","last_became_peered":"2020-11-16T22:26:48.144543+0000","last_unstale":"2020-11-16T22:36:57.588001+0000","last_undegraded":"2020-11-16T22:36:57.588001+0000","last_fullsized":"2020-11-16T22:36:57.588001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:57.587987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:57.587987+0000","last_clean_scrub_stamp":"2020-11-16T22:36:57.587987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:56.172068+0000","last_change":"2020-11-16T22:36:56.172274+0000","last_active":"2020-11-16T22:36:56.172068+0000","last_peered":"2020-11-16T22:36:56.172068+0000","last_clean":"2020-11-16T22:36:56.172068+0000","last_became_active":"2020-11-16T22:26:48.223341+0000","last_became_peered":"2020-11-16T22:26:48.223341+0000","last_unstale":"2020-11-16T22:36:56.172068+0000","last_undegraded":"2020-11-16T22:36:56.172068+0000","last_fullsized":"2020-11-16T22:36:56.172068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:56.172044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:56.172044+0000","last_clean_scrub_stamp":"2020-11-16T22:36:56.172044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"111'4","reported_seq":"101","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.732418+0000","last_change":"2020-11-16T22:36:53.732578+0000","last_active":"2020-11-16T22:36:53.732418+0000","last_peered":"2020-11-16T22:36:53.732418+0000","last_clean":"2020-11-16T22:36:53.732418+0000","last_became_active":"2020-11-16T22:26:44.681535+0000","last_became_peered":"2020-11-16T22:26:44.681535+0000","last_unstale":"2020-11-16T22:36:53.732418+0000","last_undegraded":"2020-11-16T22:36:53.732418+0000","last_fullsized":"2020-11-16T22:36:53.732418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'4","last_scrub_stamp":"2020-11-16T22:36:53.732406+0000","last_deep_scrub":"111'4","last_deep_scrub_stamp":"2020-11-16T22:36:53.732406+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.732406+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:53.191036+0000","last_change":"2020-11-16T22:36:53.191223+0000","last_active":"2020-11-16T22:36:53.191036+0000","last_peered":"2020-11-16T22:36:53.191036+0000","last_clean":"2020-11-16T22:36:53.191036+0000","last_became_active":"2020-11-16T22:26:48.142665+0000","last_became_peered":"2020-11-16T22:26:48.142665+0000","last_unstale":"2020-11-16T22:36:53.191036+0000","last_undegraded":"2020-11-16T22:36:53.191036+0000","last_fullsized":"2020-11-16T22:36:53.191036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:53.191027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:53.191027+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.191027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:59.172773+0000","last_change":"2020-11-16T22:36:59.172950+0000","last_active":"2020-11-16T22:36:59.172773+0000","last_peered":"2020-11-16T22:36:59.172773+0000","last_clean":"2020-11-16T22:36:59.172773+0000","last_became_active":"2020-11-16T22:26:48.140871+0000","last_became_peered":"2020-11-16T22:26:48.140871+0000","last_unstale":"2020-11-16T22:36:59.172773+0000","last_undegraded":"2020-11-16T22:36:59.172773+0000","last_fullsized":"2020-11-16T22:36:59.172773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:59.172759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:59.172759+0000","last_clean_scrub_stamp":"2020-11-16T22:36:59.172759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"91","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:36:52.810822+0000","last_change":"2020-11-16T22:36:52.811013+0000","last_active":"2020-11-16T22:36:52.810822+0000","last_peered":"2020-11-16T22:36:52.810822+0000","last_clean":"2020-11-16T22:36:52.810822+0000","last_became_active":"2020-11-16T22:26:48.227969+0000","last_became_peered":"2020-11-16T22:26:48.227969+0000","last_unstale":"2020-11-16T22:36:52.810822+0000","last_undegraded":"2020-11-16T22:36:52.810822+0000","last_fullsized":"2020-11-16T22:36:52.810822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:52.810808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:52.810808+0000","last_clean_scrub_stamp":"2020-11-16T22:36:52.810808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"88","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:37:00.202485+0000","last_change":"2020-11-16T22:37:00.202679+0000","last_active":"2020-11-16T22:37:00.202485+0000","last_peered":"2020-11-16T22:37:00.202485+0000","last_clean":"2020-11-16T22:37:00.202485+0000","last_became_active":"2020-11-16T22:26:48.225483+0000","last_became_peered":"2020-11-16T22:26:48.225483+0000","last_unstale":"2020-11-16T22:37:00.202485+0000","last_undegraded":"2020-11-16T22:37:00.202485+0000","last_fullsized":"2020-11-16T22:37:00.202485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:37:00.202471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:37:00.202471+0000","last_clean_scrub_stamp":"2020-11-16T22:37:00.202471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":8,"stat_sum":{"num_bytes":117916,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":257,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18628,"num_omap_keys":41,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":235832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":3},{"poolid":16,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542283,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7744,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364096,"statfs":{"total":96636764160,"available":96628834304,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.718,"15min":1.7569999999999999},"min":{"1min":1.115,"5min":0.98599999999999999,"15min":0.65100000000000002},"max":{"1min":2.7770000000000001,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.915,"5min":1.8009999999999999,"15min":1.792},"min":{"1min":1.262,"5min":1.1279999999999999,"15min":0.68899999999999995},"max":{"1min":3.0800000000000001,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":2.3759999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":2.0870000000000002,"15min":2.028},"min":{"1min":1.306,"5min":0.93000000000000005,"15min":0.92500000000000004},"max":{"1min":2.6909999999999998,"5min":4.1459999999999999,"15min":4.1459999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.748,"5min":1.8029999999999999,"15min":1.7609999999999999},"min":{"1min":1.4279999999999999,"5min":1.1060000000000001,"15min":0.73099999999999998},"max":{"1min":2.4020000000000001,"5min":2.7749999999999999,"15min":2.883},"last":1.7090000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.2400000000000002,"5min":2.1869999999999998,"15min":2.0470000000000002},"min":{"1min":1.5920000000000001,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":3.0129999999999999,"5min":3.0790000000000002,"15min":3.4780000000000002},"last":2.4750000000000001},{"interface":"front","average":{"1min":2.2570000000000001,"5min":2.274,"15min":2.2890000000000001},"min":{"1min":1.3140000000000001,"5min":1.3,"15min":1.105},"max":{"1min":2.9950000000000001,"5min":3.98,"15min":3.98},"last":2.6179999999999999}]},{"osd":3,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.6240000000000001,"15min":2.5710000000000002},"min":{"1min":2.0430000000000001,"5min":1.4910000000000001,"15min":1.2190000000000001},"max":{"1min":3.6059999999999999,"5min":3.6850000000000001,"15min":3.9969999999999999},"last":2.742},{"interface":"front","average":{"1min":2.859,"5min":2.8290000000000002,"15min":2.8260000000000001},"min":{"1min":2.2509999999999999,"5min":1.6020000000000001,"15min":1.2689999999999999},"max":{"1min":3.3370000000000002,"5min":4.266,"15min":4.266},"last":2.8279999999999998}]},{"osd":4,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.194,"5min":2.3250000000000002,"15min":2.23},"min":{"1min":1.673,"5min":1.631,"15min":1.018},"max":{"1min":3.46,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.5600000000000001},{"interface":"front","average":{"1min":2.8849999999999998,"5min":2.802,"15min":2.778},"min":{"1min":2.0529999999999999,"5min":1.748,"15min":1.1479999999999999},"max":{"1min":3.5920000000000001,"5min":3.9729999999999999,"15min":3.9940000000000002},"last":2.9620000000000002}]},{"osd":5,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":2.9500000000000002,"15min":2.9489999999999998},"min":{"1min":2.5,"5min":1.9750000000000001,"15min":1.3540000000000001},"max":{"1min":3.7120000000000002,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":3.0630000000000002},{"interface":"front","average":{"1min":2.7850000000000001,"5min":2.8119999999999998,"15min":2.7240000000000002},"min":{"1min":2.2080000000000002,"5min":1.899,"15min":1.1299999999999999},"max":{"1min":3.9390000000000001,"5min":4.0380000000000003,"15min":4.0380000000000003},"last":3.0169999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":2.907,"15min":2.871},"min":{"1min":2.3519999999999999,"5min":2.0409999999999999,"15min":1.2989999999999999},"max":{"1min":3.7250000000000001,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":2.7839999999999998},{"interface":"front","average":{"1min":3.2490000000000001,"5min":3.226,"15min":3.1739999999999999},"min":{"1min":2.5299999999999998,"5min":2.1629999999999998,"15min":1.222},"max":{"1min":3.9449999999999998,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":3.1869999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542282,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7572,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":6208,"kb_avail":94364268,"statfs":{"total":96636764160,"available":96629010432,"internally_reserved":0,"allocated":1388544,"data_stored":630469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6356992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.7250000000000001,"15min":1.6539999999999999},"min":{"1min":1.343,"5min":0.98699999999999999,"15min":0.97499999999999998},"max":{"1min":2.2330000000000001,"5min":3.206,"15min":3.206},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.774,"15min":1.75},"min":{"1min":1.1060000000000001,"5min":1.083,"15min":0.94099999999999995},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.1579999999999999},"last":1.5489999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":1.956,"5min":1.806,"15min":1.7070000000000001},"min":{"1min":1.431,"5min":1.1779999999999999,"15min":1.0469999999999999},"max":{"1min":2.8679999999999999,"5min":3.423,"15min":3.423},"last":1.9239999999999999},{"interface":"front","average":{"1min":2.3610000000000002,"5min":2.2549999999999999,"15min":2.1819999999999999},"min":{"1min":1.712,"5min":1.181,"15min":1.01},"max":{"1min":2.9580000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":1.722}]},{"osd":2,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":2.2869999999999999,"15min":2.23},"min":{"1min":1.7769999999999999,"5min":1.282,"15min":1.131},"max":{"1min":3.0720000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.149},{"interface":"front","average":{"1min":2.165,"5min":2.117,"15min":2.0790000000000002},"min":{"1min":1.361,"5min":1.133,"15min":1.0700000000000001},"max":{"1min":2.8130000000000002,"5min":2.948,"15min":2.948},"last":1.986}]},{"osd":3,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.4359999999999999,"5min":2.512,"15min":2.448},"min":{"1min":1.615,"5min":1.615,"15min":1.1850000000000001},"max":{"1min":3.1629999999999998,"5min":3.2650000000000001,"15min":3.3340000000000001},"last":2.504},{"interface":"front","average":{"1min":2.847,"5min":2.6880000000000002,"15min":2.5880000000000001},"min":{"1min":2.2930000000000001,"5min":1.671,"15min":1.377},"max":{"1min":3.5209999999999999,"5min":3.7130000000000001,"15min":3.7130000000000001},"last":2.4300000000000002}]},{"osd":4,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.234,"5min":2.323,"15min":2.355},"min":{"1min":1.4750000000000001,"5min":1.3380000000000001,"15min":1.214},"max":{"1min":2.665,"5min":3.343,"15min":3.8570000000000002},"last":2.8130000000000002},{"interface":"front","average":{"1min":2.7869999999999999,"5min":2.8199999999999998,"15min":2.8069999999999999},"min":{"1min":1.415,"5min":1.415,"15min":1.415},"max":{"1min":3.3650000000000002,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":2.6030000000000002}]},{"osd":6,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.9390000000000001,"5min":2.8740000000000001,"15min":2.835},"min":{"1min":1.907,"5min":1.8109999999999999,"15min":1.6379999999999999},"max":{"1min":3.8820000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":3.0409999999999999},{"interface":"front","average":{"1min":3.1230000000000002,"5min":3.181,"15min":3.137},"min":{"1min":2.319,"5min":2.319,"15min":1.8680000000000001},"max":{"1min":3.9119999999999999,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":2.907}]},{"osd":7,"last update":"Mon Nov 16 22:37:06 2020","interfaces":[{"interface":"back","average":{"1min":2.8130000000000002,"5min":2.867,"15min":2.8809999999999998},"min":{"1min":1.206,"5min":1.206,"15min":1.206},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":2.8620000000000001},{"interface":"front","average":{"1min":3.2770000000000001,"5min":3.2410000000000001,"15min":3.2029999999999998},"min":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":1.913},"max":{"1min":3.9700000000000002,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":3.2690000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7780,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94364060,"statfs":{"total":96636764160,"available":96628797440,"internally_reserved":0,"allocated":1404928,"data_stored":644510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5429999999999999,"15min":1.575},"min":{"1min":1.1990000000000001,"5min":0.95799999999999996,"15min":0.94899999999999995},"max":{"1min":1.9650000000000001,"5min":2.871,"15min":2.9350000000000001},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.663,"5min":1.6539999999999999,"15min":1.6419999999999999},"min":{"1min":1.325,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":2.3109999999999999,"5min":3.492,"15min":3.492},"last":1.4330000000000001}]},{"osd":1,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.954,"15min":1.871},"min":{"1min":1.258,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.9020000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.532},{"interface":"front","average":{"1min":2.5979999999999999,"5min":2.4940000000000002,"15min":2.5099999999999998},"min":{"1min":2.0299999999999998,"5min":1.417,"15min":1.417},"max":{"1min":3.2389999999999999,"5min":3.875,"15min":3.875},"last":2.6259999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.21,"5min":2.1379999999999999,"15min":2.2000000000000002},"min":{"1min":1.571,"5min":1.171,"15min":1.1259999999999999},"max":{"1min":3.0270000000000001,"5min":3.1309999999999998,"15min":3.4380000000000002},"last":1.948},{"interface":"front","average":{"1min":2.4009999999999998,"5min":2.3210000000000002,"15min":2.3410000000000002},"min":{"1min":1.833,"5min":1.3009999999999999,"15min":1.258},"max":{"1min":3.0760000000000001,"5min":3.0920000000000001,"15min":3.4860000000000002},"last":2.4710000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.157,"5min":2.1640000000000001,"15min":2.2389999999999999},"min":{"1min":1.6180000000000001,"5min":1.403,"15min":1.2949999999999999},"max":{"1min":2.9430000000000001,"5min":3.1779999999999999,"15min":5.3449999999999998},"last":1.8839999999999999},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.7040000000000002,"15min":2.7309999999999999},"min":{"1min":2.3029999999999999,"5min":1.7849999999999999,"15min":1.5009999999999999},"max":{"1min":3.484,"5min":3.7320000000000002,"15min":4.0830000000000002},"last":2.7130000000000001}]},{"osd":4,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.4180000000000001,"5min":2.5779999999999998,"15min":2.6280000000000001},"min":{"1min":1.7470000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.3410000000000002,"5min":3.782,"15min":5.6399999999999997},"last":2.2400000000000002},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.6789999999999998,"15min":2.774},"min":{"1min":2.069,"5min":1.9370000000000001,"15min":1.171},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":4.766},"last":2.2839999999999998}]},{"osd":5,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.444,"5min":2.4830000000000001,"15min":2.5110000000000001},"min":{"1min":1.7689999999999999,"5min":1.5649999999999999,"15min":1.5129999999999999},"max":{"1min":3.4630000000000001,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":2.1200000000000001},{"interface":"front","average":{"1min":2.8580000000000001,"5min":2.9769999999999999,"15min":3.0550000000000002},"min":{"1min":1.55,"5min":1.516,"15min":1.516},"max":{"1min":3.5409999999999999,"5min":4.4729999999999999,"15min":6.0380000000000003},"last":3.0089999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:37:59 2020","interfaces":[{"interface":"back","average":{"1min":2.952,"5min":3.044,"15min":3.169},"min":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":1.6379999999999999},"max":{"1min":3.7010000000000001,"5min":4.0700000000000003,"15min":5.3220000000000001},"last":3.097},{"interface":"front","average":{"1min":3.0230000000000001,"5min":3.024,"15min":3.0630000000000002},"min":{"1min":2.2429999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"max":{"1min":3.7949999999999999,"5min":4.806,"15min":5.2569999999999997},"last":3.1539999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8448,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":6976,"kb_avail":94363392,"statfs":{"total":96636764160,"available":96628113408,"internally_reserved":0,"allocated":1499136,"data_stored":731943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7143424},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":1.7390000000000001,"15min":1.8129999999999999},"min":{"1min":1.224,"5min":0.96799999999999997,"15min":0.65200000000000002},"max":{"1min":2.5539999999999998,"5min":3.02,"15min":3.02},"last":1.381},{"interface":"front","average":{"1min":2.1880000000000002,"5min":2.1669999999999998,"15min":2.1819999999999999},"min":{"1min":1.462,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.8109999999999999,"5min":2.956,"15min":3.7170000000000001},"last":2.1579999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.851,"5min":1.8089999999999999,"15min":1.7509999999999999},"min":{"1min":1.3360000000000001,"5min":1.1970000000000001,"15min":0.74199999999999999},"max":{"1min":2.8959999999999999,"5min":3.302,"15min":3.302},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.9790000000000001,"5min":2.0190000000000001,"15min":1.9119999999999999},"min":{"1min":1.3620000000000001,"5min":1.1180000000000001,"15min":1.0660000000000001},"max":{"1min":2.8359999999999999,"5min":3.1869999999999998,"15min":3.3620000000000001},"last":2.4060000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":1.9019999999999999,"15min":1.919},"min":{"1min":1.345,"5min":1.294,"15min":1.054},"max":{"1min":3.5739999999999998,"5min":4.117,"15min":4.117},"last":1.581},{"interface":"front","average":{"1min":2.1099999999999999,"5min":2.2029999999999998,"15min":2.1190000000000002},"min":{"1min":1.222,"5min":1.198,"15min":1.0129999999999999},"max":{"1min":3.3130000000000002,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.972}]},{"osd":3,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":2.3780000000000001,"15min":2.278},"min":{"1min":1.282,"5min":1.206,"15min":0.94599999999999995},"max":{"1min":3.1520000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":2.0529999999999999},{"interface":"front","average":{"1min":2.7360000000000002,"5min":2.7189999999999999,"15min":2.746},"min":{"1min":1.903,"5min":1.4490000000000001,"15min":1.4490000000000001},"max":{"1min":3.2200000000000002,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":2.9369999999999998}]},{"osd":5,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.5009999999999999,"5min":2.4180000000000001,"15min":2.448},"min":{"1min":1.613,"5min":1.361,"15min":1.075},"max":{"1min":3.3159999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":2.3370000000000002},{"interface":"front","average":{"1min":2.895,"5min":2.9199999999999999,"15min":2.9079999999999999},"min":{"1min":2.4540000000000002,"5min":1.6890000000000001,"15min":1.1160000000000001},"max":{"1min":3.4079999999999999,"5min":4.9859999999999998,"15min":4.9859999999999998},"last":2.843}]},{"osd":6,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":2.8149999999999999,"5min":2.8330000000000002,"15min":2.8370000000000002},"min":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.5529999999999999},"max":{"1min":3.859,"5min":4.3739999999999997,"15min":4.5220000000000002},"last":3.2229999999999999},{"interface":"front","average":{"1min":2.8570000000000002,"5min":2.8439999999999999,"15min":2.8199999999999998},"min":{"1min":1.899,"5min":1.8959999999999999,"15min":1.6359999999999999},"max":{"1min":4.181,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":2.5489999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:37:05 2020","interfaces":[{"interface":"back","average":{"1min":3.0600000000000001,"5min":3.1200000000000001,"15min":3.1059999999999999},"min":{"1min":2.254,"5min":2.242,"15min":1.857},"max":{"1min":3.931,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":3.1560000000000001},{"interface":"front","average":{"1min":2.9830000000000001,"5min":2.9809999999999999,"15min":2.9670000000000001},"min":{"1min":2.3980000000000001,"5min":1.8440000000000001,"15min":1.496},"max":{"1min":3.7970000000000002,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":2.7040000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7824,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94364016,"statfs":{"total":96636764160,"available":96628752384,"internally_reserved":0,"allocated":1384448,"data_stored":630427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.655,"15min":1.6359999999999999},"min":{"1min":1.347,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3260000000000001,"5min":2.6469999999999998,"15min":2.8159999999999998},"last":1.556},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7190000000000001,"15min":1.698},"min":{"1min":1.2070000000000001,"5min":1.0449999999999999,"15min":1.02},"max":{"1min":2.3940000000000001,"5min":2.653,"15min":2.653},"last":1.6659999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.671,"15min":1.657},"min":{"1min":1.2609999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.496,"5min":2.7069999999999999,"15min":2.7869999999999999},"last":2.1190000000000002},{"interface":"front","average":{"1min":1.9950000000000001,"5min":2.0950000000000002,"15min":2.0899999999999999},"min":{"1min":1.262,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.653,"5min":2.9119999999999999,"15min":3.4119999999999999},"last":2.3900000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.0230000000000001,"5min":2.125,"15min":2.0449999999999999},"min":{"1min":1.5,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2559999999999998},"last":2.282},{"interface":"front","average":{"1min":2.2709999999999999,"5min":2.3450000000000002,"15min":2.2919999999999998},"min":{"1min":1.411,"5min":1.056,"15min":0.99399999999999999},"max":{"1min":3.044,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":2.738}]},{"osd":4,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.452,"5min":2.54,"15min":2.669},"min":{"1min":1.897,"5min":1.5129999999999999,"15min":1.5029999999999999},"max":{"1min":2.8690000000000002,"5min":3.8559999999999999,"15min":4.8479999999999999},"last":2.625},{"interface":"front","average":{"1min":2.2530000000000001,"5min":2.391,"15min":2.4089999999999998},"min":{"1min":1.6519999999999999,"5min":1.5049999999999999,"15min":1.1930000000000001},"max":{"1min":3.419,"5min":3.956,"15min":4.0380000000000003},"last":2.5409999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.798,"5min":2.891,"15min":2.9430000000000001},"min":{"1min":2.0710000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"max":{"1min":3.3490000000000002,"5min":3.8700000000000001,"15min":5.0659999999999998},"last":2.9950000000000001},{"interface":"front","average":{"1min":2.706,"5min":2.7080000000000002,"15min":2.669},"min":{"1min":2.133,"5min":1.6779999999999999,"15min":1.6180000000000001},"max":{"1min":3.5129999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":2.9279999999999999}]},{"osd":6,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.7610000000000001,"5min":2.8610000000000002,"15min":2.8029999999999999},"min":{"1min":1.9510000000000001,"5min":1.772,"15min":1.772},"max":{"1min":3.4630000000000001,"5min":4.2599999999999998,"15min":4.2599999999999998},"last":2.3450000000000002},{"interface":"front","average":{"1min":3.0129999999999999,"5min":3.1240000000000001,"15min":3.1859999999999999},"min":{"1min":2.2930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"max":{"1min":3.6739999999999999,"5min":4.3490000000000002,"15min":5.2329999999999997},"last":3.206}]},{"osd":7,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":3.1110000000000002,"5min":3.2799999999999998,"15min":3.3380000000000001},"min":{"1min":2.5009999999999999,"5min":2.1320000000000001,"15min":2.0489999999999999},"max":{"1min":3.9369999999999998,"5min":4.5910000000000002,"15min":5.6379999999999999},"last":3.3959999999999999},{"interface":"front","average":{"1min":2.9710000000000001,"5min":2.9870000000000001,"15min":2.9449999999999998},"min":{"1min":2.4500000000000002,"5min":2.036,"15min":1.895},"max":{"1min":3.4249999999999998,"5min":4.6059999999999999,"15min":4.6059999999999999},"last":3.161}]}]},{"osd":1,"up_from":14,"seq":60129542282,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7796,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94364044,"statfs":{"total":96636764160,"available":96628781056,"internally_reserved":0,"allocated":1486848,"data_stored":725455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.665,"15min":1.6359999999999999},"min":{"1min":1.4059999999999999,"5min":0.97299999999999998,"15min":0.70199999999999996},"max":{"1min":2.0550000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.736,"15min":1.6890000000000001},"min":{"1min":1.3799999999999999,"5min":1.119,"15min":0.83699999999999997},"max":{"1min":2.5680000000000001,"5min":2.6829999999999998,"15min":2.718},"last":2.0609999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.8600000000000001,"5min":1.8160000000000001,"15min":1.7290000000000001},"min":{"1min":1.524,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.6659999999999999,"5min":10.023,"15min":10.023},"last":2.1890000000000001},{"interface":"front","average":{"1min":2.1230000000000002,"5min":2.2269999999999999,"15min":2.1040000000000001},"min":{"1min":1.633,"5min":0.95499999999999996,"15min":0.80400000000000005},"max":{"1min":2.492,"5min":7.7800000000000002,"15min":7.7800000000000002},"last":1.5840000000000001}]},{"osd":3,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.1789999999999998,"5min":2.069,"15min":2.1309999999999998},"min":{"1min":1.605,"5min":1.218,"15min":0.81000000000000005},"max":{"1min":2.6539999999999999,"5min":2.9159999999999999,"15min":3.3319999999999999},"last":1.671},{"interface":"front","average":{"1min":2.274,"5min":2.2719999999999998,"15min":2.2160000000000002},"min":{"1min":1.671,"5min":1.1990000000000001,"15min":0.85199999999999998},"max":{"1min":3.0720000000000001,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":1.829}]},{"osd":4,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.3599999999999999,"5min":2.4060000000000001,"15min":2.343},"min":{"1min":1.8220000000000001,"5min":1.516,"15min":1.008},"max":{"1min":3.2189999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":2.5470000000000002},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.6960000000000002,"15min":2.6859999999999999},"min":{"1min":2.0680000000000001,"5min":1.022,"15min":1.022},"max":{"1min":3.847,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":2.4609999999999999}]},{"osd":5,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.9820000000000002,"5min":2.9710000000000001,"15min":3.0110000000000001},"min":{"1min":2.274,"5min":1.276,"15min":1.276},"max":{"1min":3.6989999999999998,"5min":5.351,"15min":5.351},"last":2.802},{"interface":"front","average":{"1min":2.8620000000000001,"5min":2.758,"15min":2.802},"min":{"1min":2.1389999999999998,"5min":1.4279999999999999,"15min":1.3839999999999999},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":2.984}]},{"osd":6,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3.101,"5min":3.0169999999999999,"15min":3.0070000000000001},"min":{"1min":2.3919999999999999,"5min":1.8480000000000001,"15min":1.5269999999999999},"max":{"1min":4.3559999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":2.6150000000000002},{"interface":"front","average":{"1min":2.9769999999999999,"5min":2.9399999999999999,"15min":2.875},"min":{"1min":2.3149999999999999,"5min":1.861,"15min":1.4670000000000001},"max":{"1min":3.6299999999999999,"5min":7.2770000000000001,"15min":7.2770000000000001},"last":2.911}]},{"osd":7,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3.1720000000000002,"5min":3.121,"15min":3.1040000000000001},"min":{"1min":2.5249999999999999,"5min":2.008,"15min":1.7010000000000001},"max":{"1min":3.9049999999999998,"5min":4.1239999999999997,"15min":4.7430000000000003},"last":3.1520000000000001},{"interface":"front","average":{"1min":3.319,"5min":3.2000000000000002,"15min":3.2170000000000001},"min":{"1min":2.3140000000000001,"5min":2.302,"15min":1.712},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":3.2189999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542283,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":1368064,"data_stored":629692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6739999999999999,"15min":1.653},"min":{"1min":1.012,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.202,"5min":2.6179999999999999,"15min":3.0710000000000002},"last":2.1070000000000002},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.752,"15min":1.7849999999999999},"min":{"1min":1.0700000000000001,"5min":1.032,"15min":0.81899999999999995},"max":{"1min":2.246,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.925,"5min":1.905,"15min":1.849},"min":{"1min":1.26,"5min":1.0069999999999999,"15min":0.91500000000000004},"max":{"1min":2.4889999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.446},{"interface":"front","average":{"1min":1.9830000000000001,"5min":1.9870000000000001,"15min":1.95},"min":{"1min":1.032,"5min":0.81499999999999995,"15min":0.80500000000000005},"max":{"1min":3.0110000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.758}]},{"osd":3,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.909,"15min":1.867},"min":{"1min":1.1080000000000001,"5min":1.0840000000000001,"15min":0.92900000000000005},"max":{"1min":2.887,"5min":3.0840000000000001,"15min":3.9660000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":2.6379999999999999,"5min":2.6000000000000001,"15min":2.4830000000000001},"min":{"1min":2.2320000000000002,"5min":1.5860000000000001,"15min":1.1499999999999999},"max":{"1min":3.3090000000000002,"5min":3.944,"15min":3.944},"last":2.363}]},{"osd":4,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.5800000000000001,"5min":2.6219999999999999,"15min":2.5230000000000001},"min":{"1min":1.962,"5min":1.5800000000000001,"15min":1.498},"max":{"1min":3.464,"5min":5.0209999999999999,"15min":5.0209999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":2.569,"5min":2.552,"15min":2.5710000000000002},"min":{"1min":1.4810000000000001,"5min":1.452,"15min":1.3260000000000001},"max":{"1min":4.0830000000000002,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":1.984}]},{"osd":5,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.8180000000000001,"5min":2.7959999999999998,"15min":2.8039999999999998},"min":{"1min":1.7390000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"max":{"1min":3.5,"5min":3.738,"15min":4.0259999999999998},"last":2.2000000000000002},{"interface":"front","average":{"1min":2.444,"5min":2.4249999999999998,"15min":2.4529999999999998},"min":{"1min":2.032,"5min":1.639,"15min":1.363},"max":{"1min":3.5640000000000001,"5min":3.7970000000000002,"15min":4.3070000000000004},"last":2.0659999999999998}]},{"osd":6,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":2.6760000000000002,"15min":2.681},"min":{"1min":1.7769999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"max":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.165},"last":2.2650000000000001},{"interface":"front","average":{"1min":3.1120000000000001,"5min":3.0489999999999999,"15min":2.9660000000000002},"min":{"1min":2.4700000000000002,"5min":2.0550000000000002,"15min":1.9930000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":2.573}]},{"osd":7,"last update":"Mon Nov 16 22:37:07 2020","interfaces":[{"interface":"back","average":{"1min":3.085,"5min":3.069,"15min":3.081},"min":{"1min":2.1619999999999999,"5min":1.903,"15min":1.8879999999999999},"max":{"1min":3.8940000000000001,"5min":4.0599999999999996,"15min":4.1210000000000004},"last":2.403},{"interface":"front","average":{"1min":2.847,"5min":2.9359999999999999,"15min":2.996},"min":{"1min":1.919,"5min":1.488,"15min":1.488},"max":{"1min":3.7549999999999999,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":2.468}]}]},{"osd":3,"up_from":14,"seq":60129542281,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8316,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94363524,"statfs":{"total":96636764160,"available":96628248576,"internally_reserved":0,"allocated":1429504,"data_stored":651180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6799999999999999,"15min":1.639},"min":{"1min":1.133,"5min":1.0820000000000001,"15min":0.73599999999999999},"max":{"1min":2.1549999999999998,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.786,"15min":1.7729999999999999},"min":{"1min":0.79700000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2989999999999999,"5min":2.8780000000000001,"15min":3.6739999999999999},"last":1.536}]},{"osd":1,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6759999999999999,"15min":1.6419999999999999},"min":{"1min":1.2849999999999999,"5min":0.872,"15min":0.81100000000000005},"max":{"1min":2.2229999999999999,"5min":2.6880000000000002,"15min":2.77},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.0129999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":1.3400000000000001,"5min":1.25,"15min":1.099},"max":{"1min":2.645,"5min":5.258,"15min":5.258},"last":1.5009999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.077,"5min":2.1829999999999998,"15min":2.1749999999999998},"min":{"1min":1.401,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.6099999999999999,"5min":3.214,"15min":3.597},"last":1.8260000000000001},{"interface":"front","average":{"1min":2.3100000000000001,"5min":2.403,"15min":2.3759999999999999},"min":{"1min":1.5029999999999999,"5min":1.3700000000000001,"15min":1.075},"max":{"1min":2.907,"5min":4.0289999999999999,"15min":4.0990000000000002},"last":1.9239999999999999}]},{"osd":4,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.577,"5min":2.5739999999999998,"15min":2.5379999999999998},"min":{"1min":1.589,"5min":1.298,"15min":1.2150000000000001},"max":{"1min":3.419,"5min":3.738,"15min":3.7450000000000001},"last":1.9930000000000001},{"interface":"front","average":{"1min":2.3780000000000001,"5min":2.4020000000000001,"15min":2.4009999999999998},"min":{"1min":1.496,"5min":1.403,"15min":1.1699999999999999},"max":{"1min":3.2170000000000001,"5min":3.681,"15min":3.6829999999999998},"last":2.6760000000000002}]},{"osd":5,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.6560000000000001,"5min":2.7519999999999998,"15min":2.7469999999999999},"min":{"1min":1.6559999999999999,"5min":1.5600000000000001,"15min":1.2709999999999999},"max":{"1min":3.5059999999999998,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":3.4809999999999999},{"interface":"front","average":{"1min":2.9260000000000002,"5min":2.968,"15min":2.9950000000000001},"min":{"1min":2.2040000000000002,"5min":1.857,"15min":1.8360000000000001},"max":{"1min":3.8319999999999999,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":2.7570000000000001}]},{"osd":6,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":2.9649999999999999,"5min":3.077,"15min":3.097},"min":{"1min":1.8799999999999999,"5min":1.615,"15min":1.615},"max":{"1min":3.6360000000000001,"5min":4.3970000000000002,"15min":4.5359999999999996},"last":3.1829999999999998},{"interface":"front","average":{"1min":2.891,"5min":3.0590000000000002,"15min":3.0750000000000002},"min":{"1min":2.1850000000000001,"5min":2.04,"15min":1.7310000000000001},"max":{"1min":3.5449999999999999,"5min":4.1959999999999997,"15min":4.5590000000000002},"last":2.6259999999999999}]},{"osd":7,"last update":"Mon Nov 16 22:37:02 2020","interfaces":[{"interface":"back","average":{"1min":3,"5min":3.2109999999999999,"15min":3.1600000000000001},"min":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.8240000000000001},"max":{"1min":3.722,"5min":4.343,"15min":4.6470000000000002},"last":2.552},{"interface":"front","average":{"1min":2.7509999999999999,"5min":2.9239999999999999,"15min":2.9289999999999998},"min":{"1min":1.633,"5min":1.633,"15min":1.633},"max":{"1min":3.444,"5min":3.98,"15min":3.98},"last":3.056}]}]}]}} 2020-11-16T22:38:06.561 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-11-16T22:38:06.687 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 Processor -- start 2020-11-16T22:38:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 -- start start 2020-11-16T22:38:06.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc514156a90 0x7fc514061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:06.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140cccd0 con 0x7fc5140615f0 2020-11-16T22:38:06.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140ccf70 con 0x7fc51415f3c0 2020-11-16T22:38:06.690 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc519325700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51415e030 con 0x7fc514156a90 2020-11-16T22:38:06.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc513fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc514156a90 0x7fc514061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:06.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc518b24700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc51415f3c0 0x7fc5140cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52856/0 (socket says 172.21.15.181:52856) 2020-11-16T22:38:06.691 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc518b24700 1 -- 172.21.15.181:0/2230743831 learned_addr learned my addr 172.21.15.181:0/2230743831 (peer_addr_for_me v1:172.21.15.181:0/0) 2020-11-16T22:38:06.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc513fff700 1 -- 172.21.15.181:0/2230743831 >> v1:172.21.15.181:6789/0 conn(0x7fc51415f3c0 legacy=0x7fc5140cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:06.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc513fff700 1 -- 172.21.15.181:0/2230743831 >> v1:172.21.15.175:6789/0 conn(0x7fc5140615f0 legacy=0x7fc51415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:06.692 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc512ffd700 1 -- 172.21.15.181:0/2230743831 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969236862 0 0) 0x7fc5140ccf70 con 0x7fc51415f3c0 2020-11-16T22:38:06.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.697+0000 7fc513fff700 1 -- 172.21.15.181:0/2230743831 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51415e2d0 con 0x7fc514156a90 2020-11-16T22:38:06.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc513fff700 1 --2- 172.21.15.181:0/2230743831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc514061100 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fc504009c90 tx=0x7fc5040091d0).ready entity=mon.1 client_cookie=5fc5b449da69643a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:06.894 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc512ffd700 1 -- 172.21.15.181:0/2230743831 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400bbc0 con 0x7fc514156a90 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc512ffd700 1 -- 172.21.15.181:0/2230743831 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50400bd20 con 0x7fc514156a90 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc512ffd700 1 -- 172.21.15.181:0/2230743831 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504011610 con 0x7fc514156a90 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- 172.21.15.181:0/2230743831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 msgr2=0x7fc514061100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 --2- 172.21.15.181:0/2230743831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc514061100 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fc504009c90 tx=0x7fc5040091d0).stop 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- 172.21.15.181:0/2230743831 shutdown_connections 2020-11-16T22:38:06.895 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 --2- 172.21.15.181:0/2230743831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc514061100 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:06.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- 172.21.15.181:0/2230743831 >> 172.21.15.181:0/2230743831 conn(0x7fc5140cf630 msgr2=0x7fc5140cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:38:06.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- 172.21.15.181:0/2230743831 shutdown_connections 2020-11-16T22:38:06.896 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- 172.21.15.181:0/2230743831 wait complete. 2020-11-16T22:38:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 Processor -- start 2020-11-16T22:38:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.701+0000 7fc519325700 1 -- start start 2020-11-16T22:38:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 0x7fc5140c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:06.900 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140cc820 con 0x7fc5140615f0 2020-11-16T22:38:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc514170c10 con 0x7fc514156a90 2020-11-16T22:38:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc514170eb0 con 0x7fc51415f3c0 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc5137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc513fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 0x7fc5140c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc513fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 0x7fc5140c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.181:52556/0 (socket says 172.21.15.181:52556) 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc5137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38556/0 (socket says 172.21.15.181:38556) 2020-11-16T22:38:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc513fff700 1 -- 172.21.15.181:0/1339398049 learned_addr learned my addr 172.21.15.181:0/1339398049 (peer_addr_for_me v2:172.21.15.181:0/0) 2020-11-16T22:38:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:46400/0 (socket says 172.21.15.181:46400) 2020-11-16T22:38:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 -- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 msgr2=0x7fc51416ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 -- 172.21.15.181:0/1339398049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 msgr2=0x7fc5140c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:07.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 0x7fc5140c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 -- 172.21.15.181:0/1339398049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514171150 con 0x7fc51415f3c0 2020-11-16T22:38:07.306 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc5137fe700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc518b24700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fc50c00c760 tx=0x7fc50c0078e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0072e0 con 0x7fc51415f3c0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c007460 con 0x7fc51415f3c0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c005840 con 0x7fc51415f3c0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5141713f0 con 0x7fc51415f3c0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.705+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5140c8740 con 0x7fc51415f3c0 2020-11-16T22:38:07.307 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514170a30 con 0x7fc51415f3c0 2020-11-16T22:38:07.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc50c010070 con 0x7fc51415f3c0 2020-11-16T22:38:07.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc50bfff700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 0x7fc4f0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:38:07.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc50c06d200 con 0x7fc51415f3c0 2020-11-16T22:38:07.308 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc513fff700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 0x7fc4f0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:38:07.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.709+0000 7fc513fff700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 0x7fc4f0057550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc504009e70 tx=0x7fc504016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:38:07.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.717+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc50c043460 con 0x7fc51415f3c0 2020-11-16T22:38:07.510 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.873+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"} v 0) v1 -- 0x7fc514044f60 con 0x7fc51415f3c0 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc50c06d080 con 0x7fc51415f3c0 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc50bfff700 1 -- 172.21.15.181:0/1339398049 <== mon.2 v2:172.21.15.181:3301/0 8 ==== mon_command_ack([{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}]=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fc50c046a80 con 0x7fc51415f3c0 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 msgr2=0x7fc4f0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 0x7fc4f0057550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc504009e70 tx=0x7fc504016000).stop 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 msgr2=0x7fc5140d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fc50c00c760 tx=0x7fc50c0078e0).stop 2020-11-16T22:38:07.511 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 shutdown_connections 2020-11-16T22:38:07.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:6832/15040,v1:172.21.15.181:6833/15040] conn(0x7fc4f0055090 0x7fc4f0057550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc51415f3c0 0x7fc5140d1400 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.512 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140615f0 0x7fc5140c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 --2- 172.21.15.181:0/1339398049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc514156a90 0x7fc51416ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:38:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.885+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 >> 172.21.15.181:0/1339398049 conn(0x7fc5140cf630 msgr2=0x7fc5140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:38:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.889+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 shutdown_connections 2020-11-16T22:38:07.715 INFO:teuthology.orchestra.run.smithi181.stderr:2020-11-16T22:38:06.889+0000 7fc519325700 1 -- 172.21.15.181:0/1339398049 wait complete. 2020-11-16T22:38:07.715 INFO:teuthology.misc:Shutting down mds daemons... 2020-11-16T22:38:07.716 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:38:07.716 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.732 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:38:07.732 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:38:07.733 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.790 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:38:07.791 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:38:07.791 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.846 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:38:07.847 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-16T22:38:07.847 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.901 INFO:tasks.ceph.mds.d:Stopped 2020-11-16T22:38:07.902 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2020-11-16T22:38:07.902 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.919 INFO:tasks.ceph.mds.e:Stopped 2020-11-16T22:38:07.919 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2020-11-16T22:38:07.919 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:07.983 INFO:tasks.ceph.mds.f:Stopped 2020-11-16T22:38:07.983 DEBUG:tasks.ceph.mds.g:waiting for process to exit 2020-11-16T22:38:07.983 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.036 INFO:tasks.ceph.mds.g:Stopped 2020-11-16T22:38:08.037 DEBUG:tasks.ceph.mds.h:waiting for process to exit 2020-11-16T22:38:08.037 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.094 INFO:tasks.ceph.mds.h:Stopped 2020-11-16T22:38:08.094 DEBUG:tasks.ceph.mds.i:waiting for process to exit 2020-11-16T22:38:08.094 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.150 INFO:tasks.ceph.mds.i:Stopped 2020-11-16T22:38:08.150 INFO:teuthology.misc:Shutting down osd daemons... 2020-11-16T22:38:08.151 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-11-16T22:38:08.151 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.222 INFO:tasks.ceph.osd.0:Stopped 2020-11-16T22:38:08.222 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-11-16T22:38:08.222 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.312 INFO:tasks.ceph.osd.1:Stopped 2020-11-16T22:38:08.313 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-11-16T22:38:08.313 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2020-11-16T22:38:08.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2020-11-16T22:38:08.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g is failed for ~0s 2020-11-16T22:38:08.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.h is failed for ~0s 2020-11-16T22:38:08.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i is failed for ~0s 2020-11-16T22:38:08.406 INFO:tasks.ceph.osd.2:Stopped 2020-11-16T22:38:08.406 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2020-11-16T22:38:08.407 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.464 INFO:tasks.ceph.osd.3:Stopped 2020-11-16T22:38:08.464 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-16T22:38:08.464 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.529 INFO:tasks.ceph.osd.4:Stopped 2020-11-16T22:38:08.530 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2020-11-16T22:38:08.530 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.600 INFO:tasks.ceph.osd.5:Stopped 2020-11-16T22:38:08.600 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2020-11-16T22:38:08.601 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.661 INFO:tasks.ceph.osd.6:Stopped 2020-11-16T22:38:08.662 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2020-11-16T22:38:08.662 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.726 INFO:tasks.ceph.osd.7:Stopped 2020-11-16T22:38:08.727 INFO:teuthology.misc:Shutting down mgr daemons... 2020-11-16T22:38:08.727 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2020-11-16T22:38:08.727 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.754 INFO:tasks.ceph.mgr.y:Stopped 2020-11-16T22:38:08.754 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-11-16T22:38:08.755 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.782 INFO:tasks.ceph.mgr.x:Stopped 2020-11-16T22:38:08.782 INFO:teuthology.misc:Shutting down mon daemons... 2020-11-16T22:38:08.783 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-16T22:38:08.783 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.813 INFO:tasks.ceph.mon.a:Stopped 2020-11-16T22:38:08.814 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-16T22:38:08.814 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.874 INFO:tasks.ceph.mon.c:Stopped 2020-11-16T22:38:08.874 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-16T22:38:08.874 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:08.893 INFO:tasks.ceph.mon.b:Stopped 2020-11-16T22:38:08.894 INFO:tasks.ceph:Checking cluster log for badness... 2020-11-16T22:38:08.894 INFO:teuthology.orchestra.run.smithi181:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Scrub error on inode' | head -n 1 2020-11-16T22:38:08.934 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:38:08.934 INFO:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-11-16T22:38:09.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:38:09.234 INFO:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-11-16T22:38:09.422 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:38:09.422 INFO:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-11-16T22:38:09.566 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi181.front.sepia.ceph.com 2020-11-16T22:38:09.566 INFO:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2020-11-16T22:38:09.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:38:09.726 INFO:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2020-11-16T22:38:10.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:38:10.062 INFO:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2020-11-16T22:38:10.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:38:10.238 INFO:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2020-11-16T22:38:10.385 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2020-11-16T22:38:10.386 INFO:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2020-11-16T22:38:10.533 INFO:tasks.ceph:Archiving mon data... 2020-11-16T22:38:10.534 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/data/mon.a.tgz 2020-11-16T22:38:10.535 INFO:teuthology.orchestra.run.smithi181:> mktemp 2020-11-16T22:38:10.541 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.J2PrELBnYg 2020-11-16T22:38:10.542 INFO:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.J2PrELBnYg 2020-11-16T22:38:10.818 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.J2PrELBnYg 2020-11-16T22:38:10.874 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.J2PrELBnYg is 1017KB 2020-11-16T22:38:17.666 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.J2PrELBnYg 2020-11-16T22:38:17.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2020-11-16T22:38:17.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2020-11-16T22:38:17.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.g has been restored 2020-11-16T22:38:17.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.h has been restored 2020-11-16T22:38:17.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.i has been restored 2020-11-16T22:38:17.674 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/data/mon.c.tgz 2020-11-16T22:38:17.674 INFO:teuthology.orchestra.run.smithi181:> mktemp 2020-11-16T22:38:17.720 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.dA0NqlygOP 2020-11-16T22:38:17.720 INFO:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.dA0NqlygOP 2020-11-16T22:38:18.006 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.dA0NqlygOP 2020-11-16T22:38:18.070 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.dA0NqlygOP is 1018KB 2020-11-16T22:38:45.478 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.dA0NqlygOP 2020-11-16T22:38:45.683 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:45.690 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/data/mon.b.tgz 2020-11-16T22:38:45.690 INFO:teuthology.orchestra.run.smithi175:> mktemp 2020-11-16T22:38:45.691 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:45.694 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:45.697 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.iwUGff5doy 2020-11-16T22:38:45.698 INFO:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.iwUGff5doy 2020-11-16T22:38:46.013 INFO:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.iwUGff5doy 2020-11-16T22:38:46.078 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.iwUGff5doy is 1004KB 2020-11-16T22:38:53.898 INFO:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.iwUGff5doy 2020-11-16T22:38:53.905 INFO:tasks.ceph:Cleaning ceph cluster... 2020-11-16T22:38:53.906 INFO:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:38:53.912 INFO:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:38:53.948 INFO:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:38:53.963 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-11-16T22:38:53.964 INFO:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:38:54.008 INFO:teuthology.orchestra.run.smithi175:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:38:54.010 INFO:teuthology.orchestra.run.smithi106:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:38:54.041 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:38:54.055 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:38:54.280 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:38:54.288 INFO:tasks.ceph:Archiving crash dumps... 2020-11-16T22:38:54.289 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/remote/smithi181/crash 2020-11-16T22:38:54.290 INFO:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:38:54.309 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/remote/smithi175/crash 2020-11-16T22:38:54.311 INFO:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:38:54.330 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/remote/smithi106/crash 2020-11-16T22:38:54.331 INFO:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:38:54.372 INFO:tasks.ceph:Shutting down logrotate 2020-11-16T22:38:54.372 INFO:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.377 INFO:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.380 INFO:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.435 INFO:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.462 INFO:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.476 INFO:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-16T22:38:54.491 INFO:tasks.ceph:Compressing logs... 2020-11-16T22:38:54.491 INFO:teuthology.orchestra.run.smithi106:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:38:54.497 INFO:teuthology.orchestra.run.smithi175:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:38:54.520 INFO:teuthology.orchestra.run.smithi181:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:38:59.785 INFO:tasks.ceph:Archiving logs... 2020-11-16T22:38:59.786 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631916/remote/smithi181/log 2020-11-16T22:38:59.787 INFO:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/log/ceph -- .